Learning Outcomes
amiliarization with the topics of information theory and coding, correlation of information theory objectives with current practical communication and coding examples, training on theoretical channel analysis, training on basic coding practices
Course Content (Syllabus)
Information – entropy (entropy, joint entropy, conditional entropy, memory and memoryless information sources), channel capacity (information channels, mutual information and capacity, lossless channels, uniform information channel, binary symmetric channel, binary erasure channel), coding in noiseless communications (Kraft inequality, Shannon’s theorem, typical codes (Shannon, Shannon-Fano, Huffmann, binary coding trees). Coding in noisy communications (decoding criteria, Fano’s limit, Shannon’s second theorem, Hamming code). Arithmetic coding, algebraic coding.
Keywords
information, entropy, channel, capacity, code