Knowledge: Knowing and understanding the basic principles and techniques if information theory, understanding coding techniques, knowing how to calculate the channel capacity.
Skills: Identify the appropriate coding scheme and optimize channel capacity. Solve exercises, present in front of an audience, search the corresponding literature.
Course Content (Syllabus)
Information – entropy (entropy, joint entropy, conditional entropy, memory and memoryless information sources), channel capacity (information channels, mutual information and capacity, lossless channels, uniform information channel, binary symmetric channel, binary erasure channel), coding in noiseless communications (Kraft inequality, Shannon’s theorem, typical codes (Shannon, Shannon-Fano, Huffmann, binary coding trees). Coding in noisy communications (decoding criteria, Fano’s limit, Shannon’s second theorem, Hamming code). Arithmetic coding, algebraic coding.
information, entropy, channel, capacity, code