Recommended prerequisite for participation in the module

Content, progress and pedagogy of the module

Learning objectives

Knowledge

• knowledge of information theoretical concepts such as entropy, mutual information,
• divergence, the chain rule for entropy, empirical entropy
• knowledge of lossless data compression, entropy coding, lossy data compression (rate
• distortion theory)
• knowledge of channel capacity and error-correcting codes
• knowledge of joint source-channel coding and the separation principle

Skills

• are able to give a theoretical description of the entropy of a signal and in practice estimate the entropy of simple signals
• are able to design efficient entropy codes for simple signals
• are able to use information inequalities to provide bounds on optimal performance of simple systems
• are able to construct error-correcting codes with good properties and parameters
• are able to decode error-correcting codes efficiently (e.g. Reed-Solomon codes)
• understand the interaction between bitrate and distortion (reconstruction error) in connection with source coding
• understand the interaction between bitrate and error probability in connection with channel coding
• are able to perform calculations in finite fields

Competences

• have a good intuition and understanding of the concept of entropy and its significance
• regarding the information within a signal
• be able to use mathematical tools to discover and investigate the fundamental
• mathematical tools that describes data transmission, data reduction and data storage

Type of instruction

Types of instruction are listed in §17; Structure and contents of the programme.

Exam

Exams