Recommended prerequisite for participation in the module

The module builds on knowledge obtained by the modules Probability Theory and Linear Algebra on the Bachelor of Science (BSc) in Engineering (Mathematical Engineering).

Content, progress and pedagogy of the module

Learning objectives

Knowledge

• knowledge of information theoretical concepts such as entropy, mutual information, divergence, the chain rule for entropy, empirical entropy
• knowledge of lossless data compression, entropy coding, lossy data compression (rate distortion theory)
• knowledge of channel capacity and error-correcting codes
• knowledge of joint source-channel coding and the separation principle

Skills

• are able to give a theoretical description of the entropy of a signal and in practice estimate the entropy of simple signals
• are able to design efficient entropy codes for simple signals
• are able to use information inequalities to provide bounds on optimal performance of simple systems
• arte able to construct error-correcting codes with good properties and parameters
• are able to decode error-correcting codes efficiently (e.g. Reed-Solomon codes)
• understand the interaction between bitrate and distortion (reconstruction error) in connection with source coding
• understand the interaction between bitrate and error probability in connection with channel coding
• are able to perform calculations in finite fields

Competences

• have a good intuition and understanding of the concept of entropy and its significance regarding the information within a signal
• be able to use mathematical tools to discover and investigate the fundamental mathematical tools that describes data transmission, data reduction and data storage

Type of instruction

Lectures with exercises.

Extent and expected workload

This is a 5 ECTS course module and the work load is expected to be 150 hours for the student.
 

Exam

Exams

Name of exam	Information and Coding Theory
Type of exam	Written or oral exam
ECTS	5
Assessment	7-point grading scale
Type of grading	Internal examination
Criteria of assessment	The criteria of assessment are stated in the Examination Policies and Procedures