Information and Coding Theory


Prerequisite/Recommended prerequisite for participation in the module

The module builds upon knowledge obtained the modules "Probability Theory" and "Linear Algebra".

Content, progress and pedagogy of the module

Learning objectives


  • knowledge of information theoretical concepts such as entropy, mutual information,
  • divergence, the chain rule for entropy, empirical entropy
  • knowledge of lossless data compression, entropy coding, lossy data compression (rate
  • distortion theory)
  • knowledge of channel capacity and error-correcting codes
  • knowledge of joint source-channel coding and the separation principle


  • are able to give a theoretical description of the entropy of a signal and in practice estimate the entropy of simple signals
  • are able to design efficient entropy codes for simple signals
  • are able to use information inequalities to provide bounds on optimal performance of simple systems
  • are able to construct error-correcting codes with good properties and parameters
  • are able to decode error-correcting codes efficiently (e.g. Reed-Solomon codes)
  • understand the interaction between bitrate and distortion (reconstruction error) in connection with source coding
  • understand the interaction between bitrate and error probability in connection with channel coding
  • are able to perform calculations in finite fields


  • have a good intuition and understanding of the concept of entropy and its significance
  • regarding the information within a signal
  • be able to use mathematical tools to discover and investigate the fundamental
  • mathematical tools that describes data transmission, data reduction and data storage

Type of instruction

Types of instruction are listed in §17; Structure and contents of the programme.



Name of examInformation and Coding Theory
Type of exam
Written or oral exam
AssessmentPassed/Not Passed
Type of gradingInternal examination
Criteria of assessmentThe criteria of assessment are stated in the Examination Policies and Procedures

Facts about the module

Danish titleInformation og kodningsteori
Module codeESNCTK3K2
Module typeCourse
Duration1 semester
Language of instructionEnglish
Location of the lectureCampus Aalborg
Responsible for the module


Study BoardStudy Board of Electronics and IT
DepartmentDepartment of Electronic Systems
FacultyTechnical Faculty of IT and Design