Multimodal Perception and Cognition


Content, progress and pedagogy of the module


In interactive-immersive systems that rely on digital technology, human interactivity and responsiveness are directly linked to the processes of human perception and cognition.

This course introduces current research trends and emerging paradigms on the relation between digital technologies and multi-modal perception and cognition. Particular emphasis is put on multi-modal perception processes that are usually involved in interactive digital media (e.g., visual, auditory, haptic, proprioception) and higher cognitive processes related to interactivity (e.g. multimodal integration, enaction, intelligibility, cognitive closure, affective states and emotions, spatial cognition and navigation).

The course draws relevant knowledge from a variety of disciplines and fields such as cognitive neuroscience, ecological psychology, biology, cognitive ergonomics and cognitive technologies. Different bio-behavioral and biofeedback methods for interaction design and assessment are also introduced (e.g. EEG, EMG, ECG, galvanic skin response, ocular measures) and new trends in integration of interactive digital technologies with cognitive processes are addressed (e.g. multi-modal interfaces and set-ups, brain-computer-interfaces, enactive interfaces). Finally, the course provides the opportunity for targeting the knowledge provided towards the specialisation profile chosen by the student (Computer graphics, Interaction, Games).

Learning objectives


Students who complete the module will obtain the following qualifications:

  • Understanding of the main paradigms, concepts and disciplines that contribute to multimodal perception research and cognition studies and which have relevance for the interaction of human subjects with immersive-interactive systems
  • Knowledge about the potentialities and limits that the human “perceptual apparatus” and the cognitive system present for the technology designer
  • Understanding of the relations between multimodal perception, higher cognitive functions, affective states and action


Students who complete the module will obtain the following qualifications:

  • Ability to apply knowledge on human multimodal perception and cognition in the design of interactive digital systems
  • Ability to apply knowledge to the design perception and cognition tests related to the cross-modal action of two or more senses
  • Be able to apply biofeedback and bio-behavioral measurements in experimental designs



Students who complete the module will obtain the following qualifications:

  • Ability to synthesize knowledge and theoretical frameworks from a variety of relevant sources and disciplines, which contribute to the study of technology-cognition interaction
  • Be able to synthesize such knowledge in the design of multimodal interactive systems
  • Ability to analyse and interpret experimental work and literature in the field

Type of instruction

Refer to the overview of instruction types listed in § 17.



Name of examMultimodal Perception and Cognition
Type of exam
Written or oral exam
Permitted aids
With certain aids:
See semester description
Assessment7-point grading scale
Type of gradingInternal examination
Criteria of assessmentThe criteria of assessment are stated in the Examination Policies and Procedures

Facts about the module

Danish titleMultimodal perception og kognition
Module codeMSNMEDM1176
Module typeCourse
Duration1 semester
Language of instructionEnglish
Location of the lectureCampus Aalborg, Campus Copenhagen, Campus Esbjerg
Responsible for the module


Study BoardStudy Board of Media Technology
DepartmentDepartment of Architecture, Design and Media Technology
FacultyThe Technical Faculty of IT and Design