Multi-Sensory Interaction and Media Cognition

2023/2024

Content, progress and pedagogy of the module

Objectives

In interactive-immersive systems that rely on digital technology, human interactivity and responsiveness are directly linked to the processes of human perception and cognition.

This course introduces current research trends and emerging paradigms on the relation between digital technologies and multi-modal perception and cognition. Particular emphasis is put on multi-modal perception processes that are usually involved in interactive digital media (e.g., visual, auditory, haptic, proprioception) and higher cognitive processes related to interactivity (e.g. multimodal integration, enaction, intelligibility, cognitive closure, affective states and emotions, spatial cognition and navigation).

The course draws relevant knowledge from a variety of disciplines and fields such as cognitive neuroscience, ecological psychology, biology, cognitive ergonomics and cognitive technologies. Various bio-behavioral and biofeedback methods for interaction design and assessment are also introduced (e.g. EEG, EMG, ECG, galvanic skin response, ocular measures) and new trends in integration of interactive digital technologies with cognitive processes are addressed (e.g. multi-modal interfaces and set-ups, brain-computer-interfaces, enactive interfaces).

Learning objectives

Knowledge

Students who complete the module will obtain:

  • understanding of the main paradigms, concepts and disciplines that contribute to multimodal perception research and cognition studies and which have relevance for the interaction of human subjects with immersive-interactive systems
  • knowledge about the potentialities and limits that the human “perceptual apparatus” and the cognitive system present for the technology designer
  • understanding of the relations between multimodal perception, higher cognitive functions, affective states and action

Skills

Students who complete the module will be able to: 

  • apply knowledge on human multimodal perception and cognition in the design of interactive digital systems
  • design perception and cognition experiments related to the cross-modal action of two or more senses
  • apply biofeedback and bio-behavioral measurements in experimental designs

Competences

Students who complete the module will be able to:

  • synthesize knowledge and theoretical frameworks from a variety of relevant sources and disciplines, which contribute to the study of technology-cognition interaction and apply this knowledge in the design of multimodal interactive systems
  • analyze and interpret experimental work and literature in the field

Type of instruction

Refer to the overview of instruction types listed in § 17.

Exam

Prerequisite for enrollment for the exam

  • To be eligible to take the exam, the student must timely have handed in any mandatory assignments

Exams

Name of examMulti-Sensory Interaction and Media Cognition
Type of exam
Oral exam based on a project
ECTS5
Permitted aids
With certain aids:
See semester description
Assessment7-point grading scale
Type of gradingInternal examination
Criteria of assessmentThe criteria of assessment are stated in the Examination Policies and Procedures

Facts about the module

Danish titleMulti-sensorisk interaktion og mediecognition
Module codeMSNMEDM2222
Module typeCourse
Duration1 semester
SemesterSpring
ECTS5
Language of instructionEnglish
Location of the lectureCampus Aalborg, Campus Copenhagen
Responsible for the module

Organisation

Study BoardStudy Board of Media Technology
DepartmentDepartment of Architecture, Design and Media Technology
FacultyThe Technical Faculty of IT and Design