Multimodal Perception and Cognition

2019/2020

Content, progress and pedagogy of the module

Objectives:

In interactive-immersive systems that rely on digital technology, human interactivity and responsiveness are directly linked to the processes of human perception and cognition.

This course introduces current research trends and emerging paradigms on the relation between digital technologies and multi-modal perception and cognition. Particular emphasis is put on multi-modal perception processes that are usually involved in interactive digital media (e.g., visual, auditory, haptic, proprioception) and higher cognitive processes related to interactivity (e.g. multimodal integration, enaction, intelligibility, cognitive closure, affective states and emotions, spatial cognition and navigation).

The course draws relevant knowledge from a variety of disciplines and fields such as cognitive neuroscience, ecological psychology, biology, cognitive ergonomics and cognitive technologies. Different bio-behavioral and biofeedback methods for interaction design and assessment are also introduced (e.g. EEG, EMG, ECG, galvanic skin response, ocular measures) and new trends in integration of interactive digital technologies with cognitive processes are addressed (e.g. multi-modal interfaces and set-ups, brain-computer-interfaces, enactive interfaces). Finally, the course provides the opportunity for targeting the knowledge provided towards the specialisation profile chosen by the student (Computer graphics, Interaction, Games).

Learning objectives

Knowledge

Students who complete the module will obtain the following qualifications:

  • Understanding of the main paradigms, concepts and disciplines that contribute to multimodal perception research and cognition studies and which have relevance for the interaction of human subjects with immersive-interactive systems
  • Knowledge about the potentialities and limits that the human “perceptual apparatus” and the cognitive system present for the technology designer
  • Understanding of the relations between multimodal perception, higher cognitive functions, affective states and action

Skills

Students who complete the module will obtain the following qualifications:

  • Ability to apply knowledge on human multimodal perception and cognition in the design of interactive digital systems
  • Ability to apply knowledge to the design perception and cognition tests related to the cross-modal action of two or more senses
  • Be able to apply biofeedback and bio-behavioral measurements in experimental designs

 

Competences

Students who complete the module will obtain the following qualifications:

  • Ability to synthesize knowledge and theoretical frameworks from a variety of relevant sources and disciplines, which contribute to the study of technology-cognition interaction
  • Be able to synthesize such knowledge in the design of multimodal interactive systems
  • Ability to analyse and interpret experimental work and literature in the field

Type of instruction

Refer to the overview of instruction types listed in the start of chapter 3. The types of instruction for this course are decided in accordance with the current Joint Programme Regulations and directions are decided and given by the Study Board for Media Technology.

Exam

Exams

Name of examMultimodal Perception and Cognition
Type of exam
Written or oral exam
In accordance with the current Joint Programme Regulations and directions on examination from the Study Board for Media Technology:
Oral or written examination with internal censor. The assessment is performed in accordance with the 7-point grading scale.
ECTS5
Permitted aids
With certain aids:
See semester description
Assessment7-point grading scale
Type of gradingInternal examination
Criteria of assessmentThe criteria of assessment are stated in the Examination Policies and Procedures

Facts about the module

Danish titleMultimodal perception og kognition
Module codeMSNMEDM1176
Module typeCourse
Duration1 semester
SemesterAutumn
ECTS5
Language of instructionEnglish
Location of the lectureCampus Aalborg, Campus Copenhagen, Campus Esbjerg
Responsible for the module

Organisation

Study BoardStudy Board of Media Technology
DepartmentDepartment of Architecture, Design and Media Technology
FacultyTechnical Faculty of IT and Design