Content, progress and pedagogy of the module

Objectives:

In interactive-immersive systems that rely on digital technology, human interactivity and responsiveness are directly linked to the processes of human perception and cognition.

This course introduces current research trends and emerging paradigms on the relation between digital technologies and multi-modal perception and cognition. Particular emphasis is put on multi-modal perception processes that are usually involved in interactive digital media (e.g., visual, auditory, haptic, proprioception) and higher cognitive processes related to interactivity (e.g. multimodal integration, enaction, intelligibility, cognitive closure, affective states and emotions, spatial cognition and navigation).

The course draws relevant knowledge from a variety of disciplines and fields such as cognitive neuroscience, ecological psychology, biology, cognitive ergonomics and cognitive technologies. Different bio-behavioral and biofeedback methods for interaction design and assessment are also introduced (e.g. EEG, EMG, ECG, galvanic skin response, ocular measures) and new trends in integration of interactive digital technologies with cognitive processes are addressed (e.g. multi-modal interfaces and set-ups, brain-computer-interfaces, enactive interfaces). Finally, the course provides the opportunity for targeting the knowledge provided towards the specialisation profile chosen by the student (Computer graphics, Interaction, Games).

Learning objectives

Knowledge

Students who complete the module will obtain the following qualifications:

• Understanding of the main paradigms, concepts and disciplines that contribute to multimodal perception research and cognition studies and which have relevance for the interaction of human subjects with immersive-interactive systems
• Knowledge about the potentialities and limits that the human “perceptual apparatus” and the cognitive system present for the technology designer
• Understanding of the relations between multimodal perception, higher cognitive functions, affective states and action

Skills

Students who complete the module will obtain the following qualifications:

• Ability to apply knowledge on human multimodal perception and cognition in the design of interactive digital systems
• Ability to apply knowledge to the design perception and cognition tests related to the cross-modal action of two or more senses
• Be able to apply biofeedback and bio-behavioral measurements in experimental designs

Competences

Students who complete the module will obtain the following qualifications:

• Ability to synthesize knowledge and theoretical frameworks from a variety of relevant sources and disciplines, which contribute to the study of technology-cognition interaction
• Be able to synthesize such knowledge in the design of multimodal interactive systems
• Ability to analyse and interpret experimental work and literature in the field

Type of instruction

Refer to the overview of instruction types listed in § 17.

Exam

Exams