Umar, Hafidah (2020). An Experimental Investigation of Modality Effect: Evidence from Eye-Tracking Data. (Thesis). Universität Bern, Bern

Preview

Text 20binti_umar_h.pdf - Thesis Available under License Creative Commons: Attribution-Noncommercial-No Derivative Works (CC-BY-NC-ND 4.0). Download (2MB) | Preview

Abstract

Modality effect is a response that occur when there are manipulations of sensory modality. In this thesis, I present a series of studies about the multimodal processing of visual and auditory presentation. The aim of this dissertation is to investigate how would the different stimulations from different source of modalities affect the oculomotor response. I investigate how different stimuli are processed, recognized and retrieved when they are presented across multiple modalities. Specifically, on question of how would the visual and auditory manipulations influence the oculomotor behaviour. In the research area of the multimodal processing, it has been argued that different kinds of sensory manipulations elicit a distinct kind of cognitive and behavioural response. The study of modality effect is particularly interesting topic for investigations since the world is multimodal in nature. Humans and other living beings are constantly exposed to a wide variety of stimuli rather than to isolated single stimulus. All experiments conducted used an eye-tracking approach since eye-tracking data are known as a reliable measure to study implicit cognitive processing. In Experiment 1, I investigate how different modalities and context interplay on the allocation of visual attention during the perceptual processing of congruent and incongruent multimodal stimuli. In Experiment 2, I investigate recognition memory of multimodal stimuli, focusing on the participants’ reaction to old versus novel stimuli presented in the visual and auditory modalities. In Experiment 3, I monitored looking patterns, to understand how visual and auditory stimuli are mentally reconstructed during mental imagery. I conclude the dissertation with a discussion of how a different kinds of modality manipulations elicit distinct modality effect as revealed by oculomotor response.

Item Type:	Thesis
Dissertation Type:	Single
Date of Defense:	20 January 2020
Subjects:	100 Philosophy > 150 Psychology
Institute / Center:	07 Faculty of Human Sciences > Institute of Psychology
Depositing User:	Hammer Igor
Date Deposited:	30 Jan 2020 15:24
Last Modified:	14 Feb 2020 07:07
URI:	https://boristheses.unibe.ch/id/eprint/1774

Actions (login required)

View Item