Project Manager: Jana Holsanova

The project is linked to our previous CCL studies on Mental imagery and the Interplay between language and pictures in communication.

The goal of this project is to systematically investigate the process of simultaneous verbal description of visual scenes in the framework of cognitive science theories of visual perception and attention, multimodality, timing, embodied cognition and mental imagery.

Host environment for the project is the Humanities Lab, in particular the emerging LARM studio where top equipment will make it possible to work with sound, animation, moving images and music.

To perceive and experience visual scenes, images, gestures, facial expressions, events and movements on stage during film and theatre performances is something that the sighted audience takes for granted. Visually impaired audience can, however, miss important information that is only visible but not articulated through language or sound.

The task of the interpreter is to evoke vivid mental images by simultaneously describing visual scenes verbally for people without sight. That means that the interpreter (under time pressure) has to select relevant non-verbal information from the visual scene, linearize it, articulate it linguistically in an efficient manner to evoke such images and to time this piece of information so that it matches with what at the moment happens linguistically (ie what is said in the dialogues and what is heard in the sounds). This timing is crucial for the integration of the different modalities, for meaning–making and for achieving of the intended communicative effect.