Visual metaphors and Cognition
Visual metaphors are visual counterparts of verbal metaphors such as "man is a wolf" and "sky is crying". In a visual metaphor something the target, that is presented pictorially, is compared to something that belongs to another category the source, which is also presented pictorially. Visual metaphors are often used in advertising, political cartoons, films, computer design. Figure 1 provides an example of a visual metaphor in advertisement, which shows people sitting in an airport. All of them but one has loudspeakers for their heads. The image conveys the noisy environment of airports and suggests that people talk so loudly in airports as if “their heads were loudspeakers”. Images that can be interpreted like this are called visual metaphors.
Figure 1. An example of visual metaphor
In my research I explore following questions:
-
What is a visual metaphor?
-
What is the difference between verbal and visual metaphor processing?
-
What is the role of various cognitive processes such as language, imagery, perceptual, etc. in the comprehension of visual metaphors?
-
How visual metaphors are persuasive?
Research Findings:
-
Visual metaphors are different from verbal metaphors as the processing of verbal and visual stimuli is fundamentally different.
-
fMRI studies showed that language plays an important role in visual metaphor comprehension suggesting the multimodal nature of metaphorical processing. (Figure 2)
-
Eye movement studies show that low level perceptual similarity at the level of colour, shape, etc. is subconsciously noticed (Figure 3) and helps in generating creative metaphorical interpretations.
-
Visual metaphors are tough to interpret and require allocation of higher cognitive resources. But this cognitive effort leads to greater persuasion.
-
A Cognitive model of visual metaphor processing has been proposed (Figure 5).
Research Methods:
-
Behavioural experiments including "Response time", "feature generation", etc.
-
Eye movement
-
Pupillometry studies (Change in pupil size and eye blink rate)
-
fMRI
-
EEG (Coherence analysis).
Figure 2. Brain activation elicited for (A) Literal Verbal, (B) Metaphor Verbal, (C) Literal Visual and (D) Metaphor Visual conditions in an fMRI study.
Figure 3A. Perceptually similar pair of (decided by an algorithm) images are more likely to be interpreted metaphorically than perceptually non-similar pair of images.
Figure 3B. Difference in saccades and fixation on perceptually high similar and perceptually low similar pairs. Perceptual similarity is noticed and helps in metaphorical interpretation.
Figure 4. Proposed cognitive model for visual metaphor processing.
Related Publications
Ojha, A., Gola, E., & Indurkhya, B. (to appear). Are pictorial similes perceived stronger than hybrid pictorial metaphors. Metaphor and Symbol 33(4).
Ojha, A., Gola, E., Ervas, F., Indurkhya, B. (to appear), Creativity and difference between verbal and visual metaphor processing: An EEG coherence study. Frontiers in Psychology, section Cognitive Science.