"Perception Through Action" by Vittorio Gallese, Laila Craighero, Luciano Fadiga and Leonardo Fogassi

	Milner and Goodale's book The Visual Brain in Action may be purchased from Amazon.Com

Perception Through Action

Vittorio Gallese<1>, Laila Craighero, Luciano Fadiga & Leonardo Fogassi
Istituto di Fisiologia Umana
Universita' di Parma
Via Volturno, 39
43100 Parma
ITALY

gallese@unipr.it

PSYCHE, 5(21), July 1999
http://psyche.cs.monash.edu.au/v5/psyche-5-21-gallese.html

KEYWORDS: monkey; premotor cortex; grasping; mirror neurons; action; perception.

COMMENTARY ON: A. David Milner & Melvyn A. Goodale. (1995) The Visual Brain in Action. (Oxford Psychology Series, No. 27). Oxford: Oxford University Press. xvii + 248pp. ISBN: 0198524080. Price: $35 pbk.

ABSTRACT: The Visual Brain in Action by Milner and Goodale provides a new conceptual account of how the brain processes visual information. Milner and Goodale (1995) make two major points: (1) The dorsal stream processes visual information for motor purposes; (2) Action and perception are two completely separate domains, the latter being an exclusive property of the ventral stream. In the first part of this review we will summarize some recent neurophysiological data shedding new light on the "pragmatic" role of the visual information processed in the dorsal stream, and thus corroborating the theoretical views of Milner and Goodale (1995). In the second part we will discuss some recent neurophysiological, neuropsychological and brain imaging studies suggesting that the dichotomy proposed by Milner and Goodale between action and perception is probably too rigid.

1. Introduction

The Visual Brain in Action (Oxford University Press, 1995) by Milner and Goodale is an outstanding contribution that reshapes the concept of how the brain processes visual information. Since the early eighties, the dominant view on the cortical processing of visual information has been the "what" and "where" theory, as formalized by Ungerleider and Mishkin (1982). According to these authors, the ventral stream has its main role in object recognition, while the dorsal stream analyzes an object's spatial location. This point of view was in accordance with the classical notion of the parietal cortex as the site for unitary space perception.

Since 1991, Milner and Goodale have argued against this theory, emphasizing the pragmatic role of the dorsal stream. This point of view, primarily triggered by clinical data, has been subsequently substantiated by neurophysiological evidence. The posterior parietal cortex, as pointed out also by Milner and Goodale (1995), is now thought to consist of a mosaic of areas, each receiving specific sensory information. These different types of sensory information are transformed into information appropriate for action. It is now clear that there is no evidence of a unique supramodal "space area" within the posterior parietal cortex. Space perception appears to derive from the joint activity of a series of sensorimotor fronto-parietal circuits, each of which, according to its own motor purposes, encodes the spatial location of an object and transforms it into a potential action (see Rizzolatti et al. 1997a, b).

The idea of a motor role for the posterior parietal cortex is by no means new. Since the pioneering studies carried out by Hyvarinen, Mountcastle and their coworkers (Hyvarinen & Poranen, 1974; Mountcastle et al., 1975) it is well known that different sectors of the posterior parietal cortex are involved in the control of arm, hand and eye movements. However, this "motor" role was somehow underestimated in light of a purely "spatial" characterization of the visual information reaching these sectors of the parietal cortex.

Milner and Goodale (1995) make two major points: 1) The dorsal stream processes visual information for motor purposes; 2) Action and perception are two completely separate domains, the latter being an exclusive property of the ventral stream. In the first part of this review we will summarize some recent neurophysiological data shedding new light on the "pragmatic" role of the visual information processed in the dorsal stream, and thus corroborating the theoretical views of Milner and Goodale (1995). In the second part we will discuss some recent neurophysiological, neuropsychological and brain imaging studies suggesting that the dichotomy proposed by Milner and Goodale between action and perception is probably too rigid.

2. The "What" and "How" of the Dorsal Stream

Two recently discovered fronto-parietal circuits connecting area F4 to the ventral intraparietal area (VIP) and area F5 to the anterior intraparietal area (AIP) appear to be endowed with the functional characteristics suitable to guide arm reaching and hand grasping movements (Rizzolatti et al., 1997a, 1998). Both areas VIP and F4 contain bimodal neurons responding to tactile stimuli applied to specific cutaneous sectors (tactile receptive fields, RFs) and to 3D visual stimuli presented in specific space sectors (visual RFs). The visual RFs of these neurons are anchored to the tactile ones regardless of eye position. These neurons seem to play a crucial role in the process of visuo-motor transformation necessary to guide arm or head movements toward or away from visual stimuli. Similarly, in areas AIP and F5 there are neurons responding both to the visual presentation of 3D objects of different size and shape and to the grasping movements aimed at those same objects.

Summing up, within the dorsal stream, there are parallel cortico-cortical circuits, each of which elaborates a specific type of visual information in order to guide different types of action. The peculiarity of these circuits resides in the fact that different effectors are provided with the most suitable type of visual information required by their motor repertoire. This firm connection between vision and action seems to be the organizing principle within the circuitry connecting the parietal with the agranular frontal cortex of the monkey. Among the aforementioned parallel fronto-parietal circuits, we will discuss in some detail the AIP-F5 circuit, since we think it represents a good example of the "pragmatic" representations built in the dorsal stream, as maintained by Milner and Goodale (1995).

Area AIP (Sakata et al., 1995), which is located in the lateral bank of the intraparietal sulcus, contains three main classes of neurons related to hand-manipulation: 1) Motor dominant neurons, that discharge during grasping movements performed both with and without visual control; 2) Visuomotor neurons, whose discharge is higher when grasping is executed under visual control; 3) Visual dominant neurons, that are active only during grasping executed under visual control. Among the neurons of the last two classes, there are some (Object type neurons) that respond also to the simple visual presentation of 3D objects, in the absence of any grasping movement aimed at them. Where do Object type neurons get their visual input from? Sakata and his collaborators recently discovered neurons that respond selectively to the surface orientation of 3D objects. These neurons, located in a region (cIPS area, see Shikata et al., 1996) in the caudal part of the lateral bank of IPS, could be one of the main sources of visual information for AIP neurons. The importance of this discovery resides in the fact that for the first time neurons coding 3D object shape have been described in the dorsal stream.

The main target of area AIP is area F5 (see Rizzolatti et al., 1997a) which occupies the most rostral portion of the ventral premotor cortex. Neurons in F5 code goal-directed motor acts such as grasping, holding, manipulating objects. The role of these neurons has been conceptualized as a "motor vocabulary" of actions related to prehension (Rizzolatti et al., 1988; Rizzolatti & Gentilucci, 1988). Among grasping neurons many specify also how the action must be performed, being selectively activated by different types of prehension. How is this motor vocabulary addressed by external stimuli? A considerable percentage of F5 grasping neurons fire also during observation of specific 3D objects, even in absence of any movement directed toward them (Murata et al., 1997). The apparent paradox of a population of premotor neurons acting as visual neurons in some conditions and as motor in others can be overcome if one postulates that the "visual" and "motor" responses of F5 neurons have the same meaning. This common meaning is the representation of an action, that can be triggered either by the presence of the object or by the memory of it, as it occurs when grasping is executed without visual control. This representation is likely built by means of the interplay between AIP and F5. We have proposed (Gallese et al., 1997; Fogassi & Gallese, in press) that area AIP transforms visual information of a given 3D object into multiple descriptions, thus providing F5 with several "grasping possibilities". Area F5 would then select, on the basis of contextual information, the most suitable type of prehension. The result of this selection would be fed back to AIP, keeping active that set of AIP neurons also coding the same selected type of grip. From our point of view, these data provide the neurophysiological basis to the pragmatic role that Milner and Goodale advocate for the dorsal stream.

3. When Action Meets Perception

A strong point made by Milner and Goodale (1995) maintains that in the primates' visual system there is a sharp distinction between the role played by the dorsal and the ventral stream of visual processing: the dorsal stream would be mainly involved in the on-line control of actions, while the ventral stream would be the exclusive source of information for perception and semantics. To support this notion Milner and Goodale (1995) stress the dissociation between the behaviour of the agnosic patient D.F. and that of the ataxic patients like A.T. The former "...seems to be using a visual processing system dedicated for motor control which can only come into play when she carries out natural goal-directed actions", while the latter, despite the failure in reaching and grasping movements, "...have comparatively little difficulty in giving perceptual reports of the orientation and location of the very objects they fail to grasp". Although Milner and Goodale (1995) acknowledge that the adaptive behaviour of primates, humans included, relies on the integration of both visual streams, it seems to us that the action/perception dichotomy they advocate is perhaps too rigid.

Several lines of evidence seem to point to an important involvement of the motor system in supporting processes traditionally considered to be "high level" or cognitive, such as action understanding, mental imagery of actions, perceiving and discriminating objects. A first example is provided by the discovery of a population of neurons in the monkey ventral premotor cortex (mirror neurons) that discharge both when the monkey performs a grasping action and when it observes the same action performed by other individuals (Gallese et al., 1996; Rizzolatti et al., 1996a). Mirror neurons would provide the neurophysiological basis for the capacity of primates to recognize different actions made by other individuals: the same motor pattern which characterizes the observed action is evoked in the observer and activates its own motor repertoire. This matching mechanism, which can be framed within the motor theories of perception, offers the great advantage of using a repertoire of coded actions in two ways at the same time: at the output side to act, and at the input side, to analyse the visual percept. Recently, this matching system has also been demonstrated in humans. Transcranial Magnetic Stimulation (TMS) of the motor cortex of subjects observing hand actions made by the experimenter determined an enhancement of motor evoked potentials (MEPs) in the same muscular groups that were used by the experimenter in executing those actions (Fadiga et al., 1995). This means that when we observe an action we utilize, as monkeys do, the repertoire of motor representations used to produce the same action.

Letting aside the problem of the anatomical localization of the mirror system in humans (two recent PET studies have shown an activation of Broca's region during hand action observation, Rizzolatti et al., 1996b; Grafton et al., 1996), we will now concentrate on another aspect of the involvement of the dorsal stream in cognitive functions: motor imagery. Imagining a grasping action is a cognitive task that requires a conscious, detailed representation of the movement. Several recent PET studies have shown that during motor imagery of grasping actions premotor and inferior parietal areas are strongly activated (Decety et al., 1994; Grafton et al., 1996). Furthermore, Parsons et al. (1995) demonstrated in a PET study that motor imagery used for visual hand shape discrimination activates premotor and posterior parietal cortex. Further evidence supporting the notion of the involvement of the dorsal stream in cognitive tasks is provided by an elegant neuropsychological study by Sirigu et al. (1995). Patients with lesions restricted to the posterior parietal cortex were selectively impaired at predicting through mental imagery the time necessary to perform differentiated finger movements. Taken together, all these results seem to contradict a sharp distinction between an "acting brain" and a "knowing brain".

The idea of a major role played by action in building our perception is not new. Roger Sperry in 1952 wrote:

Perception is basically an implicit preparation to respond. Its function is to prepare the organism for adaptive action. The problem of what occurs in the brain during perception can be attacked much more effectively once this basic principle is recognized.

These sentences stress the fact that perception can be accounted for only by considering the bidirectional relationship between the agent and his/her environment. A recent neuropsychological study (Craighero et al., in press) demonstrates that preparation of a grasping movement affects detection and discrimination of a visual stimulus. In this experiment normal subjects were required to grasp a bar after the presentation of a visual stimulus whose orientation was either congruent or incongruent with that of the bar. The results showed that grasping preparation enhances the detection of a visual stimulus whose intrinsic properties are congruent with those of the object to be grasped.

The role played by handedness in performing cognitive tasks is another example of the involvement of motor processes in perceptual functions. de Sperati and Stucchi (1997) showed that right- and left-handed normal subjects used an internal simulation of the movement of their dominant hand in order to discriminate between observed screwing and unscrewing screwdrivers. In another series of experiments (Gentilucci et al. 1998a, b), normal subjects were required to judge handedness of pictures of hands and fingers assuming different postures. The results showed that the presentation of postures that hand and fingers commonly assume at rest, or when interacting with objects, facilitated the responses with respect to the presentation of less usual hand-finger postures, even when the latter were richer in visual cues useful for handedness recognition. Once again procedural motor knowledge was employed to solve a cognitive task.

4. Conclusions

The Visual Brain in Action (1995) by Milner and Goodale represents an insightful, provocative and innovative view that reshapes our knowledge on how the brain processes visual information to understand the external world and to act upon it. In our perspective their characterization of the parietal cortex as pragmatic provides a solid theoretical frame in which many recent clinical, neuropsychological and neurophysiological data can be reconciled. At this point, however, the question arises whether perception can be limited to the processing going on in the ventral stream, or rather be a more global process that must take into account the relationship existing between individuals and the environment they are continuously interacting with, by means of action. It is our suggestion that action is one of the founding principles of our knowledge of the world.

Notes

<1> All correspondence should be addressed to Vittorio Gallese.

References

Craighero, L., Fadiga, L., Rizzolatti, G., & Umilta, C.A. (in press). Movement for perception: a motor-visual attentional effect. Journal of Experimenal Psychology: Human Perception and Performance.

Decety, J., Perani, D., Jeannerod, M., Bettinardi, V., Tadary, B., Woods, B., Mazziotta, J. C. (1994). Mapping motor representations with PET. Nature, 371, 600-602.

de Sperati, C., & Stucchi, D. (1997). Recognizing the motion of a graspable object is guided by handedness. Neuroreport, 8, 2761-2765.

Fadiga, L., Fogassi, L., Pavesi, G., & Rizzolatti, G. (1995). Motor facilitation during action observation: a magnetic stimulation study. Journal of Neurophysiology, 73, 2608-2611.

Fogassi, L., & Gallese, V. (in press). The cortical basis of visuo-motor transformations fore grasping movements in the monkey. In Jackson, S., & Tipper, S: (Eds.), Vision and action. Advances in cognition series. UCL Press.

Gallese, V., Fadiga, L., Fogassi, & Rizzolatti, G. (1996). Action recognition in the premotor cortex. Brain, 119, 593-609.

Gallese, V., Fadiga, L., Fogassi, L., Luppino, G., & Murata, A. (1997). A parieto-frontal circuit for hand grasping movements in the monkey: Evidence from reversible inactivation experiments. In Thier, P., & Karnath, H-O. (Eds.), Parietal contributions to orientation in 3D space. (pp. 255-270). Experimental Brain Research Series, 25, Heidelberg: Springer-Verlag.

Gentilucci, M., Daprati, E., & Gangitano, M. (1998a). Implicit visual analysis in handedness recognition. Consciousness & Cognition, 7, 478-493.

Gentilucci, M., Daprati, E., & Gangitano, M. Right-handers and left-handers have different representations of their own hand. (1998b). Cognitive Brain Research, 6, 185-192.

Grafton, S. T., Arbib, M. A., Fadiga, L., & Rizzolatti, G. (1996). Localization of grasp representations in humans by PET: 2. Observation compared with imagination. Experimental Brain Research, 112, 103-111.

Hyvarinen, J., & Poranen, A. (1974). Function of the parietal associative area 7 as revealed from cellular discharges in alert monkeys. Brain, 97, 673-692.

Mountcastle, V. B., Lynch, J. C. G. A., Georgopoulos, A., Sakata, H., & Acuna, C. (1975). Posterior parietal association cortex of the monkey: Command functions for operations within extrapersonal space. Journal of Neurophysiology, 38, 871-908.

Murata, A., Fadiga, L., Fogassi, L., Gallese, V., Raos, V., & Rizzolatti, G. (1997). Object representation in the ventral premotor cortex (area F5) of the monkey. Journal of Neurophysiology, 78, 2226-2230.

Parsons, L. M., Fox, P. T., Downs, J. H., Glass, T., Hirsch, T. B., Martin, C. C., Jerabek, P. A., & Lancaster, J. L. (1995). Use of implicit motor imagery for visual shape discrimination as revealed by PET. Nature, 375, 54-58.

Rizzolatti, G., & Gentilucci, M. (1988). Motor and visual-motor functions of the premotor cortex. In Rakic, P., & Singer, W. (Eds.) Neurobiology of neocortex. (pp. 269-284). Chichester: Wiley.

Rizzolatti, G., Camarda, R., Fogassi, L., Gentilucci, M., Luppino, G., & Matelli, M. (1988). Functional organization of inferior area 6 in the macaque monkey: II. Area F5 and the control of distal movements. Experimental Brain Research, 71, 491-507.

Rizzolatti, G., Fadiga, L., Fogassi, L., & Gallese, V. (1996a). Premotor cortex and the recognition of motor actions. Cognitive Brain Research, 3, 131-141.

Rizzolatti, G., Fadiga, L., Matelli, M., Bettinardi, V., Paulesu, E., Perani, D., & Fazio, G. (1996b). Localization of grasp representations in humans by PET: 1. Observation vs. execution. Experimental Brain Research, 111, 246-252.

Rizzolatti, G., Fogassi, L., & Gallese, V. (1997a). Parietal cortex: from sight to action. Currrent Opinion in Neurobiology, 7, 562-567.

Rizzolatti, G., Fadiga, L., Fogassi, L., & Gallese, V. (1997b). The space around us. Science, 277, 190-191.

Sakata, H., Taira, M., Murata, A., &, Mine, S. (1995). Neural mechanisms of visual guidance of hand actions in the parietal cortex of the monkey. Cerebral Cortex, 5, 429-438.

Shikata, E., Tanaka, Y., Nakamura, H., Taira, M., & Sakata H. (1996). Selectivity of the parietal visual neurons in 3D orientation of surface of stereoscopic stimuli. Neuroreport, 7, 2389-2394.

Sirigu, A., Duhamel, J.R., Cohen, L., Pillon, B., Dubois, B., & Agid, Y. (1996). The mental representation of hand movements after parietal cortex damage. Science, 273, 1564-1568.

Sperry, R. W. (1952). Neurology and the mind-brain problem. American Scientist, 40, 291-312.

Ungerleider, L. G., & Mishkin, M. (1982). Two visual systems. In : Ingle, D. J., Goodale, M. A., & Mansfield R. J. W. (Eds.), Analysis of visual behavior. (pp. 549-586). Cambridge, MA: MIT Press.