We are better off without perfect perception

Stoffregen & Bardy’s target article is based on the assumption that our senses’ ultimate purpose is to provide us with perfect information about the outside world. We argue that it is often more important that information be available quickly than that it be perfect. Consequently our nervous system processes different aspects of information about our surrounding as separately as possible. The separation is not between the senses, but between separate aspects of our surrounding. This results in inconsistencies between judgments: sometimes because different frames of reference are used. Such inconsistencies are fundamental to the way the information is picked up, however, and hence cannot be avoided with clearer instructions to the subjects. Since the Stoffregen & Bardy (S&B) target article deals with human interactions with the environment, it is impossible to ignore the physiology involved. Once one considers the physiology, it becomes evident that in practice there can be no “specification” of the kind described in the target article. For instance, in color vision it is well known that various combinations of wavelengths of light can stimulate the three kinds of cones in exactly the same manner, so that we are unable to distinguish between them. Similarly, various combinations of ego-motion and motion of the environment can give rise to the same global optical flow (sect. 5.1). These are examples of what S&B would call many-to-one mappings. Unless all information from all the senses is considered for every judgment, many-to-one mappings will give rise to conflicting judgments. S&B examine ways to avoid such conflicts. We question whether such conflicts have to be avoided. Hidden behind worries about conflicting judgments is the assumption that our senses’ ultimate purpose is to provide us with perfect information about the world “outside.” To obtain such perfect information it makes sense to combine input from all the senses. However, attempting to gain access to perfect information has a price: time. For interacting with the environment, timing can be much more important than precision. There is no point knowing exactly when a ball will hit you if you only gain access to this information once it is too late to react to it. We have shown that it takes 200 msec to react to a change in the speed of a target that one is trying to hit (Brenner et al. 1998), but only 110 msec to react to a change in its position (Brenner & Smeets 1997). Since these kinds of movements take only a few hundred milliseconds, this difference in timing is not negligible. When hitting moving targets the direction in which subjects move their hand does not appear to depend on the target’s velocity (Brenner & Smeets 1996; Smeets & Brenner 1995), presumably because the disadvantages of waiting an additional 90 msec outweigh the advantages of having access to reliable velocity information. In our opinion, the main task of our senses is to select the most suitable information for the task at hand, and to do so fast. From the moment the information reaches our senses separate aspects of the information are selected and analyzed for specific tasks, or parts of tasks. The selection starts even earlier if one considers the movements we make to obtain the information. Separate independent processing for different judgments can result in substantial conf1icts between them (Abrams & Landgraf 1990; Brenner et al. 1996; Glennerster et al. 1996; Mack et al. 1985). Nevertheless, the separation seems be so complete that we even fail to notice conflicts between attributes when the conflicts themselves could give us valuable additional information (Brenner & Damme 1999; Brenner & Landy 1999). The main reason for our judgments normally being approximately consistent is presumably the consistency in the world “outside,” in what S&B call the “global array.” Abandoning the need to avoid conflicts allows the nervous system to rely on different information for each judgment. We assume that each judgment is based on the most reliable information for that particular judgment. Thus, relative positions are judged from retinal information alone, but egocentric localization Commentary/Stoffregen & Bardy: On specification and the senses BEHAVIORAL AND BRAIN SCIENCES (2001) 24:2 215 Figure 1 (Brenner & Smeets). Target velocity during simulated ego-motion that matched the simulated velocity in a preceding interval without simulated ego-motion. All velocities are relative to the visual surrounding. Each thick line connects the centres of the ranges of acceptable target velocities for the seven simulated ego-velocities for one of the five subjects. The thin diagonal line represents a constant velocity of 0.2 m/sec of the target relative to the observer, ignoring the visually simulated ego-motion. The thin dashed line represents a constant velocity of 0.2 m/sec of the target relative to the visual surrounding. The two panels show the same subjects’ performance with two different instructions. needs extra-retinal information about the orientation of our eyes as well (Brenner & Cornelissen 2000) In this example the difference in information is associated with a difference in the referent that is involved (see sect. 4.5). Our view implies that the referent is fixed for any given judgment, rather than being something subjects can choose as S&B suggest in section 5. We examined subjects’ freedom in choosing a referent by asking them to compare the initial and final velocities of an approaching target (simulated with both monocular and binocular cues on a large screen). The methods were very similar to those used in a similar study on lateral motion (Brenner 1991) The target initially approached at 0.2 m/sec while the background was static. During the presentation the visual background started moving in depth so that the optic flow was consistent with forward or backward ego-motion of the subject. At the same time the target could change its velocity. We determined how fast the target had to move during the simulated ego-motion for it to appear to continue to move at the same speed. The subjects were initially instructed to judge whether the targets’ velocity changed, without explaining what we meant by “the velocity.” Subjects had no difficulty with this task, and all five subjects spontaneously judged the target’s velocity relative to themselves, ignoring the visually simulated ego-motion (left panel in Fig. 1). These results are consistent with previous work on lateral motion, in which velocity judgments also appeared to be related to oneself (Brenner 1991; Brenner & van den Berg 1996). We then showed the subjects their data and explained to them that we were simulating ego-motion and that we wanted them to judge the target’s velocity relative to the surrounding. Subjects found this much more difficult. The results are shown in the right panel of Figure 1. It is clear from the variability that none of the subjects was really able to do this task. Perhaps they would have been able to do so if they had actually experienced vection, or if the simulation was not only visual, but apparently they were not able to select the visual surrounding as a referent. How many systems make a global array?