Tag Archives: information

Are we wrong to believe we have separate senses?


A bird in flight must control its movement with reference to multiple simultaneous sources of information: air mass, the ground plane, changes in stimulation caused by its own movements (image)

Eyes are for seeing and noses are for smelling. Everyone knows this. You know this. But hold on, what’s this? Ecological psychologists think you’re wrong!

The problem goes like this. Animals have to move about in an environment that already has other stuff in it besides the animal itself. All kinds of stuff. Imagine you’re a fish. You want to get over there to swallow that clod of plankton. If this was a video game you’d just press buttons until the plankton clod got big and filled up the screen. And when you can’t make the plankton clod look any bigger, that’s when you know you’ve arrived. Good old-fashioned visually-guided locomotion. But things aren’t that simple. You’re a fish, and there’s water in the way. Now you also have to move in such a way as to cancel out the movement of the water. Also, since you’re a fish, that is, a living, active organism, the visual scene before your eyes is in part structured by your own movements. Now you have to work out which bits of the stuff you’re looking at are moving because stuff out there is moving, and which bits are moving because you’re a fish and you’re twitching your face from side to side and your paddling is creating weird ripples and shadows.

One way to respond to this is to shrug and say, “I dunno, being a fish is complicated, I guess. At least they don’t have to worry about table manners or heights.” This is the normal way to respond. If you’re an ecological psychologist, however, you might respond differently. You might come out with something like this: “We propose that the senses form a single, unitary, irreducible perceptual system whose sole function is to detect parameters of the global array.”

In making this proposal, Stoffregen, Mantel, and Bardy are doing us a service for they are forcing us to question an unexamined assumption. In this case, the assumption is that the senses constitute distinct and separate systems. As the authors note, James J. Gibson also liked to question unexamined assumptions. But Gibson himself accepted this particular assumption without examining it. According to SMB, this has led to problems. One such problem: research within the ecological tradition has asked questions of the form, “What visual parameter is implicated in controlling the movement of this seabird into this body of water?” But this reductively detaches the visual component of the behaviour from everything else that is going on in the bird–air–water–gravity–etc. system.

A more esoteric problem is this: specification of structure in the environment into structure in perception does not work if we consider the senses as distinct systems. If all the seabird has to go on is vision, then it has no way of distinguishing between changes in the appearance of the water surface that are caused by its movements towards the surface, and those that are caused by the movement of the air relative to the same surface. Without being able to distinguish between these, SMB claim, the bird cannot adjust its behaviour over time in the appropriate way. Specification only works if we consider all of the relevant structure at once. And we need specification. Ambiguity is no good. If specification fails, then direct perception is also impossible, and ecological psychology itself disintegrates into an incoherent puddle.

Now, it is tempting to respond to SMB’s proposal that the senses constitute a single, unitary system in this manner: “Nuh-uhh! The senses are obviously separate! Look! Eyeballs!” But SMB have been making this argument for a few years now and have apparently come across this response before (S&B originally published this argument in BBS in 2001; the 2017 paper is largely a reassertion of that one). The authors call for their critics to go beyond merely pointing out apparent instances of counter-evidence and to engage with their arguments “at the level of general principles”. So let’s try to do that. I will make three points.

First, while every assumption deserves to come under scrutiny once in a while, it’s impossible to evaluate any given assumption except in the light of remaining assumptions that we hold to be more fundamental or secure. Thus, SMB state:

If we accept that perception yields knowledge (i.e., that perception is epistemological), then we can ask what type of knowledge is obtained. Following Darwinian principles … Gibson reasoned that perception yields (and should yield) knowledge about the environment.

But wait, what? Why should we accept this assumption? I mean, not to be rude, but “perception yields knowledge”… that sounds a little, I dunno, Cartesian? SMB also talk about perception as having content: “Our focus is on the animal–environment interaction as the contents of perception”.

Maybe the real problematic assumption here is this one: that information is contentful, i.e. that it is about structure in the environment. This argument was made by Van Dijk, Withagen, & Bongers in a paper that I blogged about here.  If information is contentless, then the issue of ambiguity does not arise. Ambiguity is not in the information because information is not being transmitted. The animal is not in the business of constructing knowledge about what the world has in it. Rather, the animal is in the business of regulating the fit between itself and its environment. If this is the case then perception does not have to be infallible, it just has to be good enough to keep the animal alive for a certain while. The fit is either maintained, in which case the animal is still alive, or it isn’t, and it’s dead. But why say that this fit is the content of perception?

Second, in their enthusiasm to do violence to the Aristotelian dogma of separate senses, SMB sometimes risk doing damage to other theoretical distinctions which we presumably want to maintain. For instance, it seems useful to distinguish between a) action and b) structure in the environment with reference to which that action is being regulated. In their more effusive moments SMB seem to want to fold this all together:

The global array is not ambient to massless, geometric points; rather, it intrinsically, inescapably is ambient to and provides information about the full dynamics of the animal–environment interaction. The global array pulses with the life of the organism; it embodies animal–environment reciprocity.

SMB argue that Gibson’s notion of the ambient optic array as a set of points of potential observation is “disembodied” and “reductionist”, and we should instead think in terms of an “embodied point of observation”. We should recognize that visual information in real life is structured simultaneously by stuff in the environment and by the activities of the living, moving animal itself. When you walk, your eyeballs jiggle in your head, which is going up and down on top of your neck thump-te-thump. This is fine, but don’t we also still want to be able to talk about structure that’s actually out there in a straightforward way, without constantly having to talk about it in terms of the animal–environment fit?

Third, and more generally: suppose we all went along with SMB and just admitted that the notion of separate senses is mere superstitious hokum and baloney that has been holding us back from pursuing a properly scientific study of perception. Where would that actually get us? In what way is it useful to treat perception as a single system? It certainly seems useful to treat vision, say, as a distinct system. To do so is consistent with our everyday conception of our experience—our way of talking about it. Don’t we want to be able to explain, say, the navigability of the inside of a building in terms of what it looks like without having to also talk about what the air feels like on our faces or what smells are wafting in from a nearby cafeteria? How does it help us to flat out deny the distinctions we have traditionally made between the senses?

It’s hard to see the global array concept getting any real uptake based only on arguments that have been presented so far. What would make the case more convincing? SMB need to show either that the assumption of distinct senses is pernicious and has misled us in important ways, or that rejecting the assumption leads to useful outcomes or applications that would otherwise be inaccessible to us. They have done neither of these so far. Maybe they’re right, though. I’m going to ask a fish.


Stoffregen, T. A., Mantel, B., & Bardy, B. G. (2017) The senses considered as one perceptual system. Ecological Psychology, 29(3):165–197 doi: 10.1080/10407413.2017.1331116



Is ecological psychology compatible with enactivism? Is information always animal-relative? A comment on Van Dijk, Withagen, and Bongers (2015)

One question I have had for a while is why it is that there seems to be so little overlap between ecological psychology and enactivism. Both approaches are explicitly anti-representational, both seek to treat the organism and its environment as constituting a system, and both draw their inspiration and metaphors from biology and ecology rather than, as is the case with cognitivism, from the fields of computer science and logic. And yet believers in the enactivist approach seem to have trouble communicating their ideas to believers in the ecological approach (among whom I count myself), and vice versa. It’s like there are two separate churches and the congregation inside each simply cannot comprehend the experience that led others to join the rival church.

So I was interested to read this paper by my fellow ecological psychologists Ludger Van Dijk, Rob Withagen, and Raoul Bongers (2015) who attempt to reach out to the enactivists and allay some of their concerns about the ecological approach. (Disclosure: I’ve had the pleasure to spend some time chatting with Ludger at a couple of conferences and have found we are generally in agreement on the deep issues in the field. I had nothing to do with the creation of this paper.) The paper has gone some way to clearing up for me just what the enactivists’ objection is to the Gibsonian programme. But it also raises a new issue. The authors ask ecological psychologists to abandon the concept of information about the environment in favour purely of talking about information for affordances.

I am sympathetic to this proposal. However, I do not have a very clear idea about how it will be received by the rest of the field. I am curious to find out. I suspect Van Dijk et al have understated just how much of a challenge this is to ecological psychology as it is currently practiced. I will lay out Van Dijk et al’s argument here in an attempt to set out as clearly as possible just what the dilemma is that ecological psychologists are faced with.

A content-free conception of information?

Enactivists have long worried that the ecological programme does not go far enough in rejecting the traditional dualisms of psychology, specifically the dualism that asserts that content or meaning exists externally to the animal and independently of the activity of that animal. As Van Dijk et al explain:

[Ecological psychologists] typically talk of information a lot, and it is not always clear whether this information is of a content-less kind. Indeed, the enactivist tradition seems worried about this. Varela et al. (1991), for example, felt that Gibsonians were building their “theory of perception entirely from the side of the environment” (Varela et al., 1991, p. 204). They worried that the ecological notion of information that grounded information in the correspondence between the structure of ambient light and the environment, required too little active participation of the animal.

In short, Varela et al accuse Gibsonians of merely appearing to talk the anti-representational talk while smuggling in content by the back door. They read Gibson’s description of what’s-out-there-to-be-perceived (his ecological optics) as focusing only on how light comes to contain structure from the surrounding world without explaining how the animal makes use of that structure. This leaves an explanatory gap between the structure in light and the biological and phenomenological processes internal to the animal. Or, put another way, Varela et al suspect us Gibsonians of being closet dualists, because for all our talk of the animal in its environment as a perceiving-acting system, we do not go far enough in recognizing that perceiving and acting are in fact one and the same process. Simply “picking up” information that already exists in the array is not acting; it is registering or representing. It is a passive process. Only if the target of perceiving (the objects and surfaces and so on) and the act of perceiving are truly flattened into the same process can we remove any whiff of mind-body dualism.

The solution that Van Dijk et al propose is to abandon all talk of information as being “about” something in the environment. Information-about implies that meaning exists externally to the animal and that it exists prior to the animal’s trying to act on the environment. It implies a notion of information that “carries content” from the objects in an animal’s environment to the animal’s sense receptors, to be passively detected. Instead, we should think of information as content-less: before the animal tries to act, there is no “information”, as such. Information arises in the animal’s acting on and exploring of its surroundings. The information that arises is information-for affordances; that is, it reveals to the animal possibilities for subsequent action:

“Information for” calls attention to the fact that ecological information need not be about anything – has no “aboutness” – prior to use, but it is for something to an active animal. It is for perceiving the environment, for acting on affordances and the likes.

As I stated above, I’m sympathetic to this proposal. But what Van Dijk et al do not really talk about (although they allude to it) is just how at odds this proposal is with the way information is generally talked about by ecological psychologists today.

Can ecological psychology make do without information-about?

Van Dijk et al argue that in his later work Gibson was moving away from talk of information-about in favour of purely talking about information-for. Abandoning information-about should, then, be no big deal. Except that there’s a problem: whatever direction Gibson himself might have been moving in, his followers did not necessarily continue to move in that direction. Indeed, the acceptance of both information-for and information-about is core to modern ecological psychology. Michaels and Carello make this point quite explicitly (1981, p. 37):

A description of the relation between the environment and the structure of light and sound provides one facet of the concept of information—that is, *information-about*. These patterns of energy describe objects, places, and events in the environment. However, there is a second and equally important facet of information that information-about does not touch upon. This second facet of information is *information-for* and the object of the preposition *for* is the animal. Information is the bridge between an animal and its environment and cannot be usefully described without a specification of *both*.

And, strikingly (Michaels and Carello, 1981, p. 38):

To summarize, a failure to consider both parts of information—information *about* an environment *for* an animal—is to miss the very essence of the concept of information.

In the formulation of ecological psychology that followed Gibson, then, information was explicitly defined as having two parts, and this two-part nature was seen as a feature of the concept of information, not a bug. This dual property is formalized in the symmetry principle which was put forward by Turvey, Shaw, and Mace, and which is discussed here by Andrew. The symmetry principle is presented as an account of how it is that perception can be direct. The environment is said to specify patterns in light (information) which in turn specify what is perceived. Conversely, what is perceived is said to specify what the pattern is that is in the light which in turn specifies what is in the environment (hence “symmetry”). According to this formulation, it is not a problem if light is said to carry information about the environment. On the contrary, this is seen as a good thing because it explains how it is that we perceive what is actually in our environment. “Information” is merely the central term in a set of 1:1 correspondences that point in two directions at the same time: to the environment and to the perceiver. (But note that while the symmetry principle makes reference to perception it does not make reference to the activity of the animal, which lends support to the enactivists’ suspicion that perception is being covertly treated as the passive inputting of content.)

Other formulations of information by ecological psychologists also rely on information-about. Both Chemero (2009, p. 116 and following pages) and Wilson and Golonka (2013) attempt to develop a version of information that draws on Barwise and Perry’s situation semantics. Here, the symmetry principle is abandoned and it is allowed that information can be “about” something that is not directly specified in the pattern in the energy array. So a spoken sentence can be said to be about something more than just the sound pattern and the vocal equipment that produced that pattern: it can be said to be about what the sentence is in fact about. Instead of appealing only to specification, these authors appeal to constraints that have to be learned, and that link the pattern to its meaning. But, crucially, the pattern is still said to have a meaning, independently of the activity of the actor. This is what Van Dijk et al are asking us to give up: the idea that meaning exists independently of the animal and its actions.

The dilemma for Gibsonians

A confession: not so long ago I used to think of enactivism as merely a form of old-fashioned idealism masquerading as modern biologically-inspired scientific psychology. Enactivism is easy to dismiss because the book that’s presented as foundational to the field, Varela et al’s (1991) The Embodied Mind, is also a self-indulgent foray into Buddhist scholarship. I used to think that the only reason that enactivists weren’t Gibsonians was because they simply hadn’t appreciated the enormity of Gibson’s achievement. Now I am not so sure. I am now persuaded that at least some of the problems that enactivists see are real, and that those problems are not merely an artefact of how the ideas have been communicated to outsiders but are internal to the ecological approach. Van Dijk et al’s paper has helped crystallize that for me. It is not enough to just insist that we are right; we need to do more work if we wish to persuade others that that is the case. So it will be useful to try to be clear about just what the challenge is that we are faced with.

Part of the problem may be that Gibson’s approach to perception is in large part really a theory of structure. Ecological optics is about the correspondence between structure in light and the structure of the world. It provides an account of what’s out there to be perceived, but not of how we learn to make use of that structure, still less of how our use of that structure comes to shape the things we can do when we’re no longer looking at the structure directly (e.g. how it is that I can remember scoffing at Varela et al’s arguments even though I no longer have their book in front of me). It’s been pointed out that the way James J. Gibson and Eleanor J. Gibson divided up their study interests—the former focusing on the senses while the latter focussed on perceptual learning—has introduced a long-standing divide into the entire field. If we really want to persuade the enactivists that Gibson is worth following then we are going to have to take on the hard work of putting the active learner back into its environment. Van Dijk et al argue that this requires a concept of information that is truly animal-relative and content-free. (They also argue that James J. Gibson was himself engaged in pursuing this conception as he attempted to develop his theory of affordances.)

The dilemma appears to be thus:

  • We can concede that much of ecological psychology is not really a theory of psychology but a theory of structure, a theory of what’s-out-there-for-an-animal-to-perceive; this need not be seen as a defeat: we can insist that Gibson’s account is the best account of structure that we have and that no anti-representational approach to psychology can succeed that doesn’t acknowledge Gibson’s great achievement here
  • Or we can insist that Gibson’s approach really does constitute a theory of psychology, and we can continue to conduct empirical research in the traditional ecological manner: taking some arbitrary task (such as stair-climbing or aperture-passing) and attempting to identify the invariant that is implicated in the control of that task; this option has worked well up to now and it has certainly been productive; but this approach may preclude ecological psychologists from having anything much to say about phenomena that go beyond the immediate perception of structure, things like language and other social phenomena (i.e., the concerns of this blog), or indeed about any phenomenon that appears to involve creativity or spontaneity on the part of the actor

The strange thing is that Van Dijk et al wrote their paper in an attempt to ride to Gibson’s defence, against enactivist attacks. In doing so they have revealed to us just how much work we still have to do. I suppose this may seem like a rather negative conclusion but I don’t think it is. I think it’s quite exciting that some of this hard work is still there to be done. I’m curious to find out if others feel the same way.