Tony Movshon tells us that we need to understand our environment to understand our brain

Tony Movshon visited San Diego this week and, as usual, gave a fantastic talk. He’s a fantastic speaker, and a great pugilist. The topic of his talk is a subject near and dear to my heart: why it’s important to understand the natural environment in order to understand how our brain works.

Neuroscientists like to study vision either by showing an animal something simple, like lines or dots, or something totally random. We think that if we show something simple it should be easy to get the visual system to respond, and if we show enough randomness eventually we can average the images that made a particular neuron respond and see what is “driving” the cell. There’s two problems: dots and lines work great for the first couple steps for vision in the brain but don’t do much for the later, more complex areas, the ones that respond to more complicated things like faces and textures. Second, the randomness that we show is just flickering black and white static – but you might need to show a lot of static before you hit on what is important to a particular neuron.

This has led neuroscientists (and physicists) to spend a lot of time studying the properties of the world around us that our senses respond to. In the visual world, for instance, there is a lot of structure. Look up at a point on the wall. Now look at a point just next to it, and then a point just next to that. They’re probably pretty similar. This is because the natural world has long spatial, as well as temporal correlations. If we just include these correlations in our ‘random’ images, we limit the number of possibilities we’d have to work through to get at what a given neuron is responding to.

And it works! By studying the structure of the natural world, you can get neurons responding to the structured environment in very different ways than they respond to noise. You really, really need to understand the ecology of an animal to understand its nervous system.

Now Tony Movshon spent a while explaining this in more detail and then focused on new work of his to understand visual cortex. We often feel like we have a good grasp of what the first layer of visual cortex – V1 – does. It responds to basic features in vision like orientation and scale. But we have no clue what the next layer, V2, does. Frankly, we’ve studied every part of visual cortex before and after V2 better than we’ve studied this one – it’s just been a mystery.

By showing noisy images (middle, above) to a monkey, they saw that they could get neurons in V2 to respond – but that wasn’t a big deal because they could get neurons in V1 to respond as well. But when they showed more naturalistic images (right, above) to the monkey, the response in V1 was pretty much the same as before – but V2 responded much better. And when they showed random variations on the textures, neurons in V1 responded more consistently to a given individual image whereas neurons in V2 responded more consistently to a class of texture. These images are realistic textures which would imply that V2 is building up a representation of the world that is looking for complex statistical structure, something they never would have found with just ‘noise’.

Interestingly, if they asked a human to quickly compare a real image with its ‘naturalistic’ counterpart (the far left and far right images above), how well neurons in monkey V2 could tell them apart predicted how well humans could tell them apart! Some are easy to us (the top row) whereas some are more difficult (the middle row). And we’re not going to get any better at it than V2 can do.


Freeman J, Ziemba CM, Heeger DJ, Simoncelli EP, & Movshon JA (2013). A functional and perceptual signature of the second visual area in primates. Nature neuroscience, 16 (7), 974-81 PMID: 23685719

Simoncelli EP, & Olshausen BA (2001). Natural image statistics and neural representation. Annual review of neuroscience, 24, 1193-216 PMID: 11520932

2 thoughts on “Tony Movshon tells us that we need to understand our environment to understand our brain

  1. I agree that it is very important to study the environment/ecology that the organism is in, but it is also very important to remember that the organism and others of its species are a very important part of this environment. Although this doesn’t apply so much to vision, it is a big deal when we are looking at ill-defined concepts like intelligence. For example, if we properly model the environment then there isn’t that much difference between things like the environmental complexity hypothesis and the social brain hypothesis.

    On second thought, some of this thinking could be applied to the visual system. For V2 the environment isn’t what is outside of us, but what is outside of us as represented by V1. I think Maturana & Varela thought extensively about this sort of stuff.

    • As to your first point, I think you could make the argument that the visual environment is ‘ill-defined’ in a way. As you progress through the visual system you get neurons that respond to progressively more complex features, eventually including faces, etc. That means it’s not just the statistics of the environment but the social one that is important, too!

      I think the broader point is that the structure of neural circuitry often seems like it has evolved to take advantage of the correlational structure of the environment/ecology. You’re right, the environment for V2 is the output of V1, but V1 is a function, eventually, of the environmental input, and will have different structure in its output depending on the structure of the environment.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s