Sophie Deneve and the efficient neural code

Neuroscientists have a schizophrenic view of how neurons. On the one hand, we say, neurons are ultra-efficient and are as precise as possible in their encoding of the world. On the other hand, neurons are pretty noisy, with the variability in their spiking increasing with the spike rate (Poisson spiking). In other words, there is information in the averaged firing rate – so long as you can look at enough spikes. One might say that this is a very foolish way to construct a good code to convey information, and yet if you look at the data that’s where we are*.

Sophie Deneve visited Princeton a month or so ago and gave a very insightful talk on how to reconcile these two viewpoints. Can a neural network be both precise and random?

Screen Shot 2016-04-23 at 11.06.22 AM Screen Shot 2016-04-23 at 11.06.27 AM

The first thing to think about is that it is really, really weird that the spiking is irregular. Why not have a simple, consistent rate code? After all, when spikes enter the dendritic tree, noise will naturally be filtered out causing spiking at the cell body to become regular. We could just keep this regularity; after all, the decoding error of any downstream neuron will be much lower than for the irregular, noisy code. This should make us suspicious: maybe we see Poisson noise because there is something more going on.

We can first consider any individual neuron as a noisy accumulator of information about its input. The fast excitation, and slow inhibition of an efficient code makes every neuron’s voltage look like a random walk across an internal landscape, as it painstakingly finds the times when excitation is more than inhibition in order to fire off its spike.

So think about a network of neurons receiving some signal. Each neuron of the network is getting this input, causing its membrane voltage to quake a bit up and a bit down, slowly increasing with time and (excitatory) input. Eventually, it fires. But if the whole network is coding, we don’t want anything else to fire. After all, the network has fired, it has done its job, signal transmitted. So not only does the spike send output to the next set of neurons but it also sends inhibition back into the network, suppressing all the other neurons from firing! And if that neuron didn’t fire, another one would have quickly taken its coding


This simple network has exactly the properties that we want. If you look at any given neuron, it is firing in a random fashion. And yet, if you look across neurons their firing is extremely precise!

* Okay, the code is rarely actually Poisson. But a lot of the time it is close enough.


Denève, S., & Machens, C. (2016). Efficient codes and balanced networks Nature Neuroscience, 19 (3), 375-382 DOI: 10.1038/nn.4243

I have seen things you wouldn’t believe (in my mind)

When two different people perceive blue, is it the same to both of them? When two people imagine it, is it the same? Can everyone even imagine it?

If I tell you to imagine a beach, you can picture the golden sand and turquoise waves. If I ask for a red triangle, your mind gets to drawing. And mom’s face? Of course.

You experience this differently, sure. Some of you see a photorealistic beach, others a shadowy cartoon. Some of you can make it up, others only “see” a beach they’ve visited. Some of you have to work harder to paint the canvas. Some of you can’t hang onto the canvas for long. But nearly all of you have a canvas.

I don’t. I have never visualized anything in my entire life. I can’t “see” my father’s face or a bouncing blue ball, my childhood bedroom or the run I went on ten minutes ago. I thought “counting sheep” was a metaphor. I’m 30 years old and I never knew a human could do any of this. And it is blowing my goddamned mind…

I opened my Facebook chat list and hunted green dots like Pac-Man. Any friend who happened to be online received what must’ve sounded like a hideous pick-up line at 2 o’clock in the morning:
—If I ask you to imagine a beach, how would you describe what happens in your mind?
—Uhh, I imagine a beach. What?
—Like, the idea of a beach. Right?
—Well, there are waves, sand. Umbrellas. It’s a relaxing picture. You okay?
—But it’s not actually a picture? There’s no visual component?
—Yes there is, in my mind. What the hell are you talking about?
—Is it in color?
—How often do your thoughts have a visual element?
—A thousand times a day?
—Oh my God.

And so on. Read the whole thing, and this as well. How common is something like this? Judging by internet comments – very common, especially for other sensory modalities. I can visualize fine, though my imaginary ‘sense of place’ is probably stronger, but I cannot ‘imagine’ a taste or smell to save my life. I once went into a fancy cocktail bar and was asking the owner how he came up with the cocktails. He just thought about how two ingredients could taste together, he said, and then he combined them like that. Whoa, whoa, whoa, I said, you can imagine tastes? And combine them in your mind?

Who knows what others imagine in their mind? Does imagining a picture mean the same thing to different people? Is it vivid or faded, cartoonish or realistic?

When we do experiments with animals – how much are we relying on this supposed universality which, even among humans, is anything but?