Sophie Deneve and the efficient neural code

Neuroscientists have a schizophrenic view of how neurons. On the one hand, we say, neurons are ultra-efficient and are as precise as possible in their encoding of the world. On the other hand, neurons are pretty noisy, with the variability in their spiking increasing with the spike rate (Poisson spiking). In other words, there is information in the averaged firing rate – so long as you can look at enough spikes. One might say that this is a very foolish way to construct a good code to convey information, and yet if you look at the data that’s where we are*.

Sophie Deneve visited Princeton a month or so ago and gave a very insightful talk on how to reconcile these two viewpoints. Can a neural network be both precise and random?

Screen Shot 2016-04-23 at 11.06.22 AM Screen Shot 2016-04-23 at 11.06.27 AM

The first thing to think about is that it is really, really weird that the spiking is irregular. Why not have a simple, consistent rate code? After all, when spikes enter the dendritic tree, noise will naturally be filtered out causing spiking at the cell body to become regular. We could just keep this regularity; after all, the decoding error of any downstream neuron will be much lower than for the irregular, noisy code. This should make us suspicious: maybe we see Poisson noise because there is something more going on.

We can first consider any individual neuron as a noisy accumulator of information about its input. The fast excitation, and slow inhibition of an efficient code makes every neuron’s voltage look like a random walk across an internal landscape, as it painstakingly finds the times when excitation is more than inhibition in order to fire off its spike.

So think about a network of neurons receiving some signal. Each neuron of the network is getting this input, causing its membrane voltage to quake a bit up and a bit down, slowly increasing with time and (excitatory) input. Eventually, it fires. But if the whole network is coding, we don’t want anything else to fire. After all, the network has fired, it has done its job, signal transmitted. So not only does the spike send output to the next set of neurons but it also sends inhibition back into the network, suppressing all the other neurons from firing! And if that neuron didn’t fire, another one would have quickly taken its coding


This simple network has exactly the properties that we want. If you look at any given neuron, it is firing in a random fashion. And yet, if you look across neurons their firing is extremely precise!

* Okay, the code is rarely actually Poisson. But a lot of the time it is close enough.


Denève, S., & Machens, C. (2016). Efficient codes and balanced networks Nature Neuroscience, 19 (3), 375-382 DOI: 10.1038/nn.4243

5 thoughts on “Sophie Deneve and the efficient neural code

  1. “The first thing to think about is that it is really, really weird that the spiking is irregular.”

    Actually, I don’t find that weird at all. In fact, I’d argue that it’s a feature, not a bug: if all spiking were regular, animals would be too predictable and not autonomous. In fact, even in C. elegans, there are apparently lots and lots of neurons whole sole purpose is to make their firing more irregular than they otherwise would be:

    It appears that Dr. Denève is explaining away a problem that does not exist.

    • I agree that the fact that the firing of neurons is irregular should be of no surprise. With the branched dendritic system, the neuron can receive several stimuli including both excitatory and inhibitory postsynaptic potentials increasing randomness and variability as well as both temporal and special summation to add to the irregularity. With this great variability in possible potential magnitudes, it is no wonder the action potentials can appear random.

    • To be more specific, it is really weird that spiking is so irregular in sensory areas (at least). Sure, there is some exploration/exploitation tradeoff in behavior but I have a hard time believing it should be at that level anywhere near sensory areas.

      Maybe a behavior like a random walk in C. elegans should be stochastic… but I certainly hope my reaching to pick up an object doesn’t reach that level of randomness.

      • Correct! The nervous system needs to be able to adjust the level of unpredictability. There are several mechanisms at play which have been shown to do just that. One is to couple ‘noisy’ neurons such that they fire a lot more regularly and together. Another is to use the ‘noise’ in stochastic resonance. Another is to have the motor system be so flexible that it can perform the same behavior with many different sets of neuronal activity. Indeed, at least in vertebrates, it seems no two seemingly identical movements are performed with identical neuronal patterns.
        Finally, for many sensory neurons (also those in C elegans!) it has been shown that they are very faithful and reproducible with little noise.
        With synapses being so expensive, any synapse after a sensory neuron that doesn’t go directly onto a muscle serves a function or it would have been eliminated (generally speaking). If the signal appears less deterministic after a synapse, it either doesn’t hurt whatever else the synapse is there for, or it serves a function.
        Giant fiber systems show that synapses (especially chemical ones) get eliminated if signal processing is not required.

  2. I also do not find the irregularity to be weird but however to be a nessacary attribute to the way our neurons function. I also believe it is this randomness that keeps us from being to predictable. If we were to become predictable we would become more vulnerable. With the complexity the branched dendritic system offers it is no wonder with so many different variables that it appears to be random.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s