As a butterfly flaps its wings in Tokyo, a neuron in your head veers slightly heavenward…

When you look at the edge of a table, there is a neuron in your head that goes from silence to pop pop pop. As you extend your arm, a nerve commanding the muscle does the same thing. Your retina has neurons whose firing rate goes up or down depending on whether it detects a light spot or a dark spot. The traditional view of the nervous system descends from experiments that have supported this view of neural activity. And perhaps it is true at the outer edges of the nervous system, near the sensory inputs and the motor outputs. But things get murkier once you get inside.

Historically, people began thinking about the brain in terms of how single neurons represent the physical world. The framework they settled on had neurons responding to a specific set of things out in the world, with the activity of those neurons increasing when they saw those specific things and decreasing when they saw their opposite. As time flowed by, this neural picture became jumbled up with questions about whether overall activity level or specific timing of an individual spike was what was important.

When it comes to multiple neurons, a similar view has generally prevailed: activity levels go up or down. Perhaps each neuron has some (noisy) preference for something in the world; now just think of the population as the conjunction of each of their activity. Then the combination of all of the neurons is less noisy than any individual. But still: it’s all about activity going up or down. Our current generation of tools for manipulating neural activity unconsciously echoes this idea of how the nervous system functions. Optogenetics cranks the activity of cells – though often specific subpopulations of cells – to move their activity up or down in aggregate.

An alternate view which I has been pushed primarily by Krishna Shenoy and Mark Churchland takes a dynamic perspective of neural activity, and I think comes from taking a premotor view of the nervous system. Generally, nervous  activity is designed to control our physical behavior: moving, shouting, breathing, looking, remaining silent. But that is a lot to have to control, and selection of the correct set of behaviors has to take a huge numbers of factors into account and has a lot to prepare for. What have I seen? How much do I like that? What am I afraid of? How hungry am I? This means that premotor cortical activity is probably representing many things simultaneously in order to choose among them.

The problem can be approached by looking at the population of activity and asking how many different things it could represent, without necessarily knowing what those are. Perhaps the population is considering six different things at the same time (a noted mark of genius)! Now that’s a slightly different perspective: it’s not about the up or down of overall activity, but how that activity flows through possibilities on the level of the whole population.

These streams of possible action must converge into a river somewhere. There are many possible options for how this could happen. They could be lying in wait, just below threshold, building up until they overcome the dam holding their behavior at bay. They could also be gated off, allowed to turn on when some other part of the system decides to allow movement.

But when we stop and consider the dynamics required in movement, in behavior, another possibility emerges. Perhaps there is just a dynamical system churning away, evolving to produce some reaching or jumping. Then these streams of preparatory activity could be pushing the state of the dynamical system in one direction or another to guide its later evolution. Its movement, its decision.

Churchland and Shenoy have worked on providing evidence for this happening in motor cortex as well as prefrontal cortex: neurons there may be tuned to move their activity in some large space, where only the joint activity of all the neurons is meaningful. In this context, we cannot think usefully about the individual neuron but instead must consider the whole population simultaneously. It is not the cog that matters, but the machine.

References

Kaufman MT, Churchland MM, Ryu SI, & Shenoy KV (2014). Cortical activity in the null space: permitting preparation without movement. Nature neuroscience, 17 (3), 440-8 PMID: 24487233

Mante V, Sussillo D, Shenoy KV, & Newsome WT (2013). Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature, 503 (7474), 78-84 PMID: 24201281

Churchland, M., Cunningham, J., Kaufman, M., Foster, J., Nuyujukian, P., Ryu, S., & Shenoy, K. (2012). Neural population dynamics during reaching Nature DOI: 10.1038/nature11129

Shenoy KV, Sahani M, & Churchland MM (2013). Cortical control of arm movements: a dynamical systems perspective. Annual review of neuroscience, 36, 337-59 PMID: 23725001

Ames KC, Ryu SI, & Shenoy KV (2014). Neural dynamics of reaching following incorrect or absent motor preparation. Neuron, 81 (2), 438-51 PMID: 24462104

Churchland, M., Cunningham, J., Kaufman, M., Ryu, S., & Shenoy, K. (2010). Cortical Preparatory Activity: Representation of Movement or First Cog in a Dynamical Machine? Neuron, 68 (3), 387-400 DOI: 10.1016/j.neuron.2010.09.015

Advertisements

Vision is for decision

Cholinergic learningWhen we typically think of how decision-making works in the brain, we think of new input coming in, perhaps through the eyes or ears, being processed in the relevant sensory areas, and then sent to the ‘decision-making’ areas (the basal ganglia, prefrontal cortex, or anterior cingulate cortex) where this information is used to make a decision.  Although useful and intuitive, this modular view ends up giving short shrift to some areas that do heavy lifting.

Sensory areas are not actually the ruthless calculating machines that we tend to think of, but are in fact quite plastic.  This ability of sensory cortex to modify its own responses allows it to participate in certain decisions: for instance, it can learn how long to wait in order to get a reward.  If a rat receives two visual cues that predict how long it will have to wait in order to receive a reward – either a short time or a long time – neurons in the initial part of visual cortex, V1, will maintain a heightened firing rate to match that duration.

This is accomplished through something like reinforcement learning.  When learning whether a visual cue is giving an animal information about how long it will have to wait for a reward, acetylcholine acts as a ‘reinforcement signal’.  The effect is to change encoding of the reward by modifying the strength of the synapses in the network.

Although we tend to think of certain ‘decision-making’ areas of the brain, in reality all of the brain is participating in every decision at some level or another.  In certain cases – perhaps when speed is of the essence or maybe when you want other areas of the brain to be involved in the computations and processing of that decision – even sensory portions of the brain are learning how to make decisions.  It is not always dopamine, the ‘rewarding’ or ‘motivational’ chemical in the brain that supports this decision-making: other neuromodulators like acetylcholine often play the very same role.

References

Chubykin, A., Roach, E., Bear, M., & Shuler, M. (2013). A Cholinergic Mechanism for Reward Timing within Primary Visual Cortex Neuron, 77 (4), 723-735 DOI: 10.1016/j.neuron.2012.12.039

Learning: positive and negative

Reward and punishment operate through two very different pathways in the human brain.  The general idea is that these two types of learning – positive and negative – operate through different unique types of dopamine receptors.  The D1 receptors (D1R) are generally ‘positive’ receptors, while the D2 receptors (D2R) are ‘negative’.  Specifically, D1Rs generally tend to increase the concentration of CamKII and D2Rs decrease it; this means that they are going to have opposite effects on downstream pathways such as receptor plasticity, intrinsic voltage channels, etc.

How are the D1 and D2 pathways distinct in terms of learning?  The hypothesis has been that in striatal projection neurons, D1R expressing medium spiny neurons (dMSNs) mediate reinforcement and D2R expressing indirect pathway neurons (iMSNs) mediate punishment.  Kravitz et al expressed channelrhodopsin selectively in dMSNs and iMSNs so they could use light to activate only one type of neuron at a time.  They figured that the striatum would be a good place to start looking for the effects of these neurons.  After all, it is a primary site of reinforcement and action selection (also, they probably tried a few other places and didn’t get great results…?).  These transgenic mice were then placed in a box with two triggers, one of which would stimulate the light and the other would do nothing.  So the mice are in this box, and able to turn on and off their neurons if they want to.  I wonder how that feels?

When the mice were able to activate their D1R (positively-reinforcing) neurons, they were much more likely to keep pressing the trigger.  The D2R (negatively-reinforcing) mice were more likely to press the other trigger.  But that’s not all!  By the third day, the effects of activating the D2R pathway had worn off – they no longer cared about the effect.  You can see this on the graph to the left, where 50% is chance.  The preference for the D1R pathway persisted, however.  Even on short time scales of 15 – 30 seconds, the mice kept their preference for stimulating D1R reward cells over D2R aversion cells.  In the figure to the right, this is seen with YFP being a control (it should have no effect); whereas activating the dMSN pathway over the first 30 seconds always is different than activating YFP, the iMSN pathway only shows a (statistical) different over the first 15 seconds.

The authors conclude by saying that that the dMSN pathway is sufficient for persistent reinforcement, while iMSNs are sufficient for transient punishment.  This is a nice finding; that the D1R pathway really is doing some positive reinforcement and that the D2R pathway is doing negative reinforcement, and one is more effective in the long-term than the other.  Remember this when raising your kids!

References
Kravitz AV, Tye LD, & Kreitzer AC (2012). Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nature neuroscience PMID: 22544310