Can we even understand what the responses of a neuron ‘represent’?

tl;dr:

  • Deep neural networks are forcing us to rethink what it means to understand what a neuron is doing
  • Does it make sense to talk about a neuron representing a single feature instead of a confluence of features (a la Kording)? Is understanding the way a neuron responds in one context good enough?
  • If a neural response is correlated with something in the environment, does it represent it?
  • There is a difference in understanding encoding versus decoding versus mechanistic computations
  • Picking up from an argument on twitter
  • I like the word manticore

What if I told you that the picture above – a mythical creature called a manticore – had a representation of a human in it? You might nod your head and say yes, that has some part of a human represented in it. Now what if I told you it had a representation of a lion? Well you might hem and haw a bit more, not sure if you’d ever seen a grey lion before, or even a lion with a body that looked quite like that, but yes, you’d tentatively say. You can see a kind of representative lion in there.

Now I go further. That is also a picture that also represents a giraffe. Not at all, you might say. But I’d press – it has four legs. It has a long body. A tail, just like a giraffe. There is some information there about what a giraffe looks like. You’d like at me funny and shrug your shoulders and say sure, why not. And then I’d go back to the beginning and say, you know what, this whole conversation is idiotic. It’s not representative of a human or a lion or a giraffe. It’s a picture of a manticore for god’s sake. And we all know that!

Let’s chat about the manticore theory of neuroscience.

One of the big efforts in neuroscience – and now in deep neural networks – has been to identify the manticores. We want to know why this neuron is responding – is it responding to a dark spot, a moving grating, an odor, a sense of touch, what? In other words, what information is represented in the neurons responses? And in deep networks we want to understand what is going on at each stage of the network so we can understand how to build them better. But because of the precise mathematical nature of the networks, we can understand every nook and cranny of them a bit better. This more precise understanding of artificial network responses seems to be leading to a split between neuroscientists and those who work with Deep Networks on how to think about what neurons are representing.

This all started with the Age of Hubel and Wiesel: they found that they could get visual neurons to fire by placing precisely located circles and lines in front of an animal. This neuron responded to a dark circle hereThat neuron responded to a bright line there. These neurons are representing the visual world through a series of dots and dashes.

And you can continue up the neural hierarchy and the complexity of stimuli you present to animals. Textures, toilet brushes, faces, things like this. Certain neurons look like they code for one thing or another.

But neurons aren’t actually so simple. Yes, this neuron may respond to any edge it sees on an object but it will also respond differently if the animal is running. So, maybe it represents running? And it also responds differently if there is sound. So, it represents edges and running and sound? Or it represents edges differently when there is sound?

This is what those who work with artificial neural networks appreciate more fully than us neuroscientists. Neurons are complex machines that respond to all sorts of things at the same time. We are like the blind men and the elephant, desperately trying to grasp at what this complex beast of responses really us. But there is also a key difference here. The blind men come to different conclusions about the whole animal after sampling just a little bit of the animal and that is not super useful. Neuroscientists have the advantage that they may not care about every nook and cranny of the animal’s response – it is useful enough to explain what the neuron responds to on average, or in this particular context.

Even still, it can be hard to understand precisely what a neuron, artificial or otherwise, is representing to other neurons. And the statement itself – representation – can mean some fundamentally different things.

How can we know what a neuron – or collection of neurons are representing? One method that has been used has been to present a bunch of different stimuli to a network and ask what it responds to. Does it respond to faces and not cars in one layer? Maybe motion and not still images in another?

You can get even more careful measurements by asking about a precise mathematical quantity, mutual information, that quantifies how much of a relationship there is between these features and neural responses.

But there are problems here. Consider the (possibly apocryphal) story about a network that was trained to detect the difference between Russian and American tanks. It worked fantastically – but it turned out that it was exploiting the fact that one set of pictures were taken in the sunlight and another set was taken when it was cloudy. What, then, was the network representing? Russian and American tanks? Light and dark? More complex statistical properties relating to the coordination of light-dark-light-dark that combines both differences in tanks and differences in light intensities, a feature so alien that we would not even have a name to describe it?

At one level, it clearly had representations of Russian tanks and American tanks – in the world it had experienced. In the outside world, if it was shown a picture of a bright blue sky it may exclaim, “ah, [COUNTRY’S] beautiful tank!” But we would map it on to a bright sunny day. What something is representing only makes sense in the context of a particular set of experiences. Anything else is meaningless.

Similarly, what something is representing only makes sense in the context of what it can report. Understanding this context has allowed neuroscience to make strides in understanding what the nervous system is doing: natural stimuli (trees and nature and textures instead of random white noise) have given us a more intimate knowledge of how the retina functions and the V2 portion of visual cortex.

We could also consider a set of neurons confronted with a ball being tossed up and down and told to respond with where it was. If you queried the network to ask whether it was using the speed of the ball to make this decision you would find that there was information about the speed! But why? Is it because it is computing the flow of the object through time? Or is it because the ball moves fastest when it is closed to the hand (when it is thrown up with force, or falls down with gravity) and is slowest when it is high up (as gravity inevitably slows it down and reverses its course)? Yes, you could now read out velocity if you wanted to – in that situation.

There are only two ways to understand what it is representing: in the context of what it is asked to report, and if you understand precisely the mechanistic series of computations that gives rise to the representation – and it maps on to ‘our’ definition.

Now ask yourself a third question: what if the feature were somehow wiped from the network and it did fine at whatever task it was set to? Was it representing the feature before? In one sense, no: the feature was never actually used, it was just some noise in the system that happened to correlate with something we thought was important. In another sense, yes: clearly the representation was there because we could specifically remove that feature! It depends on what you mean by representation and what you want to say about it.

This is the difference in encoding vs decoding. It is important to understand what a neuron is encoding because we do not know the full extent of what could happen to it or where it came from. It is equally important to understand what is decoded from a neuron because this is the only meaningfully-encoded information! In a way, this is the difference between thinking about neurons as passive encoders versus active decoders.

The encoding framework is needed because we don’t know what the neuron is really representing, or how it works in a network. We need to be agnostic to the decoding. However, ultimately what we want is an explanation for what information is decoded from the neuron – what is the meaningful information that it passes to other neurons. But this is really, really hard!

Is it even meaningful to take about a representation otherwise?

Ultimately, if we are to understand how nervous systems are functioning, we need to understand a bit of all of these concepts. But in a time when we can start getting our hands around the shape of the manticore by mathematically probing neural responses, we also need to understand, very carefully, what we mean when we say “this neuron is representing X”. We need to understand that sometimes we want to know everything about how a neuron responds and sometimes we want to understand how it responds in a given context. Understanding the manifold of possible responses for a neuron, on the other hand, may make things too complex for our human minds to get a handle on. The very particulars of neural responses are what give us the adversarial examples in artificial networks that seem so wrong. Perhaps what we really want is to not understand the peaks and troughs of the neural manifold, but some piecewise approximations that are wrong but understandable and close enough.

Other ideas

  • Highly correlated objects will provide information about each other. May not show information in different testing conditions
  • What is being computed in high-D space?
  • If we have some correlate of a feature, does it matter if it is not used?
  • What we say we want when we look for representations is intent or causality
  • This is a different concept than “mental representation
  • Representation at a neuron level or network level? What if one is decodable and one is not?
  • We think of neurons as passive encoders or simple decoders, but why not think of them as acting on their environment in the same way as any other cell/organism? What difference is there really?
  • Closed loop vs static encoders
  • ICA gives you oriented gabors etc. So: is the representation of edges or is the representation of the independent components of a scene?
Advertisements

Monday Open Question: what did you need to do to get a neuroscience job in 2018?

See last year’s post. As always, if you are a postdoc looking for a faculty job I maintain the neurorumblr with crowdsourced information on open jobs + helpful information. You should also add yourself to The List which lets faculty search committees contact you. If you are on a search committee, feel free to email me (with some evidence that you are a faculty member on a search committee…) to access The List.

First, I put up a poll on the neurorumblr twitter account to see what conceptions people had about being hired in neuroscience. I’m going to give answers a bit out of order. First, what percent of faculty job applicants do you think are male?

The median was 50-60% (roughly even) with a heavy tail toward 60-70%+. On the List last year, there were 65 postdocs of which 28 were female and 37 were male – so ~57% of the applicants (in my dataset) were male last year, a little lower than the year before (62%).

How about faculty HIRES? The estimate was about the same, but weighted more toward the hire end (60%+). Last year, 57% of the people on my hired list were male. This year it is a tick larger at 61.6%.

One of the big surprises in last year’s analysis (to naive me) was how geographically clustered that got hired were.

I expanded the analysis this year to look at where faculty hires got their PhDs from. This is much more geographically dispersed compared to postdocs. In a subset of 69 faculty hires where I was somewhat confident of PhD-granting institution, only four institutions had more than two people hired – with UC San Diego (go Tritons!) and UPenn having the most at four. I’m guessing a lot of this is statistical noise and will fluctuate widely from year to year, but that will take more annual data.

How about postdoc institutions? Again, it shows an absurd NYC+ area dominance (institutions located in and around NYC, NYC+ category below includes people in NYC category).

Alright, now on to specifics. Which model organisms did hires use? There were (roughly):

(23) Mouse
(15) Human
(12) Rat
(4) Monkey
(2) Drosophila

In terms of publications, successful faculty hires had a mean H-index of 11.1 (standard deviation ~ 4.4, statistically no different from the previous year).

How many had Cell/Nature/Science papers? 22.6% (14/62) had a first- or second-author CNS paper, 53.22% (33/62) had a first- or second- author Nature Neuroscience/Neuron/Nature Methods paper, and 64.5% (40/62) had at least one of either of these.

If you look at the IF through time, you can see a bump in the year prior to applying, suggesting that applicants get that “big paper” out just before they apply.

There’s a broader theory that I’ve heard from several people (outlined here) that the underlying requirement is really the cumulative impact factor. I have used the metric described in the link, where the approximate impact factor is taken from first-author publications and second-author publications are discounted 75% (reviews are ignored). Here are the CIFs over the past 8 years:

Note that postdocs from NY+ area institutions do not have significantly different H-index (10.42), cumulative IF (36.6), or CNS (3/19), NN/Neuron/NM (7/19), or both (9/19).

What about comparisons by gender? There appears to be a slight cumulative IF difference that is largely driven by differences in grad school publications – but I would need more data before I say something definitive here.

 

People seem to think that there are tons of internal hires out there but the data definitely does not bare that out. I found only 3/62 instances of someone who was hired by the university they either got their PhD from or did their postdoc at. However, there is some local geography at stake: I looked at all the hires made by NY+ area institutions and of these hires (6) every one either came from another NY+ area institution or a Philadelphia institution.

NeuroRumblr, 2018 – 2019

Quick announcement –

I’ve refreshed the NeuroRumblr for the 2018 – 2019 job season. If you are a postdoc looking for an academic job, add yourself to The List so that search committees can reach out to you. Note that I refresh the list yearly, so if you have added yourself in the past you should fill out the form again for the new season. If you are on a faculty committee, please feel free to email me at neurorumblr@gmail.com to gain access to The List (both this year and last year’s). Every year, I have gotten requests for access from every kind of institution across the world. As an aside, if you have been on a search committee that has used the list in the past and have ideas on how to make it more useful, or just have other thoughts, I’d be curious to hear them.

There is a page for labs that are looking for postdocs.

There is a page for labs that are looking for research staff.

There is a page to keep track of neuroscience conferences.

There is a page with collections of advice on being an academic and looking for academic jobs.

There is a twitter account (@neurorumblr) that I occasionally use to make announcements. The account will now automatically tweet, multiple times a day, about jobs that were put on the rumblr the previous day, as well as with upcoming job or conference deadlines. If you are a PI who placed an ad under postdocs or research staff, you can now add your twitter handle and it will tag you when it tweets. If you tweet and tag @neurorumblr, I will usually retweet it – more free advertisement! The twitter account gets a lot of attention and I keep hearing from people who have looked for jobs that they paid close attention to it.

Another reminder that I am looking to identify neuroscientists hired as tenure-track faculty over the previous year. I already have a lot of people on the list! But I know that’s not everyone.

Happy hunting.

Thoughts on freedom of will

Some rough notes from a personal attempt to clarify my own thinking. Consider this a work-in-progress and probably wrong.

Principles underlying feeling of freedom of will (note that this is different from freedom of action or actual agency):

  1. Bayesian updating (distribution of decisions)
  2. State/neural dynamics are unpredictable in the future
  3. Accessibility of internal state
  4. Web of concepts
  5. Feedforward-ish network

What is it that makes someone believe themselves to be free? That they can exercise willpower in pursuit of moral agency?

There are a few concepts from neuroscience that we want to understand for the feeling of freedom in a deterministic world.

The first concept we have to understand is that the brain is a neural network. It has neurons connected to other neurons connected to other neurons. We like to imagine them as step-by-step programs that slowly process information and allow an organism to make a decision but in reality they are vastly interconnected. Still, there is some validity to levels of feedforward-ness! The important thing is that not every neuron has access to the information from every other neuron – and not every ‘layer’, or collection of neurons, has access to information from every other ‘layer’.

The second concept is that the brain is interested in the fundamental unpredictability of the future. There are informational constraints on knowledge about what the internal state of a system will be, in both the sense of the current dynamics and the broader sense of ‘hunger’, ‘thirst’, ‘sleepy’, etc. Each of these modulates the activity of different layers of the brain in a variety of ways, often through neuromodulatory pathways. Further, as a corollary of this and the first concept, different layers have imperfect access to the state of other neurons and other layers.

The third concept is the implementation of ideas and memories as a web of concepts. Smelling an odor can transport you back to the first time you smelled it, with all the associated feelings. No idea exists in isolation but accessing one will inevitably access a variety of others (to a greater or lesser extent). This is foundational to how memories are stored.

The fourth and final concept is the Bayesian Brain, the idea that brains represent probability distributions and compute in terms of inference. No 19th century symbolic thinking for us – we process information in a manner that fundamentally requires us to think of the distribution of possible options or possible futures.

How does this give rise to a feeling of freedom of will? Suppose you were considering whether you were going to perform some action or not. When considering which of various possibilities you will take, there is some set of neurons making this consideration. These neurons do not have access to all of the possible neurons involved in the decision. In other words, these neurons do not know which possible action they will take. Instead they must operate on the set of probabilities for the future state of the network. The operation on this distribution is the feeling of free will – that there is an internal state, or internal dynamics, that are inaccessible from the consideration. These neurons get input from the dynamics and can plausibly provide output (the feeling of willpower).

Imagining the future is similar. Will I have the possibility of choosing freely among many options? Yes, of course, because I am unclear what my future state will be. The probability distribution conditional on external and internal state is not unitary (though it can be with properly powerful external stimuli).

Could I choose to eat bacon and eggs this morning instead of my daily bowl of granola? Possibly; it exists in probability space of possible options. Will I do so? I can sit there and sample my internal state and have some uncertainty about whether I desire that or not. That makes me feel free: I will choose to do so or not do so, unconstrained by outside forces, even though it is entirely deterministic.

Fundamentally, the feeling of freedom is a reflection of probabilistic and uncertain thinking.

Please help me identify neuroscientists hired as tenure-track assistant profs in the 2017-18 faculty job season

Last year, I tried to crowd-source a complete list of everyone who got hired into a neuroscience faculty job over the previous year. I think the list has almost everyone who was hired in the US… let’s see if we can do better this year?

I posted an analysis of some of the results here – one of the key “surprises” was that no, you don’t actually need a Cell/Nature/Science paper to get a faculty job.

If you know who was hired to fill one or more of the listed N. American assistant professor positions in neuroscience or an allied field, please email me with this information (neurorumblr@gmail.com).

To quote the requirements (stolen from Dynamic Ecology):

I only want information that’s been made publicly available, for instance via an official announcement on a departmental website, or by someone tweeting something like “I’ve accepted a TT job at Some College, I start Aug. 1!” If you want to pass on the information that you yourself have been hired into a faculty position, that’s fine too. All you’re doing is saving me from googling publicly-available information myself to figure out who was hired for which positions. Please do not contact me to pass on confidential information, in particular confidential information about hiring that has not yet been totally finalized.

Please do not contact me with nth-hand “information” you heard through the grapevine. Not even if you’re confident it’s reliable.

I’m interested in positions at all institutions of higher education, not just research universities. Even if the position is a pure teaching position with no research duties.

Learn by consuming the brains of your enemies

A few people have sent this my way and asked about it:

In a paper published Monday in the journal eNeuro, scientists at the University of California-Los Angeles reported that when they transferred molecules from the brain cells of trained snails to untrained snails, the animals behaved as if they remembered the trained snails’ experiences…

In experiments by Dr. Glanzman and colleagues, when these snails get a little electric shock, they briefly retract their frilly siphons, which they use for expelling waste. A snail that has been shocked before, however, retracts its siphon for much longer than a new snail recruit.

To understand what was happening in their snails, the researchers first extracted all the RNA from the brain cells of trained snails, and injected it into new snails. To their surprise, the new snails kept their siphons wrapped up much longer after a shock, almost as if they’d been trained.

Next, the researchers took the brain cells of trained snails and untrained snails and grew them in the lab. They bathed the untrained neurons in RNA from trained cells, then gave them a shock, and saw that they fired in the same way that trained neurons do. The memory of the trained cells appeared to have been transferred to the untrained ones.

The full paper is here.

Long and short of this is that there is a particular reflex (memory) that changes when they have experienced a lot of shocks. How memory is encoded is a bit debated but one strongly-supported mechanism (especially in these snails) is that there are changes in the amount of particular proteins that are expressed in some neurons. These proteins might make more of one channel or receptor that makes it more or less likely to respond to signals from other neurons. So for instance, when a snail receives its first shock a neuron responds and it withdraws its gills. Over time, each shock builds up more proteins that make the neuron respond more and more. These proteins are built up by the amount of RNA (the “blueprint” for the proteins, if you will) that are located in the vicinity of the neuron that can receive this information. There are a lot of sophisticated mechanisms that determine how and where these RNAs are built and then shipped off to the place in the neuron where they can be of the most use.

This new paper shows that in these snails, you can just dump the RNA on these neurons from someone else and the RNA has already encoded something about the type of protein it will produce. This is not going to work in most situations (I think?) so it is surprising and cool that it does here! But hopefully you can begin to see what is happening and how the memory is transferring. The RNA is now in the cell, it is now marked in a way that will lead it to produce some protein that will change how the cell responds to input, etc, etc.

One of the people who asked me about this asked specifically in relation to AI. Could this be used as a new method of training in Deep Networks somehow? The closest analogy I can think of is if you have two networks with the same architecture that have been trained in the same way (this is evolution). Then you train a little more, maybe on new stimuli or maybe on a new task, or maybe you are doing reinforcement learning and you have a network that predicts a different action-value pair. Now the analogy would be if you chose a few units (neurons) and directly copied the weights from the first network into the second network. Would this work? Would this be useful? I doubt it, but maybe? But see this interesting paper on knowledge distillation that was pointed to me by John O’Malia.

Controversial topics in Neuroscience

Twitter thread here.

  • Do mice use vision much?
    • They have pretty crappy eyesight and their primary mode of exploration seems to be olfactory/whisker-based
  • How much is mouse cortex like primate cortex?
    • Mouse cortex is claimed to be more multimodal than primate cortex which is more specialized
  • “The brain does deep learning”
    • Deep learning units aren’t exactly like neurons, plus we resent the hype that they have been getting
  • Is there postnatal neurogenesis? Is it real or behaviorally relevant?
    • See recent paper saying it doesn’t exist in humans
  • Brain imaging
    • Does what we see correlate with neural activity? Are we able to correct for multiple comparisons correctly? Does anyone actually correct for multiple comparisons correctly?
  • Bayesian brain
    • Do people actually use their priors? Does the brain represent distributions? etc
  • Konrad Kording
    • Can neuroscientists understand a microprocessor? Is reverse correlation irrelevant?
  • Do mice have a PFC
    • It’s so small!
  • STDP: does it actually exist?
    • Not clear that neurons in the brain actually use STDP – often looks like they don’t. Same with LTP/LTD!
  • How useful are connectomes
    • About as useful as a tangled ball of yarn?
  • LTP is the storage mechanism for memories
    • Maybe it’s all stored in the extracellular space, or the neural dynamics, or something else.
  • Are purely descriptive studies okay or should we always search for mechanism
    • Who cares about things that you can see?!

Updates

  • Does dopamine have ‘a role’?
    • Should we try to claim some unified goal for dopamine, or is it just a molecule with many different downstream effects depending on the particular situation?
  • Do oscillations (‘alpha waves’, ‘gamma waves’, etc) do anything?
    • Are they just epiphenomenon that are correlated with stuff? Or do they actually have a causative role?

What HASN’T Deep Learning replicated from the brain?

The brain represents the world in particular ways. Here are a few:

1. The visual world on the retina

The retina is thought to whiten images, or transform them so that they always have roughly the same average, maximum and minimum (so that you can see in very bright and very dark environments. This was originally shown very nicely in two papers from Atick and Redlich (1990, 1992). Essentially, you want to smooth the visual scene around each point depending on the noise. You get receptive fields that look something like this:

Or more generally this:

A denoising autoencoder – a network that tries to replicate a corrupted image which smooths locally – has neural representations that look similar:

2. The visual world in first order visual cortex

Similarly, if you want to efficiently represent the visual world (once it is denoised) you want to represent things sparsely or independently. This was shown by Olshausen and Field 1996  and Bell and Sejnowski 1997 and is equivalent to doing ICA on natural images. Note that doing PCA on natural images will give you Fourier components.

If you train a Deep Network on ImageNet (AlexNet example below), the filters on the first layer look similar:

3. The auditory world

The best representation of the auditory world is also efficiently encoded. Lewicki 2002 show that if you run ICA on acoustic data you get filters that look like nearly identical to the sounds neurons respond to (wavelet basis functions).

I have not seen a visualization of the first few layers of a neural network that classifies speech (for instance) but I would guarantee it has features that look like wavelets.

4. Spatial cells

Your sense of place in the world is encoded by a series of grid cells – which are a periodic representation of place – and place cells, which are precise locations in space. Dordek et al 2016 showed that non-negative PCA on place cells will give you grid cells. This is similar to the result that PCA on images gives you Fourier components. Note that Dordek et al also use a single-layer feedforward neural network and show that it has a similar property.

It turns out if you train a Deep recurrent network on network navigation task, you get grid cells (once you have assumed place cells).

Other

What else is left? Olfaction is a mess and doesn’t have a coherent coding principle as far as I can tell (the olfactory space is not clearly defined). Mechanosensation (touch) has been hard to define but Zhao et al 2017 can find first-order touch receptive fields with an autoencoder (like with vision). You can get CPGs (oscillatory movement generators) with recurrent neural networks by training an input signal to be associated with a particular sequence of movements. I’m struggling to think of other internal representations that are well understood.

A long-term principle in neuroscience has been that successive layers of the brain are attempting to decorrelate their responses to produce ever-finer features. Tishby and Zaslavsky 2015 suggest that a similar principle applies to Deep Networks: you have a constrained input output and networks are trying to find the representations that encode the most information between input and output given the limited bandwidth that they have (numbers of layers, numbers of units). It should not be surprising that this entails something like different forms of PCA or ICA or other signal-detection framework.

One of the nice things about Deep Networks is that you do not have to explicitly code for this in order to find these features – they are costless in a way. You can train for a particular task – a visually-driven one, a path-driven one, an acoustic-driven one – and these features will just fall out. Not only will these features fall out, but neurons which are deeper in the pathway will also have similar activity. This is a much harder problem and one in which “run PCA again” or “run ICA again” will not give a good answer to.

What other neural representations have we not yet seen in neural networks?

Behold, the behaving brain!

In my opinion, THE most important shift in neuroscience over the past few years has been the focus on how behavior changes neural function across the whole brain. Even the sensory systems – supposedly passive passers-on of perfectly produced pictures of the world – will be shifted in unique ways by behavior. An animal walking will have different responses to visual stimuli than an animal that is just sitting around. Almost certainly, other behaviors will have other effects on the animal.

A pair of papers this week have made that point rather elegantly. First, Carsen Stringer and Marius Pachitariu from the Carandini/Harris labs have gobs of data from when they were recording ~10,000 neurons simultaneously. Marius Pachitariu has an excellent twitter thread explaining the work. I just want to take one particular point from this paper which is that you can explain a surprising amount of variance in the primary visual cortex – and all across the brain – simply by looking at the movement of the animal’s face.

In the figures below, they have taken movies of an animal’s face, extracted the motion energy (roughly, how much movement there is at that location in the video), and then used PCA to find the common ways that you can describe that movement. Using this kind of common motion, they then tried to predict the activity of individual neurons – while ignoring the traditional sensory or task information that you would normally be looking at.

The other paper is from Simon Musall and Matt Kaufman in Anne Churchland’s lab. He also has a nice twitter description of their work. Here, they used a technique that is able to image the whole brain simultaneously (though I am not sure to what depth), though at the cost of resolution (individual neurons are not identifiable but are averaged together). The animals are doing a task where they need to tell the difference between two tones, or two flashes of light. You can look for the brain areas involved in choice, or the areas involved in responding to vision or audio, and they are there (choice, kind of?).  But if you look at where movement is being represented it is everywhere.

The things that you would normally look for – the amount of brain activity you can explain by an animal’s decisions or its sensory responses – explain very little unique information.
This latter point is really important. If you had looked at the data and ignored the movement, you would have certainly found neurons that were correlated with decision-making. But once you take into account movement, that correlation drops away – the decisions are too correlated with general movement variables. People need to start thinking about how much of their neural data is responding to the task the animal is doing and how much is due to movement variables that are aligned to the task. This is really important! Simple averaging will not wash away this movement.

There is a lot more to both of these papers and both will be more than worth your time to dig into.

I’m not sure if you would have noticed this effect in either case if they weren’t recording from massive numbers of neurons simultaneously. This is a brave new world of neuroscience. How do we deal with this massively complex behavioral data at the same time that we deal with massive neural populations?

In my mind, the gold standard for how to analyze this data comes from Eva Naumann and James Fitzgerald in a paper out of the Engert lab. They are analyzing data from the whole brain of the zebrafish as it fictively swims around and responds to some moving background. Rather than throwing up their hands at the complexity of this data and the number of moving pieces what they did was very precisely quantify one particular aspect of the behavior. Then they followed the circuit step by step and tried to understand how the quantified behavior was transformed in the circuit. How did the visual stimuli guide the fish’s orientation in the water? What were the different ways the retina represented that visual information? How was this transformed by the relays into the brain? How was this information further transformed in the next step? How did the motor centers generate the different types of behavior that were quantified?

The brain evolved to produce behavior. In my opinion there is no way to understand the brain – any of it – if you don’t understand the behavior that the animal is producing.

Monday Open Question: How many types of neurons are there in the brain?

How many types of neurons are in the brain? Not just number, but classes that represent some fundamental unit of computation? I tweeted an article about this a couple days ago and (justly) got pilloried for saying it counted classes in the brain rather than in two cortical regions. So what is the answer for the whole brain?

Obviously the answer depends on the brain that you are talking about. In the nematode C. elegans, we know that every hermaphrodite has 302 neurons and every male has 381. I believe these specifically male neurons get pruned in the developmental process if the animal does not become a male. These neurons tend to come in symmetric pairs or quartets, one showing up on each side of the body, so the number of neural ‘classes’ is on the order of 118 – though there is evidence that some neurons can be slightly different between their left and right side (ASEL and ASER, for example). Fruit flies (Drosophila) also show sex-specific neurons, with the genes Fruitless and Doublesex controlling whether certain neurons are masculinized or feminized. So not only are there going to be different classes of neurons in males and females, we know that there are single (or, again, symmetric) neurons that control single behaviors. On the other hand, in the fruit fly retina there are definitely distinct classes of neurons that are tiled across the eye. This should frame our thinking about the number of neural classes – there are classes with large numbers of neurons where convolution is useful (repeating the same computation across some space, such as visual or auditory or even musculature space) but perhaps neural function becomes more specific and class-less once specific outputs are needed.

The fruit fly brain may seem a bit silly, why bother comparing it to us cortical mammals? But adult Drosophila have roughly the same number of neurons as larval zebrafish, a vertebrate animal with a cerebrum that is a popular organism to study in neuroscience. So do we think that the zebrafish has just as many pre-planned neurons as Drosophila? Or is its neural structure somewhat looser, more patterned? I don’t have an answer here but I think it is worth thinking about the similarities and differences in these organisms that have similar numbers of neurons but quite different environmental and developmental needs.

Let’s turn to mammals. The area with the most well-defined number of cell classes is probably the retina. I’m not sure of the up-to-date estimates for number of cell classes but the classic description has two classes (rods/cones) in the input layer of the retina which can be further split depending on the number of colors an animal can see – for instance, humans have S, M, and L cones roughly corresponding to blue, green, and red light. This review roughly estimates that further into the eye there are two types of horizontal cells (first layer), ~12 types of bipolar cells (second layer), ~30 types of amacrine cells (third layer). From other sources we think there is something on the order of ~30 types of retinal ganglion cells, the output from the eye into the brain. Interestingly, this is roughly the same number of defined classes that we think the fruit fly has! But again, there may be species specificity here; something on the order of 95%+ of the output layer of monkey retina is a single cell class. So the eye alone has at least 80 classes of neurons and quite probably more.

The olfactory bulb is probably more complex. In mice, at least, the number of olfactory glomuleri that exist is probably on the order of one or two thousand? Though I would expect that once past this layer the classification will look more like retina or cortex – on the order of tens of subtypes.

 

Now let’s think about the cortex. The paper that inspired this post tried to estimate the number of cell classes by using single-cell RNA-sequencing in mice to identify the transcripts that are present in different cells and then attempts to cluster them into distinct sub-classes. It should be clear up front that the number of clusters you identify (1) may not be categorical but could be continuous between types of neurons and (2) may be different than if you clustered with a different method or with different types of data – functional responses, for instance. The authors in this paper make clear that they certainly find cells that look ‘intermediate’ between their clusters so whatever categories we get may not be very firm. For instance, in the following figure the size of the circles represents the number of cells they identify in a particular cluster and the thickness of the line between two circles is how many cells are intermediate between two clusters.

 

Without getting into too many details, they find that in two distinct anatomical regions they find roughly 50 inhibitory neurons that are common in their transcript types between the regions suggesting that the types of inhibition may be a common, repeated pattern across the brain. However, the types of 50 excitatory neurons were essentially unique to each of the two regions..Chuck Stevens has an interesting paper where he attempts to find lower and upper bounds on the number of possible cell classes in cortex. Let’s say that we accept the tiling principle, that the same types of cells are repeated again and again in a motif:

This argument can be extended to the neocortex. Underneath 1 mm2 of most regions of the primate cortical surface are about 105 neurons — the striate cortex is an exception with twice the number — each of which covers say 0.05 mm2 with its dendritic arbor (assumed to be 0.25 mm in diameter). Twenty neurons with dendritic arbors of this size would be required to cover a square millimetre of cortex, so the upper limit on number of cell types, if each must tile the cortex, is 105/20 = 5000, or an average of 1000 per layer. Now assume that the cortex has 10 times more neurons of each type than required to cover the cortex, a redundancy factor of 10 as guessed above for hippocampus: we thus would have about 100 neuron types per layer. If we believe there are a dozen ganglion cell types, two dozen amacrine cell types, and four dozen different kinds of inhibitory neurons in the CA1 region of hippocampus, 100 cell types per layer of neucortex seems like a reasonable number – not good news for the micromodelers.

Let’s update this estimate; we think that there may be 25 excitatory cell types per region. I don’t actually know off-hand the percentage of mouse cortex that these two regions encompass (a motor region, ALM, and a visual region, VISp) but let’s say they are roughly 10% of the cortical area each (this could be grossly wrong so feel free to correct me). We then might believe that cortex has on the order of 25 * 10 ~ 250 excitatory cell classes and ~50 inhibitory cell classes. Does this feel right? 300 classes for all of cortex?

But the cortex is the minority of the number of cells in the brain – the majority is in a single structure, the cerebellum. I don’t know of an estimate of the number of neural classes but a structure that is known for its beautiful tiling neurons seems more likely to have a fair bit of structure in its number of cell classes. What would we estimate here? Something similar to a primary sensory area, with ~50-100 cell classes? Something more something less?

And what about other subcortical regions in the brain like hypothalamus that are more directly responsible for specific behavior? Should we expect many thousands of distinct subtypes for each of the behaviors or something more patterned?

Tell me where I’m wrong.