Your mind is not YOUR mind

Sorry about the light posting these past couple of weeks, I went through an ultra busy phase.  I’ll start reviewing some social neuroscience research again tomorrow, but I thought I’d try a quick post about how intimately linked the mind and environment are.

Let’s start by asking ourselves what we mean by our mind.  Generally, we can reduce this to our brain, right?  Synapses fire, neurons compute and we think and interact with the world.  Most neurons receive input via electrical or chemical interactions with other neurons.  But not all of them do!  Clearly, I’ve been stressing the role of neurohormones and other peptides and how they relate to the brain; but the brain interfaces with and receives input from the whole body, so in a way the brain and the body are only somewhat distinct.  The body is a kind of fuzzy extension of the brain.  Our brain also receives direct input from the environment; the light hitting our eyes, the sound hitting our ears, etc.  One has to realize that our mind cannot exist without the input to the brain from the outside world.  This is one reason why projects like the Blue Brain are somewhat silly.

This philosophy is, I think, called the Extended Mind.  And this isn’t some wacky theoretical idea that will never affect you; we’re actually going through a technological phase that will radically reshape our extended mind.  CNet tries to give a good example of that:

Google, in essence, becomes a part of you. Imagine Google playing a customized audio commentary based on what you look at while on a tourist trip and then sharing photo highlights with your friends as you go. Or Google taking over your car when it concludes based on your steering response time and blink rate that you’re no longer fit to drive. Or your Google glasses automatically beaming audio and video to the police when you say a phrase that indicates you’re being mugged.

The article ends by being more than a bit silly.  But we need to focus on this part here.  We’ve put our memories on paper so long that we forget that we have an external memory (external hard drive, if you will).  These memories aren’t really well integrated into our minds – we have to go out and find and read the book or notes for them to be useful. Now, Google Goggles and such promise to fully integrate external manifestations of our mind, blurring the difference between brain and external world.

These external manifestations currently take the form of factual memories and actions.  But our remembrances are also a creation of our interaction with the world.  The Independent has a good article on how memories are not fully ‘our own’:

One 54-year-old identical twin, on hearing the other claim ownership of the memory of a roller-skating injury from when they were eight or nine, responded indignantly. “Well, that actually happened to me if you don’t mind… I think you’ll find if you think really hard it was me.” The other, yielding ground, eventually responded: “Oh well, I guess we get confused; it happened so long ago.”

Now, from the previous discussion, it should be clear that memories being ‘our own’ doesn’t truly make sense because our whole mind is extended outside of our brain.  Still, the idea that our memories are almost totally unreliable, that they can be fabricated and based on what we hear other people say?  That we can share memories?  That sounds crazy and a bit disturbing (to me at least!).  But it’s true: our minds are not our own.  They are a combination of our brain, our body, and our physical and social environment.  There is truth to the saying that no man is an island.

The best way to extort an extortionist is to be fair

One of the most popular games in the study of cooperation is the iterated prisoner’s dilemma.  It is a game that lets players cooperate or defect, with the most beneficial strategy overall being both cooperating, but the best for a single player is to defect while the other player cooperates.  The most famously successful strategy is tit-for-tat: cooperate if your partner cooperated last turn, and defect otherwise.  Two tit-for-tat players will converge onto harmonious cooperation and maximize their reward, while a single tit-for-tat player will avoid being conned into cooperating with a persistent defector.

William H. Press and colleague Freeman Dyson (!) have found a new solution to the iterated prisoner’s dilemma.  Their paper has been covered in detail well elsewhere  and has some very good commentary by the authors, so I won’t spend a ton of time explaining it.  Basically, if you know this strategy you should be able to find a strategy where you can set your average score arbitrarily; similarly you can arbitrarily set your opponents’ score.  By combining these two strategies, you get what they call the ‘extortionate’ strategy, where you try to extort as much as possible from your opponent.

This set of strategies has two parameters; one of these parameters (“\chi“) measures how much you want to extort from your opponent.  The other (“\phi“) is a bit unclear, but I think we might be able to (kind of) give one intuition a bit later on.  An interesting point to note is that the tit-for-tat strategy is one case of this class of extortionate strategies; when the chi parameter is set to its minimum, indicating fairness instead of total extortion, and the mysterious phi parameter is set to its maximum, you get the tit-for-tat strategy.

I was curious, what happens when a bunch of extortionate strategies get together and duke it out?  What’s the best way to extort an extortionist?

Let’s look at the math of the strategy; feel free to skip this next paragraph if you wish.  Here, p_{cd} for example is the probability that a player should cooperate if on the previous turn they cooperated (“c”) and their opponent defected (“d”).

p_{cc}(\chi,\phi) = 1 - \phi (\chi - 1) \frac{cc-dd}{dd - cd}

p_{cd}(\chi,\phi) = 1 - \phi(1 + \chi \frac{dc-dd}{dd-cd})

p_{dc}(\chi,\phi) = \phi (\chi + \frac{dc-dd}{dd-cd})

p_{dd}(\chi, \phi) = 0

The traditional payoff is (3,0,5,1) so we can simplify to:

p_{cc}(\chi,\phi) = 1 - 2\phi (\chi - 1)

p_{cd}(\chi,\phi) = 1 - \phi(1 + 4\chi)

p_{dc}(\chi,\phi) = \phi (\chi + 4)

And if we want fairness set \chi to 1 (we’ll come back to this later).

p_{cc}(\chi,\phi) = 1

p_{cd}(\chi,\phi) = 1 - 5\phi

p_{dc}(\chi,\phi) = 5\phi

p_{dd}(\chi, \phi) = 0

\phi has allowed values between 0 and 1/5.

Okay, back from the math!  We can figure out what the best strategies are by using a genetic algorithm.  Basically, every generation, extorters compete against each other many times and the top 20% are selected to breed the next generation.  But it’s not always a perfect copy; there is a 3% mutation rate to allow novel strategies to be introduced.  Let’s see what the average reward across time is (and, err, divide by 1000):

Aha!  Something happened there at generation ~350!  If you run the genetic algorithm for longer, it stays at this value.  It seems pretty clear that there’s one strategy that’s superior to the others.  And in fact, it’s the fairness strategy: \chi=1 dominates all the others in this model!  In other words, even if you are trying to extort as much as possible from other players, the best extortionate strategy against other extorters is to be perfectly fair!

But we don’t get back pure tit-for-tat; there’s that messy \phi parameter to worry about.  At the end of 20000 generations, let’s see what the distribution of this value is between its minimum (=0) and maximum (=1, tit-for-tat):

It’s all over the place!  If you rerun the simulation again and again,you get a different distribution of these values, but they always seem to be >.3 and never settle at 1 (tit-for-tat)!  We can measure how diverse the distribution is by the entropy of possible states:

What this is basically showing is that after its initial random set of values, the distribution of strategies oscillates up and down around ~2-3 bits, or something like 4-8 strategies of relative importance.  Sometimes one will start to be more successful against others, sending diversity slowly down, until other strategies evolve against it sending diversity back up.  But fairness always wins.

(As an interesting side-note: the paper provides a formula for estimating expected reward when two strategies compete; when you two strategies with \chi=1, I get a singularity (“infinity”?)…am I doing something wrong or what’s going on…?)

So what is this mystery \phi parameter?  If you have pure tit-for-tat, you can get into alternating defect-cooperate cycles, something less beneficial than everyone cooperating all the time.  By adding this new parameter, maybe you can push each other into that beneficial cycle of cooperation.  That would say that \phi represents a search or exploration parameter.  My intuition for this strategy is that it has two parameters; one represents fairness and the other represents sociality.  Although fairness is best, exploring your options and understanding your opponent is also critical…to being an extortionist.

Update: See this post which is much more informative than mine!  It explains all…

Reference

Press WH, & Dyson FJ (2012). Iterated Prisoner’s Dilemma contains strategies that dominate any evolutionary opponent. Proceedings of the National Academy of Sciences of the United States of America, 109 (26), 10409-13 PMID: 22615375

“Social” reward

We’ll file this paper under “things I kind of wish I didn’t know”.  Apparently, the Syrian hamster really likes to hang out in areas smeared with female hamster vaginal secretions.  So, I guess these hamsters will be sniffing around their cage like, oh this smells like vagina!  I’ll hang out here!  Oh but that area has a much more vagina-like smell, maybe I should move over there?

In case you’re wondering, the paper also investigates whether testosterone mediates this behavior and it doesn’t.  Good to know?  I guess poor grad students are going to have to continue collecting hamster vagina juice and smearing it all over the place.

That’s science.  I’ll just leave you with a picture of a hamster, so you can ponder what he’s thinking about right now.

Photo from

Economic incentives and social behavior

When studying decision-making in neuroscience, experimenters like to have participants be rewarded with money – or units of juice or ‘points’ or suchlike.  Although this may seem like a natural way to measure decisions, we have to step back and ask ourselves whether using this as a basis for reward will affects decision-making in anyway.  Looking around, I found a recent review paper by Bowles and Polania-Reyes that examined how explicit economic incentives change motivation.  Even though it is meant for economists, it has good things to think about for everyone interested in decision-making, motivation, and interpersonal behavior.

Their motivation for this review is clearly set up by a great anecdote.  Before early 2001, Boston firemen were given unlimited paid sick days, trusting them not to abuse it.  In 2001 this policy was replaced with a 15-sick day limit, with penalties for those who went over.  What happened?  Firemen were ten times as likely to call in sick on Christmas and New Years Day, and the total number of sick days claimed more than doubled.  What had been a social privilege instead became an economic transaction.

Rewards are unlikely to be represented in the brain in a purely one-dimensional manner; not everything can be converted equivalently to ‘money’ in some way.  To wit:

Economists know that money is the perfect gift – it replaces the giver’s less well-informed choice of a present by the recipient’s own choice. But when the holidays come around few economists give money to their friends, family and colleagues. This is because we also know that money cannot convey thoughtfulness, concern, whimsy, or any of the other messages that non-monetary gifts sometimes express. A gift, we know, is more than a transfer of resources; it is a signal about the giver and her relationship to the recipient, and money changes the signal.

How we feel about money, reward, and trust is also socially contingent; Bowles and Polania-Reyes report two experiments in societies in Africa, Asia, and Latin America which showed that individuals from more market-integrated societies gave more in the Ultimatum Game.  But it’s not just that they are socially contingent; each offer of resources also sends a social message beyond the purely monetary one.  A low offer can indicate lack of trust, a lack of respect, or a wide range of other things.  The point is that monetary rewards will always have a social effect.  Since monetary offers are in essence social in nature, offers of money for behavior can have side effects on morality; being asked by society to perform an act for money may cause an individual to act more immoral because society has sent the message that morality is unimportant.  And the same offer of money for a behavior can have totally different social meanings depending on the culture:

The fact that fines often work more as messages than as incentives poses a problem for the sophisticated planner because the same intervention may bear radically different messages in different cultures. Bohnet and her co authors implemented a Trust Game in which in one treatment the investor had the option of reducing the payoffs of trustees who betrayed their trust (Bohnet, Herrmann, Al-Ississ, et al. (2010)). Compared to the treatment in which this socalled “revenge” option was not available, when they had the revenge option a substantially larger fraction of Saudi investors trusted their partner, while a substantially smaller fraction of American investors trusted. Making trust more incentive compatible thus had diametrically opposed effects in the two cultures.

Economists like to use revealed preference as a measure of desire or utility.  However, money is not just a reward to individuals, it also contains information and will have side effects on behavior.  This review paper is meant to guide policy makers on how to best provide incentives, with the point that the perceived intent of the incentive is just as important as the incentive itself.  For neuroscience, the point we should take is that we have a lot more to think about when we try to understand decision-making in all its messy glory.

Reference
Samuel Bowles, & Sandra Polania-Reyes (2011). Economic incentives and social preferences: substitutes or complements?  Journal of Economic Literature DOI: 10.1257/jel.50.2.368

Photo from

Genes for savings behavior

The genoeconomics revolution is on!  Kind of.  Via Evolving Economics, a recent paper described how savings across life are explainable genetically.  Using twins, roughly one-third of the variance is explained by shared genes, in line with the genetic heritability of other behavioral traits..  This shouldn’t be remotely surprising.  Not only do they have another similar paper, but so do other people.  Evolving Economics points out one interesting (though again unsurprising) part of the study, though:

The evidence that parental influence fades out for older subjects and disappears by age 45, compared to the relatively constant genetic effects, is interesting. The break down of effects by age is not a regular feature of studies such as these (it comes at the cost of sample size). The authors write:

Our interpretation of this evidence is that social transmission from parents to their children affects children’s savings behavior early on in life, but unlike genetic effects, parenting does not have a lifelong impact on an individual’s savings behavior. These results are broadly consistent with research in behavioral genetics which has found a significant effect of the common family environment in early ages on, e.g., personality, but also shown that such effects approach zero in adulthood

The really interesting thing, though, should be: what genes are responsible for these behaviors?  Are risk-taking and overall savings rate related to the same genes?  How do these genes interact with the environment?  A quick search reveals a relationship between 5-HTTLPR (serotonin transporter; how much serotonin is in the body) as well as DRD4 (dopamine D4 receptor, a D1-like receptor that is mostly expressed in prefrontal cortex, iirc) with economic risk-taking.

But these papers are, hopefully, proof of principle for the economic community.  If papers like this can garner some influence, maybe a broad behavioral economics community can arise that studies these genetics.  God knows, listening to the first author of this paper talk makes it evident how much biology he needs to learn.

Learning: positive and negative

Reward and punishment operate through two very different pathways in the human brain.  The general idea is that these two types of learning – positive and negative – operate through different unique types of dopamine receptors.  The D1 receptors (D1R) are generally ‘positive’ receptors, while the D2 receptors (D2R) are ‘negative’.  Specifically, D1Rs generally tend to increase the concentration of CamKII and D2Rs decrease it; this means that they are going to have opposite effects on downstream pathways such as receptor plasticity, intrinsic voltage channels, etc.

How are the D1 and D2 pathways distinct in terms of learning?  The hypothesis has been that in striatal projection neurons, D1R expressing medium spiny neurons (dMSNs) mediate reinforcement and D2R expressing indirect pathway neurons (iMSNs) mediate punishment.  Kravitz et al expressed channelrhodopsin selectively in dMSNs and iMSNs so they could use light to activate only one type of neuron at a time.  They figured that the striatum would be a good place to start looking for the effects of these neurons.  After all, it is a primary site of reinforcement and action selection (also, they probably tried a few other places and didn’t get great results…?).  These transgenic mice were then placed in a box with two triggers, one of which would stimulate the light and the other would do nothing.  So the mice are in this box, and able to turn on and off their neurons if they want to.  I wonder how that feels?

When the mice were able to activate their D1R (positively-reinforcing) neurons, they were much more likely to keep pressing the trigger.  The D2R (negatively-reinforcing) mice were more likely to press the other trigger.  But that’s not all!  By the third day, the effects of activating the D2R pathway had worn off – they no longer cared about the effect.  You can see this on the graph to the left, where 50% is chance.  The preference for the D1R pathway persisted, however.  Even on short time scales of 15 – 30 seconds, the mice kept their preference for stimulating D1R reward cells over D2R aversion cells.  In the figure to the right, this is seen with YFP being a control (it should have no effect); whereas activating the dMSN pathway over the first 30 seconds always is different than activating YFP, the iMSN pathway only shows a (statistical) different over the first 15 seconds.

The authors conclude by saying that that the dMSN pathway is sufficient for persistent reinforcement, while iMSNs are sufficient for transient punishment.  This is a nice finding; that the D1R pathway really is doing some positive reinforcement and that the D2R pathway is doing negative reinforcement, and one is more effective in the long-term than the other.  Remember this when raising your kids!

References
Kravitz AV, Tye LD, & Kreitzer AC (2012). Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nature neuroscience PMID: 22544310

Oxytocin, the complicated hormone

Over at the Notes&Theories blog, there is a good post about the complicated role of oxytocin.  Oxytocin is commonly called the ‘love hormone’, a striking simplification that should immediately set off your Overly Anthropomorphized radar.  Meadow voles are the promiscuous cousins of the monogamous prairie voles:

But oxytocin and vasopressin are released in brains of all mammals, not just those that are monogamous. The differences between species have nothing to do with how much oxytocin or vasopressin is released, but rather they depend on exactly where these hormones act. Vasopressin and oxytocin act only at specific receptors – and in the brain, these receptors are only made in certain places…Then they modified a harmless virus in such a way that it carried the code for making prairie vole vasopressin receptors, and injected it into a small part of the brains of male meadow voles. This part of the brain now began to make vasopressin receptors where none had been before – and the meadow voles began to behave like prairie voles, forming strong attachments to their current sexual partners.

Like so many things in the nervous system, oxytocin and vasopressin have a multitude of possibly contradictory roles that are determined by when they are released, where their receptors are expressed, and what else is released at the same time.  A burst of dopamine+oxytocin and a burst of oxytocin+testosterone will surely have different meanings in the brain!  And with highly plastic gene expression, what they mean will vary between individuals.

Investigating Wall Street through neuroscience

There’s a good collection of links on metafilter about decision-making in finance, and includes a bit about John Coates, who investigates the role of testosterone and cortisone in decision-making.  This article from Businessweek about how this ex-trader got interested in neuroscience:

The more Coates learned, the more he became convinced that traders were, as he put it, “a clinical population.” The stimuli of a trading floor triggered chemical changes in people’s brains, emotionally whipsawing them. During the tech bubble, he recalls, “People just really slipped their moorings: They were motor-mouthing, they weren’t sleeping, they were on this high. It was initially reasonable to assume it was cocaine, but I don’t know many traders that do that. There was something going on, it was just incredibly noticeable, and I realized that at times I had also felt that way.”

I think this is true not just about financial traders, but about anyone in a high-stress occupation during good times.  I know I get this way when everything is working perfectly in lab.

With enough victories, though, testosterone can reach levels that make the animal act foolishly. He picks fights he can’t win, tries to claim too much territory, and roams around in the open where predators might pick him off. A human being on a trading floor might take massive, risky bets on the strength of the American housing market or on U.S. corporate bonds. One of the traders Coates studied went on a hot streak, making twice his average profit-and-loss ratio for five days in a row. By the end of it his testosterone levels had risen 80 percent. If Coates had followed the trader long enough, he believes, there was a good chance “he would be irrationally exuberant and blow up.”

For losers, the effect is the opposite: The stress and worry of losing money cause the endocrine system to flood the body with cortisol, which makes people afraid to take even favorable bets. In the wake of a financial crisis, it’s not just Wall Street traders who suffer from this, but anyone making decisions about money, whether it’s an employer who balks at hiring or a bank officer leery of making a loan even when the Federal Reserve is offering her free money to do so.

Obviously, testosterone and cortisol have wider effects, and the effects they have are contingent on a lot of other environmental variables.  Studying testosterone and cortisol on the trading floor will elucidate just one (important!) aspect of their function.

Old bees get a new lease on life (through glutamate!)

Have you ever heard a story about an elderly person who seems surprisingly fine and with it in the outside world, but is then transferred to a nursing home where they quickly slide from their mental peak?  Have you ever stayed at home all day, playing video games (ahem) and feeling a bit sluggish only to go back to mentally stimulating work and feel more alert?  No matter what people say, our work is our life.

Honeybees spend the first two or three weeks of their life as nurses, taking care of the young, tending to the queen, building out and cleaning the hive.  When they get older, they get reassigned to a job outside the hive as a forager.  Now they have to search out nectar and pollen and live in the dangerous outside world.  They are quick to die off as the stress of the outside environment and downright intense physical work causes them to age.  Not only are there physical effects, but mental ones, too: their ability to learn and associate is impaired.

But not all is lost for these bees!  Sometimes disaster falls a hive and more nurses are needed.  When this happens, some forager bees return to become nurses.  Baker et al studied these bees to see how returning to the hive affected them.  Although in some ways the returned bees looked like their foraging compatriots, in terms of learning and memory they were identical to their younger nursing brethren.  They had a new lease on life!

Some of these returned nurses did better than others.  Baker et al looked at what proteins were differentially expressed between these two groups, and the data pointed to proteins that affected physical structure (alpha-tubulins), stress and cell maintenance, and neuronal functioning and signaling.  One of the most abundantly different proteins was the glutamate transporter homologous to EAAT2.  Glutamate is the primary neurotransmitter in the brain, and is the basis for the most common form of long-term learning.  The glutamate transporter will remove glutamate from the extracellular space, so different amounts of glutamate transporter will change the concentration.  This means that cells will be generally more or less excitable and will have different levels of plasticity.

There are clearly a couple of problems with this study which can basically be labeled statistics.  Do the bees learn better because they have returned to nursing?  Or do they return to nursing because they are the better bees?  This is selection bias.  Also, if the bees are learning better, is it because of this change in proteins?  Or were those differences in proteins there before they returned, and something totally different has changed?

What the paper may provide evidence for, though, is the social brain hypothesis.  This hypothesis suggests that the reasons humans got smarter is because we lived in social groups, and the fittest individual was the one that was smartest at dealing with the social group.  Perhaps the bees that return need to be the smartest because they have to return and deal with a social environment, a possibly more intellectually demanding environment.  These bees have more to keep track of, a variety of other bees to placate.  Not only does your job affect you, but so does your social environment.

Well, it’s something to think about at least.

References

Baker et al (2012). Age-related learning deficits can be reversible in honeybees Apis mellifera
Experimental Gerontology DOI: 10.1016/j.exger.2012.05.011

Photo from

The basic unit of human relationship is the pair

Posting has been light (ie, nonexistent) because I’ve been preparing for/been at a conference.  While I was gone, I read the book Escape From Camp 14, about someone who not only was born and raised in a North Korean political prison, but also managed to escape from it.  Not only was the story interesting, but there was a lot of good stuff in it relating to how the human brain interacts with the environment.  Take this:

“It was in the pairs that the prisoners kept alive the semblance of humanity,” concluded Elmer Luchterhand, a sociologist at Yale who interviewed fifty-two concentration camp survivors shortly after liberation.

Pairs stole food and clothing for each other, exchanged small gifts, and planned for the future.  If one member of a pair fainted from hunger in front of an SS officer, the other would prop him up.

“Survival … could only be a social achievement, not an individual accident,” wrote Eugene Weinstock, a Belgian resistance fighter and Hungarian-born Jew who was sent to Buchenwald in 1943.

The death of one pair often doomed the other.  Women who knew Anne Frank in the Bergen-Belsen camp said that neither hunger nor typhus killed the young girl who would become the most famous diarist of the Nazi era.  Rather, they said, she lost the will to live after the death of her sister, Margot.

There was a bit more to the quote, but the book has already been returned to the library and I only have Google Books to quote from.  The point here is that the pair-bond seems to be the basic unit of human relationship.  This shouldn’t be too surprising; humans are generally monogamous on the order of a few years at a time.  But this pair-bonding isn’t solely romantic, but also extends to friendships.  What we know about pair-bonding comes primarily from work on prairie voles who are a uniquely monogamous species of vole.  This monogamy (or should I say, “monogamy”) connect to the neurohormone oxytocin.  Oxytocin seems to stimulate pair-bonding and social recognition.  It unfortunately gets a lot of press as the ‘love hormone’, even though it can have some darker effects.

Escapees from North Korea also seem to share certain personality traits that make it hard for them to prosper as refugees: they have a hard time holding down a job, they refuse to take personal responsibility for their actions, they are exceedingly suspicious of others, etc.  Not too surprising, obviously.  But taken together this illustrates certain facts about how the brain interacts with the environment to create personality: some things are hardwired in pretty solidly, like pair-bonding.  Others are plastic and interact with the environment, albeit in stereotyped ways.  In order to fully understand the brain, we will have to understand how interactions with the environment create neural mechanisms – the neuroscience of ecology.