Monday open question: What have you read that inspired you in science?

In a twitter discussion of inspirational scientists, I realized that a more interesting question was whether other scientists had particular papers or books that had profoundly inspired them.

For very young me, the answer would clearly have been Jurassic Park. This made me want to do science to the extent that several friends and I found a microscope and had one of us (not me) pick their nose until it bled so that we could look at the DNA in the blood, with grand visions of cloning near at hand. Needless to say, this did not work – it turns out that you can’t see DNA under a 40x microscope.

More near at hand, the text that continues to fascinate and inspire me is a book by Joshua Epstein and Robert Axtell, Growing Artificial Societies. This presents a simulation of behaving agents in a land called the Sugarscape. Epstein and Axtell then try to show what happens as these agents live and die in this brutal land. The idea that one could simulate the rules of life and use it to understand how living creatures create societies was breathtaking to me – much more so than something so abstract as Conway’s Game of Life. To this day, that is why I want to understand the clockwork neuroscience that drives organisms as they interact with each other and the ecological environment.

A suggestion by someone else was David Marr‘s Vision: A computational investigation into the human representation and processing of visual information. Marr died of leukemia tragically young, but his sketch of how to attack the problem vision is still considered fundamental. Here are some selections from the book and reading it one is left 35 years later marveling at the intellect behind it.

So what readings have influenced you? I want to read them!

Unrelated to all that, 6/26

Visualizing algorithms [Absolutely stunning]

Algorithms are a fascinating use case for visualization. To visualize an algorithm, we don’t merely fit data to a chart; there is no primary dataset. Instead there are logical rules that describe behavior. This may be why algorithm visualizations are so unusual, as designers experiment with novel forms to better communicate. This is reason enough to study them.

But algorithms are also a reminder that visualization is more than a tool for finding patterns in data. Visualization leverages the human visual system to augment human intellect: we can use it to better understand these important abstract processes, and perhaps other things, too.

National geographic photo contest

Mayana Soora Thiruvizha

Kahneman vs Gigenzer

In an increasingly complex and specialised world, Gigerenzer preaches a gospel of greater simplicity. He suggests that the outcome of decisions of any complexity – a complexity of, say, trying to organise a successful picnic or greater – are impossible to accurately predict with any mathematical rational model, and therefore more usefully approached with a mixture of gut instinct and what he calls heuristics, the learned rules of thumb of any given situation. He believes, and he has some evidence to prove it, that such judgments prove sounder in practice than those based purely on probability.

(see also: Instinct can beat rational thinking)


The goal of the NeuroElectro Project is to extract information about the electrophysiological properties (e.g. resting membrane potentials and membrane time constants) of diverse neuron types from the existing literature and place it into a centralized database.

Statistical modeling: the two cultures

The statistical community has been committed to the almost exclusive use of data models. This commitment has led to irrelevant theory, questionable conclusions, and has kept statisticians from working on a large range of interesting current problems. Algorithmic modeling, both in theory and practice, has developed rapidly in fields outside statistics. It can be used both on large complex data sets and as a more accurate and informative alternative to data modeling on smaller data sets.

Statistics vs. Machine Learning

So it’s pretty clear by now that statistics and machine learning aren’t very different fields. I was recently pointed to a very amusing comparison by the excellent statistician — and machine learning expert —Robert Tibshiriani. Reproduced here…

Hah. Or rather, ouch! I had two thoughts reading this. (1) Poor statisticians. Machine learners invent annoying new terms, sound cooler, and have all the fun. (2) What’s wrong with statistics? They have way less funding and influence than it seems they might deserve.

Machine learning done wrong

As pointed out in my previous post, there are dozens of ways to solve a given modeling problem. Each model assumes something different, and it’s not obvious how to navigate and identify which assumptions are reasonable. In industry, most practitioners pick the modeling algorithm they are most familiar with rather than pick the one which best suits the data. In this post, I would like to share some common mistakes (the don’t-s). I’ll save some of the best practices (the do-s) in a future post.

The oculus rift is going to be a game-changer…behold our sedentary future

I started him off easy with a walk around an art gallery and then graduated to a simple undersea stroll. He was blown away by it. He has never been so amazed by a new technology he told me and this is a guy whose favorite programs as a kid were radio shows. I asked if he was ready to try something more intense and he said yes so I put him in the roller coaster. We recorded the results.

The anarcho-surrealists who took over Reykjavik

A glance at the most important campaign promises of the Best Party is more than enough to highlight the audacity of Reykjavik’s voters. They were promised free towels at swimming pools, a polar bear for the zoo, the import of Jews, «so that someone who understands something about economics finally comes to Iceland», a drug-free parliament by 2020, inaction («we’ve worked hard all our lives and want to take a well-paid four-year break now»), Disneyland with free weekly passes for the unemployed («where they can have themselves photographed with Goofy»), greater understanding for the rural population («every Icelandic farmer should be able to take a sheep to a hotel for free»), free bus tickets. And all this with the caveat: «We can promise more than any other party because we will break every campaign promise.»

Is the BRAIN initiative ‘set up to fail’?

We need a National Neurotechnology Initiative (NNTI) that requires $2B in yearly funding. Expanding the BRAIN Initiative to NNTI, we will achieve the goals of curing diseases and understanding brain function.  Mapping the brain is just one step in the process. We need to supplement the NIH investment with funding from each major government agency. Interdisciplinary research is an integral part to success in this national endeavor. Additionally, we need to create a National Coordinating Office that will oversee the investments from other agencies to synchronize the research efforts. Without continued coordination, we will lose sight of what we cherish most – our health, our minds, and our future.

66 facts about the English language

6. A growlery is a place you like to retire to when you’re unwell or in a bad mood. It was coined by Charles Dickens in Bleak House (1853).

28. In the 18th century, a clank-napper was a thief who specialized in stealing silverware.

30. 11% of the entire English language is just the letter E.

56. In mediaeval Europe, a moment was precisely 1/40th of an hour, or 90 seconds.

Cyborg grape vines?

How USA vs Portugal played on twitter

USA vs Portugal

How often do men think about sex? Probably about once per day

If we believe the stats, thinking about sex every seven seconds adds up to 514 times an hour. Or approximately 7,200 times during each waking day. Is that a lot? It sounds like a big number to me, I’d imagine it’s bigger than the number of thoughts I have about anything in a day. So, here’s an interesting question: how is it possible to count the number of mine, or anyone else’s thoughts (sexual or otherwise) over the course of a day?

Neuroscience History: Otto Loewi

According to Loewi’s account, one night in 1921 he fell asleep while reading. He then had a dream in which he visualized an experiment that could put an end to the debate over how nerves communicated with one another. He woke up in the middle of the night, scribbled some notes about this potentially groundbreaking experiment, and then fell back to sleep. To his great frustration, however, when he awoke again he couldn’t read the notes he had written.

Abandoned robots in university waste dump

Meet the father of digital life

Barricelli’s experiments had an aesthetic side, too. Uncommonly for the time, he converted the digital 1s and 0s of the computer’s stored memory into pictorial images. Those images, and the ideas behind them, would influence computer animators in generations to come. Pixar cofounder Alvy Ray Smith, for instance, says Barricelli stirred his earliest thinking about the possibilities for computer animation, and beyond that, his philosophical muse. “What we’re really talking about here is the notion that living things are computations,” he says. “Look at how the planet works and it sure does look like a computation.”

Signal from noise: an economic interpretation of the Neyman-Pearson lemma

Rather than actually doing math, let’s think like economists. Picking the set R gives us a certain benefit, in the form of the power Q(R) , and a cost, tP(R) . (The ts term is the same for all R .) Economists, of course, tell us to equate marginal costs and benefits. What is the marginal benefit of expanding R to include a small neighborhood around the point x ? Just, by the definition of “probability density”, q(x) . The marginal cost is likewise tp(x) . We should include x in R if q(x)>tp(x), or q(x)/p(x)>t . The boundary of R is where marginal benefit equals marginal cost, and that is why we need the likelihood ratio and not the likelihood difference, or anything else. (Except for a monotone transformation of the ratio, e.g. the log ratio.) The likelihood ratio threshold t is, in fact, theshadow price of statistical power.

Naming overactive PNAS contributors

Scientist ejected from classical music concert for attempting to crowd surf 

But Dr Glowacki, a Royal Society Research Fellow, was so overcome during the ‘Hallelujah Chorus’ he began lurching from side to side with his hands raised and whooping before attempting to crowd-surf, witnesses claimed.

Two day old zebrafish larvae  (°o°)

Animals in zoos are often on antipsychotics

Zoos just plain drive animals crazy:

In the mid-1990s, Gus, a polar bear in the Central Park Zoo, alarmed visitors by compulsively swimming figure eights in his pool, sometimes for 12 hours a day. He stalked children from his underwater window, prompting zoo staff to put up barriers to keep the frightened children away from his predatory gaze.* Gus’s neuroticism earned him the nickname “the bipolar bear,” a dose of Prozac, and $25,000 worth of behavioral therapy…

Many animals cope with unstimulating or small environments through stereotypic behavior, which, in zoological parlance, is a repetitive behavior that serves no obvious purpose, such as pacing, bar biting, and Gus’ figure-eight swimming. Trichotillomania (repetitive hair plucking) and regurgitation and reingestation (the practice of repetitively vomiting and eating the vomit) are also common in captivity. According to Temple Grandin and Catherine Johnson, authors of Animals Make Us Human, these behaviors, “almost never occur in the wild.” In captivity, these behaviors are so common that they have a name: “zoochosis,” or psychosis caused by confinement…

Drugs are another common treatment for stereotypic behavior. “At every zoo where I spoke to someone, a psychopharmaceutical had been tried,” Braitman told me. She explained that pharmaceuticals are attractive to zoos because “they are a hell of a lot less expensive than re-doing your $2 million exhibit or getting rid of that problem creature.” But good luck getting some hard numbers on the practice. The AZA and the Smithsonian National Zoo declined to be interviewed for this article, and many zookeepers sign non-disclosure agreements. Braitman also found the industry hushed on this issue, likely because “finding out that the gorillas, badgers, giraffes, belugas, or wallabies on the other side of the glass are taking Valium, Prozac, or antipsychotics to deal with their lives as display animals is not exactly heartwarming news.” We do know, however, that the animal pharmaceutical industry is booming. In 2010, it did almost $6 billion in sales in the United States.

It is like a bored person locked in a room, pacing around the room while waiting to get out.. If you want to see this kind of repetitive behavior in action, here are some short movies illustrating them. Repetitive behaviors are also often seen in people with developmental disabilities, especially autism, where external stimulation is hypothesized to be a motivating reason. Here is a review of some of the neural mechanisms of stereotypy.


What is the goal of the nervous system, part 2 (a twitter story)

My question from Monday spawned some good discussion on twitter (none, alas, on the blog). What is the goal of the nervous system? Well, a lot of people thought it was pretty simple:

As evidence for how important movement is for having a brain:

Yup, when a sea squirt decides to stop moving it just sits there… and consumes its own brain (for some definitions of ‘brain’).  The video at the top, as suggested by Chris Maidan, is a great explanation of the reasoning behind the idea that the brain is for movement. Yet, despite Daniel Wolpert explicitly saying that the reason we have a brain is for movement… he actually makes an argument for something else that came up in the twitter discussion:

Another good alternative:

And to throw in a curveball:

But the answer that I like the best:

I have some thoughts that I’m writing up, though that is taking some time. Let me take a step back, though, and reword what my question was originally intended to be (though clearly in my sleep-deprived state I did a poor job of describing it).

As others suggested above, the overall goal of any part of a biological system is to maximize the transmission of genetic material (by one mechanism or another) from one generation to the next. Yet when examining the nervous system on a more local level that can be hard to notice; the visual system, especially the early stages, are there to extract information about visual scenes; the auditory system is there to extract information about sound; the spinal chord is there to (mostly) generate movement. If you don’t believe me, look at the peripheral sensory nerves, at the retina, at the cochlea: all part of the nervous system, and on a local level they are guided by local goals. It is not until you get to more internal structures that general ideas of value or desire are found.

And we can write this out mathematically; sensory neurons are often said to be maximizing the Shannon mutual information between their response and the visual world which we can write in a very simple way I(neural response;visual stimulus) with plenty of experimental support. In other words, the visual neurons represent as much information about the visual scene as is possible.

But given that we know that the ultimate goal of the nervous system is to enhance reproductive success – why isn’t that included in the equation? Where would we need to include something like that to have a good understanding of the function of some area of the visual system? Is a proxy like ‘utility’ good enough?

The man who asked the simplest question

“Claude Shannon answered a question that no one else was even asking.”

This is a nice little video essay on Claude Shannon; even as someone bathed in information theory day in, day out, I found it interesting. Sadly, it ends with a standard #einsteincomplex.

If you haven’t read it yet, James Gleick’s The Information is well worth reading… or at least the first three quarters is. As important as Shannon was, it’s worth remembering what Hamming had to say about him:

When you are famous it is hard to work on small problems. This is what did Shannon in. After information theory, what do you do for an encore? The great scientists often make this error. They fail to continue to plant the little acorns from which the mighty oak trees grow. They try to get the big thing right off. And that isn’t the way things go…

When you go to a new field, you have to start over as a baby. You are no longer the big mukity muk and you can start back there and you can start planting those acorns which will become the giant oaks. Shannon, I believe, ruined himself. In fact when he left Bell Labs, I said, “That’s the end of Shannon’s scientific career.” I received a lot of flak from my friends who said that Shannon was just as smart as ever. I said, “Yes, he’ll be just as smart, but that’s the end of his scientific career,” and I truly believe it was.

via kottke

Is it okay to eat fish if they don’t have any feelings? (Updated)


When a scientific paper begins its list of keywords with “fish cognition”, you know you’re in for a good read.

Culum Brown is tired of people eating fish, and he’s not going to take it anymore. Fish, he says, are smarter than you think. We need to cast off our view of them as dumb slimy creatures and recognize what they can really accomplish.

First, we have to realize that though they may have separated from us evolutionarily more than half a billion years ago, they are not ‘primitive’; it is not as if they stopped evolving. If a fish had stopped evolving could it do this:


That’s right – this bad boy, the cutlips minnow, gathers stones to build a mound to attract mates. And these aren’t the only ‘fishy masons’ (as Brown calls them). The jawfish builds itself a wall in front of its burrow, searching for rocks that fit together like lock and key, leaving only a hole just big enough for them to scurry through.  The Rockmover Wrasse builds itself a stone house every night. It also hunts in pairs, one member pushing rocks around so that the other can watch carefully in order to grab any prey that is revealed.

Fish also have sophisticated social intelligence. Take, for instance, the Cleaner Wrasse. They occupy stations – which I’ll generously call a storefront – where other client fish come by to have parasites and dead skin removed by the Cleaner. Brown points out that the fish have the option of several cleaners, so it is important to have a good reputation; should a Cleaner accidentally bite a client, they’ll chase after their fleeing clients and give them a good back rub to make up for it. They also prioritize certain customers over others. Model that, economists.

cleaner wrasse

Many other fish can recognize multiple individuals, and can count the number of fish in a group at a glance.

Some fish also use tools: a number of species use rocks to break open shellfish, or glue their eggs to leaves that they can them drag around as they go about their errands.

I actually came away impressed from this paper; I hadn’t known most of these fishy facts. Yet despite how smart fish are, people will still eat them; after all, they’re pretty okay eating piggies (they’re pretty smart). What matters more than any kind of intellectual empathy is a anthropomorphic one. After all, which would be more okay to eat: a really dumb monkey or a really smart (but ugly) fish?

(My biggest take-away from this is not to eat a Wrasse; those guys are pretty smart, and have a much larger brain for their size than you’d expect.)

via Marginal Revolution

Update: Ed Yong happened write an article today on this very subject! Lionfish are strategic, social hunters:

During night dives, Lönnstedt often saw teams of two to four lionfish positioning themselves around schools of smaller fish and using their fan-like pectoral fins to corral their prey “like fishermen with their nets”. The hunters then take turns to dart into the school of prey, picking them off one at a time…

“Fish social behaviour is much more complex than previously assumed. Moving away from a stimulus of major interest—prey—in order to actively recruit a partner that is initially out of sight suggests planning and awareness of objects that [they can’t see].”…But in these pursuits, the two partners are merely hunting next to each other and relying on their complementary abilities. The lionfish are doing something more impressive: they’re working together to corral their prey and taking turns to go in for the kill.


Brown C (2014). Fish intelligence, sentience and ethics. Animal cognition PMID: 24942105

What is social behavior, and how has that changed?

Consider someone praying, alone, in front of an altar.  Is this a social behaviour?  Most psychologists working before 1950, certainly 1920, would probably have answered ‘yes’; the activity is demonstrably being shaped by, and takes the form it does, because of that person’s previous social experiences and group membership.  It seems exceedingly unlikely that someone who had never been immersed in the traditions of the church would find themselves praying at this alter, in this physical position…

If you were to ask experimental social psychologists and neuroscientists the same question today, we would find the opposite answer most frequently given: praying is not a social behaviour.  The reason for this is that, within today’s experimental psychology and neuroscience, the social is characterised by two features.  Firstly, within contemporary thinking, the social refers toobjects of cognition (the things which our cognitions are directed towards) and not forms of cognition (the particular shape of those cognitions).  Cognitions, or behaviours, which are present, or altered, by group membership (such as praying) are not social under this framework.  Instead, a social cognition is simply one related to the understanding of other people in the immediate vicinity…

An exception to the rule that we are inherently social creatures is believed to be found in autism.  As described in the introduction, social abnormally is taken to be a, or even the, primary symptom in autistic spectrum conditions.  At the most general level, I think we can easily show that the description of autism as social disorder is reliant upon the contemporary construction of the social, outlined above.  In psychology’s first sense of the social, where praying is social, individuals with autism are demonstrably able: as noted earlier in this essay, many individuals with autism take part in one of the most significant self-advocacy movements of all time.  People with autism are clearly able to join groups, have their behaviours shaped by membership of those groups, and so forth.  It is only when the social is understood as being related to interpersonal conduct that autism becomes conceivable as social disorder residing within an individual who has difficulty with, for example, feeling empathy.

A fantastic essay on the intersection of our ideas of sociality, how those ideas have changed, and autism.

Monday thought/open question: What is the goal of the nervous system? (Updated)

In systems neuroscience, we like to say that the goal of the visual system is to extract as much information about the world as possible. We start in the retina with points of light; those points are correlated (look around you: the color of one part of the visual world is often very similar to the color right next to it). So then the next set of neurons represent sudden changes in the brightness (ON/OFF neurons) to decorrelate. In the first stage of visual cortex, we find neurons that respond to edges – areas where you could put a several ON/OFF receptive fields in a row (see above). The responses of the visual neurons get successively more unrelated to each other as you go deeper into cortex – they start representing more abstract shapes and then, say, individual faces. But our guiding principle through this all is that the visual neurons are trying to present as much information about the visual world as possible.

But now let’s look at the nervous system from a broader view. What is it trying to accomplish? If we were economists, we might say that the nervous system is trying to maximize the ‘utility’ of the animal; an ecologist might say that it is trying to maximize the reproductive success of an animal (or: of an animal’s offspring, or its genes).

Is this a reasonable view of the ‘goal’ of the nervous system? If so, where do the goals of the input and the output meet? When do neurons in the visual system of the animal begin representing value, or utility, at some level? Is there some principle from computer science that has something to say about value and sensory representation?

Update: There was a lot of discussion on twitter, which I have partially summarized here.

21st century advances in art: optical illusions


Never let it be said that science has contributed nothing to art! The study of optical illusions not only gives us crazy cool images to look at, but tells us about who we are and how we function in the world. Contemplate that.

I somehow forgot to link to the 2014 Optical Illusions finalists, which is apparently a thing, but there you are. There are some pretty cool optical illusions in there.

Of course, you could just watch the new OK Go music video instead, which is one long set of optical illusions.

Business Insider has an explanation of how many of the illusions work and made us some pretty GIFs while they were at it. Go read!

two heads illusion


Unrelated to all that, 6/20 edition

Why do some animals forgo reproduction in complex societies? The importance of ecological and social constraints

Are you sure that you have hands? Plot twist: I don’t because I’m currently trapped in Roko’s Basilisk

Beliefs about willpower determine the impact of glucose on self-control. Your beliefs affect more things than you’d think

What do we mean when we talk about “AI”? A lot of different things

How does the brain speak to itself? Christof Koch and Gary Marcus on neural coding.

How to respond to criticism. Step 1: Give up on all of your goals immediately.

How to turn a “good” proposal into an “excellent” one. See also the link above

Formula for linear equations by country; y=kx + n??? (with caveats)

tumblr_n70zfxMZAp1s6c1p2o1_1280 (1)


It turns out that the trick with the lamb brains is to treat them as a spread and an accompaniment to bread. They have a difficult texture—“in between foie gras and fish sperm”—and you can’t overcook them (they fall apart) or let them dry out (the results are apparently too horrifying for words).’

Take me somewhere that glitters. Don’t let go.

This is the best game ever invented.