Should small labs do fMRI experiments?

Over at Wiring The Brain, Kevin Mitchell asks whether it is worth it for small labs to do fMRI:

For psychiatric conditions like autism or schizophrenia I don’t know of any such “findings” that have held up. We still have no diagnostic or prognostic imaging markers, or any other biomarkers for that matter, that have either yielded robust insights into underlying pathogenic mechanisms or been applicable in the clinic.

A number of people suggested that if neuroimaging studies were expected to have larger samples and to also include replication samples, then only very large labs would be able to afford to carry them out. What would the small labs do? How would they keep their graduate students busy and train them?

I have to say I have absolutely no sympathy for that argument at all, especially when it comes to allocating funding. We don’t have a right to be funded just so we can be busy. If a particular experiment requires a certain sample size to detect an effect size in the expected and reasonable range, then it should not be carried out without such a sample. And if it is an exploratory study, then it should have a replication sample built in from the start – it should not be left to the field to determine whether the finding is real or not….Such studies just pollute the literature with false positives.

At the end of the day, you are doing rigorous science or you are not.

I do have a silly little theory – which I keep meaning to write up – on the economics of science. In some cases, it may be worth doing underpowered studies as a cost-effective way to generate hypotheses. However, this depends on the cost of the experiment – and fMRI seems to fall way too far into the “too expensive per data point” field to be worth it.