Can we improve human intelligence through training? Generations of researchers have tackled this apparently simple question, and yet, after approximately a century of efforts, the malleability of intelligence remains controversial. The potential payoff is high—imagine what it would mean if we could actually train intelligence—but the road to conclusive evidence on the matter appears to be long and rough.

A few years ago there was a new surge of hope, driven by studies using training on working-memory tasks and showed improved IQ scores in trained groups compared to control groups. Measures of working-memory capacity have been shown to statistically predict IQ scores better than any other construct from cognitive psychology, supporting the idea that the capacity of working memory is the main limiting factor of our intellectual faculties. Therefore, if we could expand our working-memory capacity through training, that should make us smarter. In the meantime, several dozen studies have put this idea to the test. Frustratingly, we still don’t know whether training working memory leads to improvements of intelligence. Training people on a working-memory task reliably improves performance on the trained tasks, but transfer effects on intelligence test scores have been highly variable across studies: Now you see it, now you don’t.

Two recent meta-analyses have attempted to achieve closure by pulling together all available data on a focused question: Jacky Au and her colleagues evaluated the effects of training with one working-memory paradigm, the n-back task, on measures of fluid intelligence in young and middle-aged adults. Julia Karbach and Paul Verhaeghen investigated the effects of a broad range of trainings, among them working-memory trainings, on measures of fluid intelligence in old adults. Both meta-analyses concluded that there is a small to medium, significant transfer effect. Happy End!

Or not? The happiness was not to last long: A new article in the Psychonomic Bulletin and Review by Monica Melby-Lervåg and Charles Hulme questions the conclusions of both the Au et al. and the Karbach & Verhaeghen meta-analysis. Melby-Lervåg and Hulme raise three main points of critique: First, the authors of the previous two meta-analyses omitted some studies that fit their inclusion criteria. Second, they did not control for differences between training and control groups at pre-test. Third, they did not distinguish rigorously enough between studies with a passive control group—which does the pretest and the posttest but no training in between—and an active control group, which receives a placebo training in between pre- and posttest. Placebo trainings sometimes involve training the same task as the experimental group but with a low level of working-memory load, and sometimes involve different tasks presumably unrelated to working memory, such as a knowledge quiz. After fixing these three limitations in their own meta-analyses, Melby-Lervåg and Hulme conclude that there is evidence for transfer in the studies using a passive control group, but not in those studies using an active control group.

A second critical evaluation of the study by Au and colleagues, recently published in the Psychonomic Bulletin and Review, came to the same conclusion: Michael Dougherty, Toby Hamovitz, and Joe Tidwell re-analyzed the data of Au et al. with Bayesian statistical methods, which allowed them to gauge the strength of evidence for the null hypothesis as well as the alternative hypothesis. Although they analyzed only the group differences at posttest, without control for pre-test differences—an omission that Melby-Lervåg and Hulme criticized—they found substantial evidence for the alternative hypothesis of positive transfer in the studies with passive control groups, but evidence in favor of the null hypothesis of no transfer in the studies with active control groups.

The same result had already been reported by Au and colleagues for an analysis that split the sample of studies by the kind of control group. Hence, despite differences in the analysis methods, the three meta-analyses focusing on n-back training come to the same descriptive result: There is evidence for transfer in the subgroup of studies with a passive control group, and no evidence for transfer – and even evidence against transfer—in the studies with an active control group. When all studies are analyzed together, disregarding the kind of control group, all four meta-analyses—including the one by Karbach and Verhaeghen—report evidence for a modest effect of transfer.

The different conclusions that these author groups draw therefore come down to a difference in interpretation: The skeptics maintain that studies with active and with passive control groups must be looked at separately. They argue that an active control group is the gold standard for any training study, because only active control groups can be equated with the training group with regard to motivation, expectance of cognitive gains, degree of social interaction with the experiments, and other potentially confounding variables. Dougherty and colleagues conclude that the transfer effects in the studies with passive control groups reflect a placebo effect; Melby-Lervåg and Hulme even go as far as suggesting that journals should no longer publish training studies without an active control group. This argument has much in its favor, but it overlooks the problems that an active control group entails: What is an adequate placebo training? Such a training should be as motivating and challenging as the working-memory training, it should be plausible to participants that it could improve their intellectual abilities, but it should not tax working memory. Such a placebo training is hard to find, because there are few, if any, non-trivial cognitive demands that do not to some extent tax working memory. It is conceivable that the active control training also improves working-memory capacity to some extent, leading to an underestimation of the effectiveness of a working-memory training when it is compared to the active control group.

One way forward could be to ask whether an active control group improves more from pretest to posttest on the transfer tasks than a passive control group. This would be expected if there was a placebo effect, and it would also be expected if the active control training improved working-memory capacity to some degree. Few studies so far included both an active and a passive control group, so there is little experimental evidence speaking directly to this question, but Au and colleagues compared the pre-post gains of active and passive control groups across different studies. They made the puzzling observation that passive control groups gained more than active control groups between pretest and posttest on the transfer task. It appears that there are differences between the existing studies with passive and those with active control groups that cannot be explained as placebo effects. It would be helpful if future studies included both an active and a passive control group.

Those who have not left the battlefield disheartened will begin to wonder: Should we care? Even if working-memory training has an effect on intelligence, that effect appears to be so small that it is hard to detect even with meta-analytic methods. Would such a small effect be relevant? I think it would be highly relevant for two reasons. First, from a purely scientific point of view, it makes a large difference for our conceptualization of intelligence whether or not it can be improved by training at all: Despite our enormous learning powers and the pervasive plasticity of our brains, there might be at the core of our mental abilities a mechanism that is impenetrable to experience. Second, from a practical point of view, a small beneficial effect of existing trainings could potentially be much amplified if we found out which features of a training are effective, and learned how to maximize the effectiveness of trainings. Much reason, therefore, to continue the long march, and to be friendly to each other despite strongly held convictions, to make the road a bit less rough for everyone on it.