The NIH has posted a new version of Case 18.

If that sentence means nothing to you, you might want to visit my post from last week, “Basic research can be open and transparent without being a clinical trial” in which I summarized the problem with the NIH’s plan to label much of human behavioral basic research as “clinical trials”. In an effort to explain the new definition of “clinical trial”, they posted a set of “case studies”. The idea was that you could look at these and conclude, “My study is like Case N so it is/is not a clinical trial. Responding to criticism of those cases, NIH has just issued a revised set.

There are lots of interesting changes but, for present purposes, the central case for us is Case 18. It had seemed to define almost anything we would do as a clinical trial. Thus, we were very interested when Case 18 became Cases 18a-f. Are we out of the woods? Read on—this is a response that I have posted to the Open Mike web page:

The new set of variants of Case 18 represent a valiant effort to clarify the distinction between what is and is not a “clinical trial”. This is interesting and tricky work and I don’t claim to have worked through all the implications. However, I do not think that the new cases solve the fundamental problems with this approach to the issue of registering and reporting human subject research. I will illustrate the problem by using Cases 18a and 18c to show that the same experiment can be a clinical trial under one case and not under the other.

Before I do that, however, let me make it clear that I (and, I think, most of my colleagues) are NOT opposed to registering and reporting basic behavioral research. We are concerned with a set of unintended consequences, discussed elsewhere, of trying to force the basic science peg into the clinical trials hole.

Turning to Cases 18a and 18c: Here is a sample experiment for purposes of illustration. We will use working memory because that is the example used in 18 a and c.  I will show my observers four colored squares. I will cover them up and then ask the observers to specify the color of one of those squares from memory.  On some trials, I will ask the observers to count backwards by 3s during the working memory task. So, I have two tasks or conditions: working memory in isolation and working memory with a secondary task. A reasonable guess would be that the secondary task will interfere with working memory.

First, you might ask yourself if you want this experiment classified as a clinical trial.

Under Case 18a, this is not a clinical trial because “The purpose of administering these measures is not to modify a health-related outcome.” I am doing two tests and looking at the answers. 18a imagines a study with “various cognitive performance measures (e.g., working memory tasks)”. We have two working memory tasks. This is not a clinical trial.

Under 18c, the same experiment is a clinical trial because it is designed to “to enhance or interfere with cognitive performance”. The differences between the conditions “will alter cognitive task performance and associated brain activity. “ Indeed, Case 18c would classify as a clinical trial any cognitive experiment with an independent variable (unless, I suppose, it produced a perfect null result and, thus, did not “alter cognitive task performance”). As such, 18c leaves us in the same position as the original Case 18. It sweeps vast portions of basic human behavioral research into the clinical trial category.

My sample experiment would not be a clinical trial under any common usage.  However, I understand that may not be the point here. This policy serves other goals. NIH funded my hypothetical study. My hypothetical significance section on that grant said that we need to understand working memory in order to look for early signs of dementia (or something like that). I used the tax payers’ money and my observers’ time to run this study. It is reasonable to argue that I should register and report this study. But note, I would have written the same Significance sections for the “observational” Case 18a version and the experimental Case 18c version. This makes it hard to see why I should not register the observational study. Indeed, it is a bit hard to know why I shouldn’t register the hypothetical version that I might do with mice who I would have argued are an animal model of dementia.

Let’s add one more variation on this theme. Suppose, I give the same working memory test to every child in a big, hypothetical observational study. I collect socio-economic data and I correlate working memory score with SES. Presumably, that is a very clear, Case 18a, observational study. But why, for purposes of the reporting requirement, is the interference produced (hypothetically) by low SES different from the interference produced by counting backwards by 3s?

The more I think about this issue, the more I come to believe that there is no definition that divides basic human behavioral research into studies that should be registered as clinical trials and those that need not be registered. What to do?

• You could define “health-related” or “intervention” in a manner that moved nearly all of basic research out of the category of studies needing registrations.
• You could assert that all NIH funded human behavioral work needs to be registered and you could create an appropriate pathway to do that (perhaps by working with something like Brian Nosek’s “Open Science Framework”).
• You could do both. Define our research as basic research. Define clinical trials as clinical trials. Then tell us: Clinical trials register on this path (and follow the policies on clinical trials)—Basic research register on this path (and follow the policies for basic human research).

The basic behavioral science community is committed to openness and transparency and has been working on these issues for years. We are eager to work with you on a national system of registration and results reporting that meets the interests of all stakeholders, including scientists engaged in basic discovery research in humans.