There are many lessons in life that we have all had to learn the hard way: don’t cram the dishwasher full of plates, arrive at the airport well before your flight time, and maybe take two trips instead of carrying a huge armload of laundry up the stairs in one go. When we make mistakes like this, we try to learn from them so that we don’t end up tripping on the stairs.
Science is no exception. When developing, setting up, and running behavioral experiments, there are many issues that scientists run into at some point or another. Perhaps the conditions in the experiment weren’t properly randomized, or maybe there was some variable that didn’t get saved out in the data file, or worse—the software crashed partway through and the data were lost. Every scientist working on behavioral studies runs into these kinds of problems sooner or later. An important (but rarely taught) skill in carrying out scientific studies is being able to understand and anticipate potential issues before they arise.
These kinds of issues were the main motivation for the Barbosa, Stein, Zorowitz, Niv, Summerfield, Soto-Faraco, and Hyafil (pictured below) recent paper in Behavior Research Methods, a Psychonomic Society journal. Their paper, titled “A practical guide for studying human behavior in the lab” is a how-to for scientists carrying out behavioral research. The authors’ guide was inspired by what they wished they knew when they were just starting out. By outlining the best practices for behavioral studies, the goal is to help others avoid common mistakes or pitfalls.
Instead of writing a primer on research within a specific sub-discipline (e.g., visual psychophysics, memory research), Barbosa, Stein, and colleagues wrote a guide that can be broadly applied across many research topics and methods. This is especially important when we consider that behavioral studies are among the most powerful tools that we have for understanding the mind! Having a step-by-step framework for conducting this research sets up scientists to better understand the mechanisms of how the brain processes information.
Their guide includes a workflow for the whole process, shown below, starting with developing an idea for an experiment. For example, if you can’t think through the arguments you will make in your paper or explain why your experiment is important, maybe it’s time to go back to the drawing board. Carefully think about aspects of your experiment design, select your sample size, and think about the tools you’ll use. Once you obtain ethics approval, spend time piloting and fine tuning your study (all of this may take longer than you might think!).
Then, consider pre-registering your study, so that you have a document that outlines your hypotheses and how you’re going to analyze your data once it comes in. Make sure the experiment is optimized from the participant’s point of view as well (are they allowed frequent breaks?). Once you’ve finished collecting your data, analyze your data to answer your original question. A model that is based on your design can give you greater insight into the underlying mechanisms of the processes you’re examining. Last but not least, share your data and code.
If all of this sounds like a lot to keep track of, that’s because behavioral studies have many elements to consider, and why having a roadmap is so essential. Even for scientists who have been doing behavioral research for a long time, some elements are easy to forget, and having a guide can help more senior researchers train their students to be good scientists. Although the authors’ goal of this guide was to cover the things they wished they knew when they were starting out, their roadmap is a helpful tool across all career stages, and something that behavioral scientists can all benefit from.
Featured Psychonomic Society Article
Barbosa, J., Stein, H., Zorowitz, S., Niv, Y., Summerfield, C., Soto-Faraco, S., & Hyafil, A. (2022). A practical guide for studying human behavior in the lab. Behavior Research Methods, 10.3758/s13428-022-01793-9.
