Reproducible studies may not generate reliable individual differences

The scientific process relies on the ability to replicate findings. This is as true in psychology as in any other discipline. If findings can be reliably replicated, researchers can draw theory-changing conclusions from relatively few data points. But all is not well, and psychology has been dealing with the famous “replication crisis.” Recently a very […]

Continue Reading

Are the times a-changin’? Reporting before and after the 2015 statistical guidelines

“Progress is not possible without deviation.” — Frank Zappa The ways by which psychological science deals with methodological problems are many. There are bottom-up approaches such as peer-reviewed papers and workshops given by methodologists who advocate particular types of changes. There are also top-down approaches, such as funding agencies requiring data sharing or journals requiring […]

Continue Reading

Putting p’s into lmer: mixed-model regression and statistical significance

Since Herb Clark published his famous “Language as a fixed effect fallacy” in 1973, there has been a slow realization that standard techniques, such as ANOVA, are the wrong tools for the jobs that most psychologists tackle. The basic problem is that most psychological questions involve generalization beyond a sample of people and beyond a […]

Continue Reading

Online data collection: The good, the better, and the advantages

This is the first in a series of posts on online data-collection. The popularity of collecting behavioral data online continues to rise. The reasons are many: ease of getting large numbers of participants, relatively low cost, and access to a more diverse population. Early concerns that online data collection is inherently unreliable are gradually evaporating, […]

Continue Reading

From scatter (plot) to statistical perception: you can see a lot by looking

Yogi Berra once famously said that “You can observe a lot by watching”. Yogi Berra observed and said a lot of things, but this line has a lot going for it. The idea that information can be gathered by “just looking” entered statistics many decades ago. For example, John Tukey, one of the 20th century’s […]

Continue Reading