“(…) a substantial proportion of research effort in experimental psychology isn’t expended directly in the explanation business; it is expended in the business of discovering and confirming effects” —Cummins (2000). I am contributing to this digital event from a theoretician’s perspective. I thought I’d be upfront about this to set the right expectations. Theoretical perspectives […]
Statistics and Methodology
I have a confession to make. I like the idea behind preregistration a lot, and over the last few years I’ve been making a concerted effort to start using it more often, but I still don’t preregister as much as I “should”. Ever since I was asked to write this post I’ve been wondering where […]
It is too early to know whether the recent period of methodological introspection in psychological science, and the sciences in general, will lead to positive changes in practices. As with any revolution, there is the potential of moving backward. One problem that has been consistently acknowledged is the problem of incentives: what gets one attention […]
Preregistration has many advantages, which have been pointed out in Steve Lindsay’s post yesterday and many other places. The most important advantage is probably that it demonstrates without doubt that the hypotheses and data-analysis path chosen for a study were not chosen in response to the data with an eye towards obtaining the desired results. […]
Richard Shiffrin’s 2018 PNAS article on the nature of scientific progress is beautifully written, erudite, and insightful. I learned a lot from it and agree with most of his arguments. But with the greatest respect I would like to counter Shiffrin’s expressions of doubt regarding the value of preregistering research plans. Those doubts were lightly […]
We all want to be free. Freedom of choice is perhaps one of the most foundational principles of western societies, and it can empower lives. As academics, we cherish academic freedom, and academic freedom has been called essential to democracy for very good reasons. However, no matter how enthusiastically we may endorse academic freedom, we […]
When we talk about statistical modeling, we often encounter the concept of “degrees of freedom.” Remember? It’s the n-1 in t[n-1] or the [1] in χ2[1]. In our off-the-shelf statistical procedures, the degrees of freedom refers to the information content of some statistical construct. It can loosely be thought of as the number of independent […]
Reproducibility is the hallmark of science. It has been argued that a finding needs to be repeatable to count as a scientific discovery and that replicability is a line of demarcation that separates science from pseudoscience. The fact that a recent large replication effort of 100 studies found that fewer than half of cognitive and social psychology […]
Automatic detection of automatic response generators: How to improve data quality in online research
In recent years, researchers have started using Amazon Mechanical Turk and similar services to collect data from online participants. Two big benefits are the speed and ease of data collection. A study that might take a year to run using a participant pool at a small university could now be completed in a day, and […]
The International Meeting of the Psychonomic Society in Amsterdam wrapped up on Saturday (12 May). The meeting was attended by around 700 delegates and featured keynote addresses by John Wixted and Dedre Gentner. Some photos of the meeting are available on the Society’s website. The meeting also featured 7 symposia: Tackling the Confidence Crisis with […]