I have written about a number of issues concerning the practice of science out of concern that the present narrative is unbalanced: I believe that science is doing very well, even in our fields, despite the problems many have identified. One essay, albeit aimed at all of science, is found in the recent PNAS article […]
Statistics and Methodology
“(…) a substantial proportion of research effort in experimental psychology isn’t expended directly in the explanation business; it is expended in the business of discovering and confirming effects” —Cummins (2000). I am contributing to this digital event from a theoretician’s perspective. I thought I’d be upfront about this to set the right expectations. Theoretical perspectives […]
I have a confession to make. I like the idea behind preregistration a lot, and over the last few years I’ve been making a concerted effort to start using it more often, but I still don’t preregister as much as I “should”. Ever since I was asked to write this post I’ve been wondering where […]
It is too early to know whether the recent period of methodological introspection in psychological science, and the sciences in general, will lead to positive changes in practices. As with any revolution, there is the potential of moving backward. One problem that has been consistently acknowledged is the problem of incentives: what gets one attention […]
Preregistration has many advantages, which have been pointed out in Steve Lindsay’s post yesterday and many other places. The most important advantage is probably that it demonstrates without doubt that the hypotheses and data-analysis path chosen for a study were not chosen in response to the data with an eye towards obtaining the desired results. […]
Richard Shiffrin’s 2018 PNAS article on the nature of scientific progress is beautifully written, erudite, and insightful. I learned a lot from it and agree with most of his arguments. But with the greatest respect I would like to counter Shiffrin’s expressions of doubt regarding the value of preregistering research plans. Those doubts were lightly […]
We all want to be free. Freedom of choice is perhaps one of the most foundational principles of western societies, and it can empower lives. As academics, we cherish academic freedom, and academic freedom has been called essential to democracy for very good reasons. However, no matter how enthusiastically we may endorse academic freedom, we […]
When we talk about statistical modeling, we often encounter the concept of “degrees of freedom.” Remember? It’s the n-1 in t[n-1] or the [1] in χ2[1]. In our off-the-shelf statistical procedures, the degrees of freedom refers to the information content of some statistical construct. It can loosely be thought of as the number of independent […]
Reproducibility is the hallmark of science. It has been argued that a finding needs to be repeatable to count as a scientific discovery and that replicability is a line of demarcation that separates science from pseudoscience. The fact that a recent large replication effort of 100 studies found that fewer than half of cognitive and social psychology […]
Automatic detection of automatic response generators: How to improve data quality in online research
In recent years, researchers have started using Amazon Mechanical Turk and similar services to collect data from online participants. Two big benefits are the speed and ease of data collection. A study that might take a year to run using a participant pool at a small university could now be completed in a day, and […]