Since Herb Clark published his famous “Language as a fixed effect fallacy” in 1973, there has been a slow realization that standard techniques, such as ANOVA, are the wrong tools for the jobs that most psychologists tackle. The basic problem is that most psychological questions involve generalization beyond a sample of people and beyond a […]
Statistics and Methodology
This is the first in a series of posts on online data-collection. The popularity of collecting behavioral data online continues to rise. The reasons are many: ease of getting large numbers of participants, relatively low cost, and access to a more diverse population. Early concerns that online data collection is inherently unreliable are gradually evaporating, […]
Yogi Berra once famously said that “You can observe a lot by watching”. Yogi Berra observed and said a lot of things, but this line has a lot going for it. The idea that information can be gathered by “just looking” entered statistics many decades ago. For example, John Tukey, one of the 20th century’s […]
One of the essential goals of psychology is generalization: describing ways in which people are similar. Of course, human behaviour varies across situations, times, and individuals, and hence often defies generalization. Ignoring this variability and assuming that people are the same can lead to improper generalizations about human behaviour. In a new paper in the Psychonomic Bulletin […]
I sometimes use MTurk for running experiments. As an old-timer, there is something of a magical quality to MTurk. You post your study and only a few hours later you receive data from perhaps one thousand or so people. I can remember when we used to meet the people participating in our studies face-to-face, with […]
There are no statistics that inflame the passions of statisticians and scientists as does the p value. The p value is, informally, a statistic used for assessing whether a “null hypothesis” (e.g., that the difference in performance between two conditions is 0) should be taken seriously. It is simultaneously the most used and most hated statistic in all of […]
Since the enlightenment, openness has been a core part of the ethos of science. Scientific openness takes many forms: from its inception the Royal Society, for instance, published reports from all over the world, not just Great Britain. Science is politically open, a collective search for truth and human well-being ideally not concerned with national […]
It is Christmas time and everything these days seems to be covered with singing Santas and stuffed stockings, shining brightly in red, white and green. Now imagine that sneaky red and white striped Waldo is hiding among the Christmas decorations. Telling him apart from the rest will be tedious. Needless to say, the similarity between […]
In his influential book, Understanding the New Statistics, Geoff Cumming makes the case that psychologists should change the way they report their statistics. Psychologists, he argues, would be far better off if they stopped reporting p-values and started reporting confidence intervals. When I read his book I was struck by the information presented about p-value misconceptions based on a survey in […]
In yesterday’s post, I showed that conventional frequentist confidence intervals are far from straightforward and often do not permit the inference one wants to make. Typically, we would like to use confidence intervals to infer something about a population parameter: if we have a 95% confidence interval, we would like to conclude that there is a […]