Statistics and Methodology

Automatic detection of automatic response generators: How to improve data quality in online research

In recent years, researchers have started using Amazon Mechanical Turk and similar services to collect data from online participants. Two big benefits are the speed and ease of data collection. A study that might take a year to run using a participant pool at a small university could now be completed in a day, and […]

Continue Reading

#psAmsterdam18: A retrospective on the meeting and expert opinions

The International Meeting of the Psychonomic Society in Amsterdam wrapped up on Saturday (12 May). The meeting was attended by around 700 delegates and featured keynote addresses by John Wixted and Dedre Gentner. Some photos of the meeting are available on the Society’s website. The meeting also featured 7 symposia: Tackling the Confidence Crisis with […]

Continue Reading

The ABC in #BayesInPsych: Approximating likelihoods in simulation models

(This post was co-authored with Brandon Turner). Sharon Bertsch McGrayne’s 2012 book, The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy, traces the difficulties that statisticians and empirical researchers alike have had in embracing Bayesian methods. Despite the obvious […]

Continue Reading

#BayesInPsych: Spiking a slab with sleepless pillow talk and prior inequalities

I recently finished reading Suzanne Buffam’s, A Pillow Book. This is a book of non-fiction poetry about thoughts and musings that may enter the mind as one drifts off to sleep, ranging from the historical consideration of pillows to comprehensive lists of sleeping aids. I’ve spent more than a few nights drifting off to sleep considering […]

Continue Reading

We often know more than we think: Using prior knowledge to avoid prior problems #BayesInPsych

One of the unique features of Bayesian statistical and computational modelling is the prior distribution. A prior distribution is both conceptually and formally necessary to do any sort of Bayesian modelling. If we are estimating the values of model parameters (e.g., regression coefficients), we do this by updating our prior beliefs about the parameter values […]

Continue Reading

The four horsemen of #BayesInPsych

I see four benefits to the use of Bayesian inference:  Inclusion of prior information.  Regularization.  Handling models with many parameters or latent variables.  Propagation of uncertainty. Another selling point is a purported logical coherence – but I don’t really buy that argument so I’ll forget that, just as I’ll also set aside philosophical objections against […]

Continue Reading

From classical to new to real: A brief history of #BayesInPsych

The #BayesInPsych Digital Event kicked off yesterday and as the leading Guest Editor of the special issue of Psychonomic Bulletin & Review, I take this opportunity to provide more context for this week’s posts. The simple act of deciding which among competing theories is most likely—or which is most supported by the data—is the most […]

Continue Reading

#BayesInPsych: Preventing miscarriages of justice and statistical inference

Your brilliant PhD student ran an experiment last week that investigated whether chanting the words “unicorns, Brexit, fairies” repeatedly every morning before dawn raises people’s estimates of the likelihood that they will win the next lottery in comparison to a control group that instead chants “reality, reality, reality”. The manipulation seems to have worked, as […]

Continue Reading

Learning to classify better than a Student’s t-test: The joys of SVM

Is a picture necessarily worth a thousand words? Do bilinguals always find some grammatical features in their second language to be more difficult than native speakers of that language? Is the Stroop effect necessarily larger when the task is to name the color ink of a color word than when the task is to read […]

Continue Reading

A new look at old data: Results may look better but different

Replication and reanalysis of old data is critical to doing good science. We have discussed at various points how to increase the replicability of studies (e.g. here, here, here, and here), and have covered a few meta-analyses (here, here). Maybe it is because technology is constantly changing, and because we forget where we leave files […]

Continue Reading