What percentage of Americans are Muslim? What percentage of Americans are LGBT individuals? What percentage of Americans are Christian? Think about it and take real guesses. If you are like the average person, your guesses were likely overestimates of the real numbers for the smaller percentages (for Muslims; for LGBT), and underestimates for the larger percentages (for Christians). Clearly, you are uninformed, confused, and possibly racist. Your fear of Muslims and gays intrudes into your estimates, making you think they are everywhere!

Let’s not jump to conclusions.

Perhaps that’s too harsh. Perhaps you are just sensitive to the saturation media coverage about Muslims, immigration, and gay rights. That coverage makes you over-represent the number of Muslims and gays in your life (even if you aren’t afraid of those groups), further biasing your estimate upwards.

Both of these versions are nice stories, and ones that the media has told at length about an (at worst) racist and (at best) misinformed public.

The problem with these stories is that these exact error patterns occur for far more mundane tasks, like reading bar charts, or estimating the proportion of letters in a string that are “a,” as compared to “c.” (For example, in a string like: “acccccaccaccaaccccccc”). I’ll spare you the diatribe ending with me calling you “vowel-ist.” Perhaps, people systematically overestimate the proportion of small values, from Muslims to vowels, and systematically underestimate the proportion of large values, from Christians to consonants.

So, what might explain the bias in estimates of demographics? A group of researchers led by David Landy argue in a recent Psychonomic Bulletin & Review article that it may be a result of domain-general psychophysical processes. Psychophysics is the scientific study of perceptual judgments. Landy and colleagues suggest that the systematic overestimation of small values and underestimation of large values is better explained using two psychophysical concepts, which have been demonstrated in magnitude estimation tasks across a large set of studies. Only after these general biases are accounted for, Landy and colleagues argue, should specific topic-based biases—such as homophobia—be analyzed.

The first psychophysical principle is uncertainty-based rescaling. The basic idea (similar to regression toward the mean) is that when you encounter a small sample of extreme values, you tend to weight that extreme value back toward the average. For example, say the first three basketball players you see are over 7 feet tall. If you were asked to estimate the average basketball-player height, you could reason that basketball players tend to be over 7-feet. What uncertainty-based rescaling says you are likely to do, is combine that estimate (over 7-feet) with your average for male height (around 5’9″ tall), and guess something like 6’10” tall. You’d be wrong – overestimating the height of the average basketball player by 3 inches – but you’d be closer to the real average of basketball players than if you’d guessed over 7 feet.

The second psychophysical principle is that people tend to report odds when you ask them for proportions. In other words, when estimating the number of Muslims out of 100 (e.g., 10), people tend to report the ratio of Muslims to non-Muslims (e.g., 10/90 = 11).

The key to Landy and colleagues’ argument is that the psychophysical account provides the most parsimonious account of the demographics bias effects. People exhibit these biases across domains of magnitude estimation – why should demographics be different? Once the general magnitude biases are accounted for, only then should we examine whether people show demographics biases beyond what is predicted by the psychophysical models.

To test their models, Landy and colleagues used publicly available data from the European Social Survey and an Ipsos Mori 2014 poll. These surveys asked respondents to indicate the proportion of various minorities, teen pregnancy, number of people voting, and a number of other estimates of demographics. For each such variable X, people were asked to respond with the number of people out of 100 who were X.

The data, and the models the researchers tested are in panels A and C in the figure below (the same data are also plotted in panels B and D, just adjusted for a log scale).

The data are very close to the model (see the grey line for the model’s prediction). The red lines indicate perfect performance. Smaller values (to the left of each graph) are overestimated (above the red line) whereas larger values (to the right of each graph) are underestimated (below the red line).

These fairly simple models explain the pattern in the data quite well. So, is there any evidence of a domain-specific demographics effect that might reveal prejudice? If that were the case, we would expect demographics to be even more overestimated compared to what a psychophysical model would predict. In other words – we know that small magnitudes are always overestimated, but does homophobia boost the estimate further? If there is something specifically related to demographics (media coverage, fear, etc.), causing an additional bias, this should be clear after the baseline magnitude biases are accounted for.

The next figure displays the errors of people’s estimates, sorted within each graph from highest proportions (voting) to lowest (teen pregnancy). In blue are the errors of the raw proportions. These blue bars show clear bias – higher proportions are too low, lower proportions are too high. But again, this bias is consistent with the psychophysical model. The red bars show the errors that remain after the psychophysical model has been fit.

If there was an additional effect, specific to demographics, we would expect all red bars to line up with all blue bars, which is not the case. That is, after controlling for the bias that is expected for a magnitude estimation task like this, if there were additional bias, the error in the red bars would not be random – it would be systematically in the direction of the blue bar. Instead, the red bars appear random, occurring equally likely on either side of the dotted line. In these data, there is no evidence of a domain-specific account beyond what is predicted by the psychophysical model.

The public may be misinformed on a number of things (just ask our editor). But, as Landy and colleagues point out, it can be dangerous to purport that people are misinformed, racist, or confused, when in fact they are exhibiting perfectly normal human biases. These biases can occur across contexts, regardless of domain, and are generally widespread. Attributing racial bias, fear of immigrants, or homophobia to public perceptions of demographics should require a stronger test than what can be explained by basic psychology.

Reference for the Psychonomics article discussed in this post:
Landy D., Guay, B., & Marghetis, T. (2017). Bias and ignorance in demographic perception. Psychonomic Bulletin & Review. DOI: 10.3758/s13423-017-1360-2.