Once upon a time, Kepler and Descartes proposed that vergence is the critical absolute distance cue. And that theory lived happily ever after. Until Paul Linton‘s paper. In this podcast, I interview Paul (pictured below) about his work published in the Psychonomic Society journal Attention, Perception & Psychophysics. 

Learn about longstanding theories and Linton’s research to better understand our visual perceptions by listening to the interview and reading the transcripts below.

Transcription

Kosovicheva: You’re listening to All Things Cognition, a Psychonomic Society podcast. Now, here is your host, Laura Mickes.

Mickes: I’m talking with Paul Linton about his paper published in Attention, Perception & Psychophysics.

Hi Paul.

Linton: Hey, how’s it going?

Mickes: Good. How are you doing?

Linton: Yeah. Good, good, very good.

Mickes: Your paper is called “Does vision extract absolute distance from vergence?” And it’s not often that I come across papers that cite work from the 17th century, but yours does. My students sometimes think that 2010 papers are ancient.

You cited Kepler and Descartes, which leads me to my first question. And that is, will you give us some background about your research?

Linton: Sure. Great. The essential question is how do we know the size and distance of objects in the world? Really fundamental question of visual perception.

And sometimes when you look at the textbooks, there’s a sort of whole bunch of different lists. So they might say, well, you see a familiar object, so you know its size and you infer its distance from that. Or you see an object on the ground and, you know your relationship to the ground plane and you sort of infer its distance from that.

But what Kepler and Descartes were looking for was like a general principle or general mechanism that applied to pretty much every instance. So you didn’t have to know what the object was. It didn’t have to be in any specific context, like the ground plane. And so what they thought was, was the most plausible suggestion was the following idea, which is known as vergence.

That’s the idea that the closer an object is, the more the two eyes have to rotate in order to fixate on it. So if I’m looking at a far object, like the moon, my two eyes are going to be parallel, right? They’re going to be looking directly at it. But if I take a pencil and move that as close as say 30 centimeters, 20 centimeters, then my right eye is going to have to rotate leftwards to fixate on it, and my left eye is going to have to rotate rightwards. And it’s that, that rotation of the eyes that Kepler and Descartes thought was especially important for our distance and size perception, especially at near distances. And so that’s, that’s generally been the account for the last 400 years.

Mickes: So everyone just went with it. They didn’t question it much or they, or was there a lot questioning?

Linton: It was difficult to test. So for about a hundred years, they sort of went with the theory. And then in, in the 1700s, what, what you had was little suggestions that that was the case. So there used to be something known as the wallpaper illusion. So if you look at a constant pattern on a wall and then you cross your eyes artificially, so that your right eye is sort of looking left with your left eye is looking rightwards, the wallpaper pattern seems to, as it were, jump forwards toward you. And so when people experienced that, they were like, okay, oh, that’s the Kepler and Descartes principle that we know and love.

But it was only really in the 1830s, So another, a hundred years after that, when Charles Wheatstone invented the stereoscope (pictured below). He was presenting one image to the right eye, one image to the left eye.

And that enabled him to manipulate the angular rotation of the eyes, independent of the image that was going to each eye. And so once he started manipulating it, then he himself sort of reported. And it was all in terms of sort of introspection, right? It wasn’t sort of psychophysics or experimentation, It was introspected that as you manipulated vergence, the distance decreased.

And then since Wheatsone, especially since over the last hundred years, there have been numerous attempts to quantify this effect. And especially the last 20 years, there’s been some very, very strong evidence. So basically what happens is participants will be presented with a point of light in a dark box. And then they have to point with a hidden hand as the distance, as a point of light. And so the evidence that comes out of these experiments, so it’s a really strong relationship between the change in vergence, so the change in the rotation of the eyes, and the distance people are able to point to.

Mickes: So that’s support for Keppler’s and Descartes’s idea of vergence.

Linton: Yeah.

Mickes: Okay. But when you came around and said, now this can’t be. There are too many issues that you didn’t control for. Is that right?

Linton: What slightly worried me about the way that vergence as a distance cue has been tested over the last a hundred years and in, in these papers is it’s all about sudden eye movements. So you’re putting people in darkness and then you’re suddenly presenting like a dot of light as close as 20 centimeters. And they’re having to suddenly rotate their eyes and then point to its distance. And when you think about it, that’s very different from our everyday perception of the world, right?

So in that experimental paradigm, you’re clearly going to be aware of your own eye movements. If your eyes are responding, that’s, that’s what you’re fixating on. In the real world, we’re virtually never aware of our own eye movements.

And there’s some other sort of technical concerns. So in the first case, if you suddenly present a dot, there’s going to be all sorts of changes in your retinal image. So it’s going to appear as double. And then when you converge, it’s going to move on the retina, all these kinds of things. So that’s another thing you want to control for, but the sort of fundamental thought was if vergence is only ever effective, when people are consciously aware of their own eye movements in these very extreme, sudden eye movement cases, but that’s never the case in sort of real-world viewings, then, then we’ve got a real concern.

So the challenge was to come up with a paradigm, which basically manipulated vergence, but without informing the participants. So they didn’t consciously know that what was going on in the experiment was that their eye rotation was changing.

Mickes: How did you do it?

Linton: Basically, what was happening in their previous experiments was, as I said, you’re in sudden darkness, and then a light’s suddenly presented. And so what I did instead was have participants point to the distance of a point of light and then between trials have them just fixate on a target naturally. You know, don’t do anything special, just relax, just look at the target. And then what, what was happening between trials is, is the, the vergence distance of that target was changing. And it was changing gradually, but not terribly gradually. And so it was varying from between 46 centimeters and 23 centimeters. And then on each trial, after the vergence had gradually varied, that target would turn into a dot and participants would then have to point to its distance. [See experimental setup below.]

Mickes: What were their instructions?

Linton: I kept them completely in the dark, both literally and metaphorically. They were told virtually nothing they were bought in. They weren’t allowed to see the apparatus. So I didn’t want them to know anything about the distances involved. So they had to put a blindfold on, they had to sort of be within the sort of dark apparatus before they could take the blindfold off. And then all I said was just watch this target, and then when it turns into a dot point to its distance, and then it’ll turn back into a target and we went through that.

Mickes: That’s it.

Linton: Yeah.

Mickes: Now these were friends you have on Facebook, is that right?

Linton: The initial experiment was, yeah, it was, it was an advertisement I put on Facebook

Mickes: And your friends came and you blindfolded them and put them in your lab [laughs].

Linton: It was quite an ordeal. For the second experiment, then tested a whole bunch of new participants. And interestingly enough, there’s the second part, it was beneficial that the second load of participants were, were slightly younger than the first set of participants, because what that also brings into account as what’s known as accommodation, which is the focal distance of the eye.

And so in the, in the second experiment, I also controlled for that using a whole bunch of different lenses. Although the title of the paper is, “does vision extract absolute distance from vergence?” It could equally be a “does vergence extract absolute distance from vergence and accommodation.”

Mickes: Oh.

Linton: Even with that accommodation cue added in there was still virtually no effect. Participants were just sort of guessing at the distance,

Mickes: In both of these experiments, they were just at chance?

Linton: There’s a very small slant. So again, I think it’s 0.12 to 0.16, a very small effect. And when I interviewed participants afterward and said, well, how did you find it? They said, well, you know, sometimes I could feel a bit of eyestrain.

Mickes: Oh.

Linton: Once you get that, and then you sort of look at who said that compared to who was having the sort of largest effect of vergence, you realize that that very negligible effect was primarily just an effect of essentially eyestrain. People just felt that as they were fixating close at 20 centimeters or 22 centimeters, this was going to be a close trial. But the real key is when you look at the raw data,

Mickes: Yea.

Linton: because it’s just a mess.

Mickes: [laughs]

Linton: … people, people are essentially guessing; they have got no clue. And the vast majority of participants when you interviewed them afterward were just like, I was just guessing. For a couple of participants, it was challenging to get them to point in the first instance, because they were kind of like, well, where do I point? And you say, well, just give it your best shot.

Mickes: yea.

Linton: And this is meant to be for a cue, that’s meant to be essential for reaching and grasping. And so they were just absolutely guessing in this context,

Mickes: This is amazing. Were you surprised by the findings when you looked at the data or you probably put yourself through the experiment to test it?

Linton: I read about the vergence claim and the reason I sort of tested it was like I did mock something up for myself in darkness with sort of bars moving. And I just didn’t experience the effect myself. But you always, you always wonder, you always think, well, maybe I’m just, you know, an odd one out. Yeah. It was good to finally run participants through it and see a consistent effect or lack of effect, which is the main point. [See some of the results below.]

Mickes: Right. And that happened in Experiments 1 and 2. In both, participants were just guessing.

Linton: Yes.

Mickes: I often battle in the literature with people who are alive. I wonder what Kepler and Descartes would say to you these days.

Linton: I’d like to think they’d be intrigued. They did go back and reread them. And, uh, they really write like vision scientists. So Descartes is saying, look, you know, we’ve got this vergence cue, but also he tries to quantify over what distances it’s effective. And he’s also talking about accommodation, so the focus part of the lens, he’s also talking about motion parallax. He’s also talking about familiar size, but he’s saying, look, you know, don’t think that’s perceptual. That’s just as it were your knowledge, giving you a cognitive inference. I mean, it’s really rich discussions there. So, uh, yeah, I don’t want to say Descartes’ been neglected cause that’s certainly not the case, but, uh, I, yeah, it was an interesting read.

Mickes: This leaves it open. We don’t know. That’s where we stand now?

Linton: Yeah. So I got a quite mischievously titled pre-print called “Do we see scale?“, Which is sort of investigates this question because part of a broader project, which I was discussing in my books, I wrote a book, The Perception and Cognition of Visual Space. What I was trying to get at there was okay, you know, everyone has their own favorite, absolute distance cue. But when you look at them closely, none of them really seem to do the job. So if you want to get rid of a vergence, then you might look at accommodation, which was a focal power of the lens.

But again, the literature is the question that a lot as do my results in the paper. Other people talk about motion parallax, but that’s been extensively tested in virtual reality.

Mickes: I’m sorry, what is that?

Linton: Oh, sorry. Uh, so motion parallax is this idea that as you move through the world objects that are close to you are going to move more in your visual field and objects from the far distance.

So as you sway back and forth a mountain on the horizon, isn’t going to move much, but another person quite close to you will. But that’s one of those claims that superficially looks quite convincing, but our eyes aren’t fixed. So basically if I then fixate on someone who’s quite close to me, that person won’t actually move, but the, the mountain behind them will move as I sway back and forth.

Mickes: Right.

Linton: So for people who think that motion parallax is going to get you out of needing to know about eye movements, it’s a problem. Essentially what you’re doing is you’re relying on the very same rotation of the eyes that you’re looking at in vergence, but now you’re moving over seconds at a time. So if you’re not convinced by vergence, you’re not going to be convinced by motion parallax. And so, yeah, I don’t think there’s a general, one size fits all mechanism.

You can look at some more cognitive cues, like familiar size. Ground plane, as well as, has had a lot of discussions, but you’ve got to think,

Mickes: What is that?

Linton: So this is the idea that if there’s an object on the ground, because we’re so used to the angle between ourselves and objects on the ground in relation to us that if there is an object in the ground and you think, oh, okay, well, that’s that angle downwards. Then we have some sort of idea of estimating its distance. Which is great, and you know, and there’s evidence for it in the literature, but we very rarely encounter, uh, objects in the ground as our sort of our primary concern. So often I think it may have a role to play, but in the literature itself it’s sometimes overestimated, I feel.

Mickes: Do you think then a lot of these findings are suspect or, or the interpretations of the data…

Linton: I think there’s a slight conflict between the textbook approaches and the papers themselves. So the textbook will very convincingly and very confidently assert five, six, seven absolute distance cues, and then say your visual system works it up. But then as you go to each and every one of these individual distance cues and you dig about in the literature, it seems less and less convincing – apart from vergence. That was the reason why you really wanted to put emphasis on vergence, because that was the one experimental cue where the evidence was sort of consistent or seem to be consistent.

Mickes: That’s interesting.

Linton: What I’m working on at the moment is to try and say, well, what could be happening instead is that the, the visual system, at least when we’re talking about distance estimation with two eyes, the visual system is relying on distortions of shape with distance, which is sort of a counter-intuitive way of thinking about it.

But some of your listeners may know that one of the ways the visual system extracts 3d shape is through binocular disparity. So that’s the difference in the two perspectives of the two eyes. But what’s interesting about binocular disparity is, is that that falls off dramatically with distance. So it falls off with the distance squared. As far as binocular disparity is concerned, the, the shape of objects essentially gets flattered with distance,

Mickes: Right.

Linton: We don’t really notice that that much in, in our everyday lives, but it may be something that sort of subconsciously the visual system or whatever’s processing visual scale is picking up on. So if you have some sort of notion of the shape of this thing looking at, maybe from other pictorial cues, like perspective or shading, things like that, or even familiar knowledge, and yet you’re encountering different degrees of flatness or accentuated depth, depending on what distance you’re encountering these objects, maybe those are the kinds of computations the visual system is relying on. So,

Mickes: So that’s what you’ve done next, in this pre-print?

Linton: Yeah, that’s what I’m looking at because what, what traditionally is said in the literature that we need to know distance in order to effectively extract 3d shape.

Mickes: And you say no.

Linton: Yeah, I try and flip it the other way round. I try and say, well, you know, we’re not particularly good at these sort of 3d shape constancies, as they’re known. When you present people with binocular disparity, things do seem to get flatter with distance when they’re when they’re presented in the laboratory. So maybe that’s what the visual system is doing and relying on to extract distance.

Mickes: Right, but you’ve written it.

Linton: Preprint’s out. But I do want to revise that I do want to sort of alter it a bit more. There’s a number of different things that need to be explored because if you don’t know vergence, if you don’t have this distance, then trying to make sense of binocular disparity, uh, becomes so that the difference in the perspectives of the two eyes, becomes very, very difficult. It’s very unclear how you’re even going to go about extracting 3d shape from a stimulus that, that presupposes that, you know, the vergence distance.

That’s the next project is trying to explain, well, okay, you’ve got two retinal images. You don’t know the vergence, but the literature tells you, you need to know the vergence to get 3d shape from it. So how are you going to do it? Right. So that’s a really difficult computational question.

Mickes: Wow, you’re asking big questions.

Did you have a hard time getting past the reviewers or they think, okay, this should be out there. This is a convincing set of experiments.

Linton: Yeah, I think that’s a very fair question. One of the jobs of reviewers is to say, well, you know, is there an alternative explanation for what’s going on? Right? Cause you’ve got plenty of papers saying there is a strong effect. And then you’ve got Paul coming along with his one paper saying there isn’t.

I should add though, that I have subsequently done a second paper, which is a pre-print, which looks at the manipulating vergence with size. There’s a suggestion, known as vergence micropsia that if you change vergence, if you bring something closer, you see it as shrinking. If the retinal image is fixed. If you look at an afterimage in darkness and you, and you move the after image closer, it’s meant to shrink according to this account. Anyways, I run an experiment and built as sort of quantification mechanism into the experiment to test if this really was the case and I found no effect.

So yeah.

Mickes: again? [laughs]

Linton: Yeah. So that’s, that’s two papers on the, on the anti-vergence distance size.

One of the key concerns of the reviewers was to say, well, you’re graduating, manipulating vergence between trials and much as I was criticizing the literature for not being sort of naturalistic and reflecting our everyday perception, they were saying, well, you know, maybe vergence isn’t effective when we make gradual eye movements, but it is effective when we make sudden eye movements. So I think there’s two answers to this. Uh, the first thing and quick answer is in the second paper, the speed of the eye movements is equivalent to the speed of the eye movements in previous papers.

Mickes: Oh.

Linton: So that’s sorted. But the second answer more conceptual is I think is also convincing. Which is this, the following three things, can’t be true: if I gradually changed your viewing distance from far to near or from near to far in the real world, and then asked you to, to pick something up, I think you’d be pretty effective. You wouldn’t all of a sudden lose your ability to just pick a cup up or something like that. Right? There’s no problem there.

Mickes: Right.

Linton: The second claim is that vergence is very effective. One of our most important cues for picking up nearby objects like cups. And then the third claim is well Paul’s manipulation of vergence is too gradual for vergence to be effective. So those three things can’t all be true, right? One of those two things have to,

Mickes: Right.

Linton: So my worry is, although reviewers were forcibly pushing this point, I was trying to push back and say, well, if you take that point, then you’re implicitly admitting that in real-world viewing conditions. Vergence just isn’t an important distance cue. And if you want to admit that from the outset, I’m happy. That’s fine. But that was kind of the point of the paper.

Mickes: When we had email exchanges about your paper earlier, I asked you for some real-world links and one, you mentioned that I was hoping you had talked more about, was about virtual reality and how this work might link to augmented reality, virtual reality. Can you talk a little bit about how it’s related or how it might be an issue?

Linton: Sure. So even now with the leading virtual and augmented reality headsets, people are still experiencing spatial distortions, right? So they might underestimate or overestimate distances. And so one of the concerns is, well, that might have to do with what’s known as vergence-accommodation conflict. So what that is is that you can imagine in virtual reality, you’re, you’re positioning objects in a whole bunch of different vergence distances, but your eyes are looking far and then they’re looking near, then they’re looking far, etc. At the same time, the actual objects in virtual reality are only ever presented on a screen that’s at one focus, distance, one optical distance, which is where your accommodation, your eyes are focusing. So if you think of the eye as just a single eye, it’s always focused at the same distance.

Mickes: yeah.

Linton: But if you think of the eyes as two eyes working together, they’re always rotating, inwards and outwards, depending on what the distance is.

And so that conflict between the, as it were, the one-eye perspective and the two-eye perspective is a really key concern in virtual and augmented reality. So there are two things. So one is that kind of conflict leads to problems with comfort. And that’s still very much the case. I’m afraid my paper can touch that. But in terms of explaining why you might get spacial distortions in virtual and augmented reality, it really does question that idea that it has to do profoundly with something to do with the vergence-accommodation conflict, because the paper test vergence and it tests accommodation, and it finds virtually no effect of them on distance.

Mickes: So in creating programs for virtual and augmented reality, something else needs to be taken into account?

Linton: Yeah. So that’s the challenge. It’s one of those really interesting things going back to the textbook, right? You’ve got that list of five, eight, whatever, absolute distance cues. And you think, well, you know, virtual granted reality pretty much tick those off, right? You can get accurate motion parallax. You can get all these, these different cues, but yet you still have these distortions. So it’s a really interesting question, you know, what, what are we missing, right? What additional element do we need to really bring into to those kinds of headsets to get them over the line?

Mickes: I think what you’ve done is really neat and I hope to see the paper cited a lot and people really pick up on it. Is there anything else you want to add?

Linton: Ah, no. No, thanks so much for having me it’s been great.

Mickes: Oh, it’s so cool. Thanks for teaching me some of the basics.

All right. Thanks, Paul!

Linton: Thanks so much.

Concluding statement

Kosovicheva: Thank you for listening to All Things Cognition, a Psychonomic Society podcast.

If you liked this episode, please consider subscribing to the podcast and leaving us a review. Reviews help us grow our audience and reach more people with the latest scientific research.

See you next time on All Things Cognition.

Featured Psychonomic Society article

Linton, P. (2020). Does vision extract absolute distance from vergence? Attention, Perception & Psychophysics, 82, 3176–3195. https://doi.org/10.3758/s13414-020-02006-1