You are in the cognitive laboratory and you focus on the screen in front of you. A few color patches are flashed for 1/10^th of a second, and 2 seconds later another array of patches appears that stays on the screen until you respond. Your task is to decide whether any one of the patches of color in the second array differs from the colors briefly flashed earlier. Tasks such as this are used to explore visual short-term memory (VSTM); that is, our ability to encode and remember purely visual stimuli that often escape verbalization.

One key aspect of understanding VSTM involves the processes that govern encoding into VSTM and access to its contents. Concerning encoding, we rely on external visual attention. Several metaphors for visual attention have been proposed, for example the notion that it serves as a spotlight that can be pointed at regions of potential interest, or the notion that attention can highlight certain features over others. Common to all metaphors is the idea that attention can be selectively cued, and that subsequent processing of the cued information is expedited compared to performance on non-cued information.

Another manifestation of attention is known as internal attention and it allows us to focus on features that have already been encoded in our memory, without the original visual stimulus being present. The canonical demonstration of how internal attention can guide memorial performance is the “iconic memory” paradigm developed by George Sperling in 1960: when people are presented with a large array of letters and they are subsequently cued (e.g., by a tone) to report only letters from one row of the display (e.g., a high tone for the top row, a low tone for the bottom, and so on), recall performance is superior compared to a condition in which no cue is provided.

How do those two different manifestations of attention interact? What is the relationship between internal (memorial) and external (visual) attention?

A recent article in the Psychonomic Bulletin & Review addressed these questions. Researchers Kalogeropoulou, Jagadeesh, Ohl, and Rolfs designed a visual memory study that combined pre-cuing of visual attention with post-cuing of memory contents. The basic idea of their study was quite simple: people were briefly shown an array of two sets of differently oriented gratings (one black, one white). Participants had to memorize the orientations and report the orientation of one set of gratings at the end of each trial. The crucial manipulation was when and how the critical set of gratings was cued.

The figure below shows the sequence of events in the procedure used by Kalogeropoulou and colleagues:

Panel (a) shows the sequence of events, and panel (b) illustrates the probabilities with which the various cues were presented. To illustrate, on a valid pre-cue trial, people would see a black circle before the memory display, signaling that the black gratings were relevant and ought to be preferentially memorized. The brief memory display might then be followed by another valid “retro”-cue or a neutral cue that provided no further information. The retro-cue was followed by the retrieval phase, during which participants used a knob to adjust the orientation of the gratings in the probe display until it matched the memorized orientation of the relevant gratings. The gratings in the probe display were at different locations from the memory set and were presented at a random orientation. The events were similar for the other types of trials except that the pre-cue might be neutral or invalid.

Participants’ performance was assessed by various measures, including the angular error between the recalled orientation of the gratings and their actual orientation in the memory display. The figure below shows the main results.

The figure shows the angular error for the conjunction of all 6 cueing conditions. Unsurprisingly, performance is best when the pre-cue is valid (least error; left-most points), and it is worst when that pre-cue is invalid (right-most points). In addition, the retro-cue further reduces error when it is valid, irrespective of the status of the pre-cue.

This overall pattern was replicated across a variety of other measures (e.g., the proportion of responses that were identified as random).

These results are quite informative and perhaps also somewhat counter-intuitive: Suppose that the pre-cue selected information for encoding such that people only encoded gratings of the cued color. In that case, on invalid trials, the wrong gratings would be encoded whereas the correct stimulus set would be ignored. If that were the case, the retro-cue would not be expected to have a fortifying effect on the memory representations—after all, cueing for information that was never encoded is unlikely to be terribly successful.

The data reject this simple model: the retro-cue was beneficial whenever it was valid, even when the pre-cue had been invalid. This result suggests that while external (visual) attention selects features for encoding, the internal (memorial) attention can provide an added boost to memory irrespective of the quality of initial encoding.

Kalogeropoulou and colleagues conclude that visual attention during “the initial encoding of a stimulus appears to set an upper limit for the precision of a memory representation”, whereas the deployment of internal attention during retention further “prioritizes visual information during the maintenance of memoranda independently of the priorities the observer imposed during sensory encoding: reinforcing these initial priorities as well as changing priorities to a new feature in VSTM benefitted observers’ performance.”

These results are particularly remarkable because the deployment of attention could not have been based on spatial characteristics: the gratings in the probe display that observers adjusted to match the memorized orientations were in different locations from those in the memory display. Moreover, all cues were based on the color of the stimuli which was always independent of location. It follows that observers were able to deploy attention globally, across the entire visual space, based on specific targeted features.

Our internal attention, therefore, is apparently able to fortify encoded representations on the basis of relatively high-level features, such as color, even when the crucial attribute of the information—namely its orientation—is entirely spatial in nature.

Article focused on in this post:

Kalogeropoulou, Z., Jagadeesh, A. V., Ohl, S., & Rolfs, M. (2016). Setting and changing feature priorities in visual short-term memory. Psychonomic Bulletin & Review. DOI: 10.3758/s13423-016-1094-6.