Does anyone remember how to write a check these days? I imagine some people reading this post might be scratching their heads and wondering what a check is. In today’s world, where paper and hard currency are quickly being replaced with credit cards, and checks are mostly obsolete due to Venmo, Zelle, or PayPal, one may wonder about my opening question.

Checks, like addresses on an envelope, have standardized features that must be reported and followed to avoid returned checks or mail. A check must have the Payee labeled properly, the amount to be received in the correct box and spelled out properly on the “to-be-paid” line, and finally, the signature line requires a signature – in cursive! Side note: Perhaps the removal of cursive from school curricula was the underlying root of the demise of checks(?).

Regardless, standardization of practices in all facets of life is needed to ensure consistency in applying a procedure or practice. Science is no different. Standardizing a procedure ensures others can replicate the study with a different sample. Standardizing reporting metrics provides the most information by which other researchers can evaluate the results and conclusions. Throughout the years, researchers have provided recommendations on various subjects to help organize or standardize a process of some kind. For example, in 2008, a group of researchers published statistical guidelines emphasizing the importance of reporting effect sizes for all statistical analyses within a paper. This recommendation was promoted by many journal editors, resulting in enhanced robustness of research findings.

This post focuses on a paper in which a large consortium of researchers from around the world spent three years gathering guidelines on eye-tracking research in an attempt to standardize the reporting process in published research. The paper follows up and formalizes a foundational article published in Behavior Research Methods earlier this year. The featured paper, aimed at those using eye-trackers in their research, offers publishing guidelines and outlines essential reporting criteria for ensuring reproducibility.

To me, two of the most critical components of this paper were (1) the collaborative spirit of the community to create this set of guidelines and (2) the persistence to keep refining the guidelines until all had agreed upon 12 core requirements and some ancillary components that were considered helpful, but not necessary. The table below summarizes the guidelines established by this consortium.

Summary of established reporting guidelines for studies using eye-tracking technology within an easy-to-use checklist.

As described in the paper, most of these guidelines were typically reported but not always inclusive. The majority of the terms are self-explanatory. “A” items (in the table above) are expected to be reported in all studies and include topics that typically come with the manual. Basic information includes:

1. the manufacturer and model of the eye tracker device
2. the software and firmware used to run the eye tracker as well as to calibrate, record, and process data
3. the actual eye tracking technology, which includes a wide range of options from electrooculography to dual Purkinje imaging and retinal-image-based tracking, among others
4. the frequency with which the eyes are scanned and recorded per second (Hertz)
5. the manner in which the head is held still such as on a chin rest or brace
6. whether one or both eyes are recorded
7. the parameters recorded, including units (e.g., degrees, pixels, etc.), frame of reference origin (zero), and the directions (e.g., up/down) represented by positive and negative values
8. the type of lighting within the environment (e.g., bright natural light or fluorescents)
9. whether calibration occurred using the software from the eye tracker or a unique program developed external to the eye tracker system
10. the amount of error that could be present in the system
11. the steps used to clean, organize, and process the data,
12. the degree to which data are lost in the system due to participant actions or other organization processes.

The consortium of eye-tracker researchers also suggested that the latencies between signals and the distance between the eye-tracker and the participant be reported as additional information.

Dunn, the featured paper’s first author, stated:

“This guideline was a community effort over a period of three years and we hope it will lead to improvements in the quality of research outputs across the eye tracking disciplines. It should be a useful reference when preparing research manuscripts, both for seasoned eye tracking experts and newcomers to the field. It will be updated at regular intervals and as new technologies emerge, ensuring it remains relevant into the future.”

I think this approach is a wonderful demonstration of the collaborations scientists can share. Ideally, more scientific disciplines will be inspired to share research objectives rather than continuing in the competitive spirit that seems to follow scientists within any discipline. Unfortunately, the time it takes science to become more collaborative within and across disciplines will likely make our need for standardized banking or mailing procedures obsolete. Hopefully, the consortium will update the current eye-tracking guidelines more timely.

Featured Psychonomic Society article

Dunn, M. J., Alexander, R. G., Amiebenomo, O. M., Arblaster, G., Atan, D., Erichsen, J. T., … & Sprenger, A. (2023). Minimal reporting guideline for research involving eye tracking (2023 edition). Behavior Research Methods, 1-7. https://doi.org/10.3758/s13428-023-02187-1