03-30-2022, 04:47 AM
What is Eye Tracking Data Quality?
Understanding how to maximize, measure, and report eye tracking data quality is a critical skill for eye tracking researchers. This post provides a brief overview of some of the key concepts...
For a more detailed discussion please see our Eye Tracking Data Quality webinar.
When researchers talk about eye-tracking data quality, they are typically referring to a number of features of the sample level data. The most critical, and widely reported measures of eye tracking data quality are accuracy, precision, and data loss. Other features of sample level data, such as temporal precision and delays, may also be important - particularly for gaze-contingent research.
What factors impact on data quality?
Eye-tracking data quality is the result of several factors, including:
To check that your equipment has been set up optimally, please see our Setup and Usage Video Tutorials:
EyeLink 1000 Plus Setup and Usage Training Videos
EyeLink Portable Duo Setup and Usage Training Videos
The following resources provide information about the calibration process:
Why is calibration so important?
Which calibration model should I choose?
Data quality measures:
The following sections provide some brief definitions of some critical terms and concepts. For more detailed coverage of how to calculate metrics such as accuracy and precision, please see the webinar linked above.
Accuracy and Precision:
The concepts of accuracy and precision are critical to understanding eye tracking data quality, and are illustrated below. The center of the target represents the location the participant is instructed to look at, and the blue crosses represent the eye-tracker's gaze data (samples).
Data loss (or track loss) refers to periods where gaze data is missing. It's typically measured as the percentage of total samples for which no gaze data is available. While some data loss is unavoidable (e.g., during blinks), it can also be caused by preventable issues like excessive head movement or poor camera setup / poorly adjusted thresholds. You can significantly reduce preventable data loss by ensuring an optimal physical setup. A common cause of data loss is forcing the participant to rotate their eyes beyond the system's trackable range. To avoid this, make sure the monitor is positioned at the correct distance and height, as detailed in (see the Setup and Usage Video Tutorials).
Temporal Resolution (Sampling Rate)
Temporal resolution, or sampling rate, is the number of times per second (Hz) that an eye tracker measures eye position. A higher sampling rate directly improves data quality in several key ways:
A related concept is temporal stability, which refers to the consistency of the time interval between samples. An eye tracker with high temporal stability samples at a perfectly regular rhythm (e.g., exactly every 1 ms for a 1000 Hz tracker) without missing or skipping any measurements.
End-to-End Delay / Latency
End-to-end delay, is the total time from the moment an eye movement occurs to the moment the updated gaze location is available to your stimulus computer. This delay includes the time required to sample the eye, compute its position, and transmit the data. This measure is directly related to the tracker's sampling rate—a higher sampling rate results in a lower delay. For example, EyeLink systems have an end-to-end delay of just 2 ms when sampling at 1000 Hz.
Understanding how to maximize, measure, and report eye tracking data quality is a critical skill for eye tracking researchers. This post provides a brief overview of some of the key concepts...
For a more detailed discussion please see our Eye Tracking Data Quality webinar.
When researchers talk about eye-tracking data quality, they are typically referring to a number of features of the sample level data. The most critical, and widely reported measures of eye tracking data quality are accuracy, precision, and data loss. Other features of sample level data, such as temporal precision and delays, may also be important - particularly for gaze-contingent research.
What factors impact on data quality?
Eye-tracking data quality is the result of several factors, including:
- Hardware and software properties (e.g., the tracker's sampling rate).
- Participant-specific factors (e.g., wearing glasses).
- Operator-specific factors (e.g., level of training or an un-optimized setup).
To check that your equipment has been set up optimally, please see our Setup and Usage Video Tutorials:
EyeLink 1000 Plus Setup and Usage Training Videos
EyeLink Portable Duo Setup and Usage Training Videos
The following resources provide information about the calibration process:
Why is calibration so important?
Which calibration model should I choose?
Data quality measures:
The following sections provide some brief definitions of some critical terms and concepts. For more detailed coverage of how to calculate metrics such as accuracy and precision, please see the webinar linked above.
Accuracy and Precision:
The concepts of accuracy and precision are critical to understanding eye tracking data quality, and are illustrated below. The center of the target represents the location the participant is instructed to look at, and the blue crosses represent the eye-tracker's gaze data (samples).
- Spatial Accuracy
A perfect definition of spatial accuracy would be "the difference between the actual location of gaze and the location reported by the eye tracker." However, we have no independent way to know the actual location of a person's gaze. Therefore, in practice, spatial accuracy is defined as "the difference between the location of a target (where the participant is assumed to be looking) and the location reported by the eye tracker." Accuracy is typically measured in degrees of visual angle. In the illustration above accuracy would be operationalized as the average distance of each of the blue crosses (eye tracker samples) from the center of the target (the location we hope the participant is looking at), in degrees of visual angle.
It is important to note that any spatial inaccuracy that is measured in this way may reflect some issue with the eye tracker itself, or the participant's relative inability to accurately fixate a target, or some combination of both.
Spatial accuracy may differ across screen locations so accuracy is typically measured at different screen locations, not just the center. The calibration / validation procedure gives feedback about spatial accuracy at each of the validation target locations. For high-quality data, we recommend aiming for:- An average error of 0.5 degrees or less.
- A maximum error of less than 1 degree at any single point.
A common cause of spatial inaccuracies in some screen locations but not others is a non-linear (asymmetrical) calibration model. Please see the following posts for further details:
Why is calibration so important?
Changes in accuracy that occur over time are sometimes referred to as "drift". Regular drift checks (typically between each trial) can be used to monitor and correct (via a recalibration) any spatial inaccuracy that may develop during the experiment, thus ensuring that data remains spatially accurate during the actual trial recordings.
- An average error of 0.5 degrees or less.
- Precision (noise)
Precision refers to the repeatability of a set of measurements, regardless of their accuracy. It is often called "noise". Precision is typically reported in two ways:- Standard Deviation (SD): The average distance of samples from their mean location.
- RMS-S2S (Root Mean Square - Sample-to-Sample): The average distance between consecutive samples.
Precision can be measured both in real participants and with artificial eyes, which (unlike human eyes) can be kept perfectly motionless, allowing the precision of the eye tracker itself to be isolated. These measures are also typically reported in degrees of visual angle. Details on how to compute precision metrics are provided in the Eye Tracking Data Quality Webinar.
- Standard Deviation (SD): The average distance of samples from their mean location.
Data loss (or track loss) refers to periods where gaze data is missing. It's typically measured as the percentage of total samples for which no gaze data is available. While some data loss is unavoidable (e.g., during blinks), it can also be caused by preventable issues like excessive head movement or poor camera setup / poorly adjusted thresholds. You can significantly reduce preventable data loss by ensuring an optimal physical setup. A common cause of data loss is forcing the participant to rotate their eyes beyond the system's trackable range. To avoid this, make sure the monitor is positioned at the correct distance and height, as detailed in (see the Setup and Usage Video Tutorials).
Temporal Resolution (Sampling Rate)
Temporal resolution, or sampling rate, is the number of times per second (Hz) that an eye tracker measures eye position. A higher sampling rate directly improves data quality in several key ways:
- More Accurate Event Detection: It allows for a more precise measurement of fixation and saccade durations.
- Faster Data Recovery: The system can recover more quickly from data loss caused by blinks.
- Reduced Noise: It can help reduce noise in velocity calculations.
- Enables Advanced Tasks: High sampling rates are critical for implementing time-sensitive, gaze-contingent paradigms.
A related concept is temporal stability, which refers to the consistency of the time interval between samples. An eye tracker with high temporal stability samples at a perfectly regular rhythm (e.g., exactly every 1 ms for a 1000 Hz tracker) without missing or skipping any measurements.
End-to-End Delay / Latency
End-to-end delay, is the total time from the moment an eye movement occurs to the moment the updated gaze location is available to your stimulus computer. This delay includes the time required to sample the eye, compute its position, and transmit the data. This measure is directly related to the tracker's sampling rate—a higher sampling rate results in a lower delay. For example, EyeLink systems have an end-to-end delay of just 2 ms when sampling at 1000 Hz.