Eye tracking

For general information about eye tracking, see Eye tracking.

Using eye tracking with Varjo XR plugin

Calibration

To reliably track user’s eye movements, eye tracking must be calibrated first. While Varjo headsets support automatic one-dot calibration, we recommended that you request a separate calibration if more accurate eye tracking data is required. You can request calibration by calling a method in the VarjoEyeTracking class.

// Requests gaze calibration: "Fast" 5 dot calibration is used by default.
// Returns true if calibration was successfully requested.
bool VarjoEyeTracking.RequestGazeCalibration();

// Requests gaze calibration with specific GazeCalibrationMode: allows to
// choose between "Fast" and "OneDot" calibration.
// Returns true if calibration was successfully requested.
bool VarjoEyeTracking.RequestGazeCalibration(gazeCalibrationMode);

It is possible to query quality assessment of current user gaze calibration by calling method GetGazeCalibrationQuality.

// Returns score assesment of currently used gaze calibration
GazeCalibrationQuality VarjoEyeTracking.GetGazeCalibrationQuality();

Returned struct GazeCalibrationQuality contains score GazeEyeCalibrationQuality for left and right eye calibration. These values indicate how well each of the eyes passed through the calibration procedure and how accurate returned gaze data could be for the eye. Possible returned scores are Invalid (score not available), Low, Medium and High.

Accessing eye tracking data

If eye tracking is calibrated, you can access eye tracking data by using either the XR Input subsystem or the methods provided in the VarjoEyeTracking class.

With the Input subsystem, the eye tracking data can be retrieved using the Eyes interface.

The methods in VarjoEyeTracking allow you to poll the eye tracking data at the full frequency supported by the headset, which is especially important for research scenarios. These methods provide eye tracking data frames as GazeData structs and estimates for user’s eye properties as EyeMeasurements:

GazeData

  • long frameNumber - A unique identifier of the frame at the time when the data was recorded.
  • long captureTime - A timestamp, in nanoseconds, of when the video frame was recorded by the eye tracking cameras. It is relative to the Varjo system time epoch.
  • GazeStatus status - A status for eye tracking.
  • GazeRay gaze - Gaze ray combined from both eyes.
  • float focusDistance - The distance between eye and focus point in meters. Values are between 0 and 2 meters.
  • float focusStability - Stability of the user’s focus. Value are between 0.0 and 1.0, where 0.0 indicates least stable focus and 1.0 most stable.
  • GazeEyeStatus leftStatus - A status for the left eye.
  • GazeRay left - Gaze ray for the left eye.
  • [deprecated] float leftPupilSize - Pupil size for the left eye, calculated according to the pupil size range detected by the headset. Values are between 0 and 1.
  • GazeEyeStatus rightStatus - A status for the right eye.
  • GazeRay right - Gaze ray for the right eye.
  • [deprecated] float rightPupilSize - Pupil size for the right eye, calculated according to the pupil size range detected by the headset. Values are between 0 and 1.

GazeStatus is a value for the eye tracking status of the headset as follows:

  • 0 – Data unavailable: User is not wearing the headset or eyes cannot be located
  • 1 – User is wearing the headset, but gaze tracking is being calibrated
  • 2 – Data is valid

GazeRay struct contains data about eye position coordinates in meters [origin (x, y, z)] and a normalized direction vector [forward (x, y, z)]. Gaze data is given in the left-hand coordinate system and is relative to head pose.

GazeEyeStatus is a value for each eye as follows:

  • 0 – Eye is not tracked and not visible (e.g., the eye is shut)
  • 1 – Eye is visible but not reliably tracked (e.g., during a saccade or blink)
  • 2 – Eye is tracked but quality is compromised (e.g., the headset has moved after calibration)
  • 3 – Eye is tracked

Gaze data should be polled frequently as all frames older than 500ms are discarded. Gaze data is polled using the following methods:

// Returns the latest gaze data frame. The result may change even within a single Unity engine frame.
GazeData VarjoEyeTracking.GetGaze();

// Writes all gaze data frames since the last call to this method in GazeData list.
// Returns the number of retrieved gaze data frames.
int VarjoEyeTracking.GetGazeList(out gazeDataList);

// Pulls most recent gaze data and eye measurements from the queue.
// Returns the number of retrieved gaze data frames.
int VarjoEyeTracking.GetGazeList(out List<GazeData> gazeData, out List<EyeMeasurements> eyeMeasurements)

EyeMeasurements

  • long frameNumber - A unique identifier of the frame at the time when the data was recorded.
  • long captureTime - A timestamp, in nanoseconds, of when the video frame was recorded by the eye tracking cameras. It is relative to the Varjo system time epoch.
  • float interPupillaryDistanceInMM - Estimate of user’s inter-pupillary distance measured in millimeters. 0.0 if estimate is not available.
  • float leftPupilIrisDiameterRatio - Ratio of user’s left pupil diameter estimate to estimated iris diameter. 0.0 if either estimate is not available and ratio cannot be calculated
  • float rightPupilIrisDiameterRatio - Ratio of user’s right pupil diameter estimate to estimated iris diameter. 0.0 if either estimate is not available and ratio cannot be calculated
  • float leftPupilDiameterInMM - Estimate of pupil diameter for the left eye in millimeters. If estimate is not available e.g. when pupil is not tracked, value is set to 0.0.
  • float rightPupilDiameterInMM - Estimate of pupil diameter for the right eye in millimeters. If estimate is not available e.g. when pupil is not tracked, value is set to 0.0.
  • float leftIrisDiameterInMM - Estimated diameter of left iris in millimeters. Estimation is updated when user puts the headset on his head. Estimate values are valid until user removes headset from his head. 0.0 is returned when estimate is not available.
  • float rightIrisDiameterInMM - Estimated diameter of right iris in millimeters. Estimation is updated when user puts the headset on his head. Estimate values are valid until user removes headset from his head. 0.0 is returned when estimate is not available.
  • float leftEyeOpenness - Estimated openness ratio of the left eye. Value 1.0 corresponds to a fully open eye, 0.0 to a fully closed eye.
  • float rightEyeOpenness - Estimated openness ratio of the right eye. Value 1.0 corresponds to a fully open eye, 0.0 to a fully closed eye.

Eye measurements data can be polled using the following methods:

// Returns the latest eye measurements. The result may change even within a single Unity engine frame.
EyeMeasurements VarjoEyeTracking.GetEyeMeasurements();

// Pulls most recent gaze data and eye measurements from the queue.
int VarjoEyeTracking.GetGazeList(out List<GazeData> gazeData, out List<EyeMeasurements> eyeMeasurements)

Please note that leftPupilIrisDiameterRatio, rightPupilIrisDiameterRatio, leftPupilDiameterInMM, rightPupilDiameterInMM, leftIrisDiameterInMM, rightIrisDiameterInMM, leftEyeOpenness, rightEyeOpenness are experimental features and provided in this release without any accuracy guarantees.

Data stream options

The Varjo XR plugin provides several options for the eye tracking data stream. You can choose between smoothed (default, filter type Standard) and unfiltered gaze data (filter type None), and select a gaze output frequency from 100Hz, 200Hz and the maximum supported frequency by connected headset (default). See the general Eye tracking documentation for frequencies supported by different headsets.

You can set the options using the following methods:

// Sets GazeOutputFilterType: None / Standard
// Returns true if output filter type was successfully set.
bool VarjoEyeTracking.SetGazeOutputFilterType(outputFilterType);

// Sets GazeOutputFrequency: MaximumSupported / Frequency100Hz / Frequency200Hz
// Returns true if output frequency was successfully set.
bool VarjoEyeTracking.SetGazeOutputFrequency(outputFrequency);

You can also query currently used options using the following methods:

// Returns currently set GazeOutputFilterType
GazeOutputFilterType VarjoEyeTracking.GetGazeOutputFilterType();

// Returns currently set GazeOutputFrequency
GazeOutputFrequency VarjoEyeTracking.GetGazeOutputFrequency();

Eye tracking examples

Examples of using eye tracking can be found in the Eye tracking example of the Varjo Unity XR Plugin.