Eye tracking

For general information about eye tracking, see Eye tracking.

Using eye tracking with Varjo XR plugin

Calibration

To reliably track a user’s eye movements, eye tracking must be calibrated first. While the Varjo XR-3 headset supports automatic one-dot calibration, we recommended that you request a separate calibration if more accurate eye tracking data is needed. You can request calibration by calling a method in the VarjoEyeTracking class.

// Requests gaze calibration using the default settings.
VarjoEyeTracking.RequestGazeCalibration();

// Requests gaze calibration with specific GazeCalibrationMode.
VarjoEyeTracking.RequestGazeCalibration(gazeCalibrationMode);

Accessing eye tracking data

If eye tracking is calibrated, you can access eye tracking data by using either the XR Input subsystem or the methods provided in the VarjoEyeTracking class.

With the Input subsystem, the eye tracking data can be retrieved using the Eyes interface.

The methods in VarjoEyeTracking allow you to poll the eye tracking data at the full frequency supported by the headset, which is especially important for research scenarios. These methods provide eye tracking data frames as GazeData structs, which include the following information:

• long frameNumber - A unique identifier of the frame at the time when the data was recorded.
• long captureTime - A timestamp, in nanoseconds, of when the video frame was recorded by the eye tracking cameras.
• GazeStatus status - A status for eye tracking.
• GazeRay gaze - Gaze ray combined from both eyes.
• float focusDistance - The distance between eye and focus point in meters. Values are between 0 and 2 meters.
• float focusStability - Stability of the user’s focus. Value are between 0.0 and 1.0, where 0.0 indicates least stable focus and 1.0 most stable.
• GazeEyeStatus leftStatus - A status for the left eye.
• GazeRay left - Gaze ray for the left eye.
• float leftPupilSize - Pupil size for the left eye, calculated according to the pupil size range detected by the headset. Values are between 0 and 1.
• GazeEyeStatus rightStatus - A status for the right eye.
• GazeRay right - Gaze ray for the right eye.
• float rightPupilSize - Pupil size for the right eye, calculated according to the pupil size range detected by the headset. Values are between 0 and 1.

GazeStatus is a value for the eye tracking status of the headset as follows:

• 0 – Data unavailable: User is not wearing the headset or eyes cannot be located
• 1 – User is wearing the headset, but gaze tracking is being calibrated
• 2 – Data is valid

GazeRay struct contains data about eye position coordinates in meters [origin (x, y, z)] and a normalized direction vector [forward (x, y, z)]. Gaze data is given in the left-hand coordinate system and is relative to head pose.

GazeEyeStatus is a value for each eye as follows:

• 0 – Eye is not tracked and not visible (e.g., the eye is shut)
• 1 – Eye is visible but not reliably tracked (e.g., during a saccade or blink)
• 2 – Eye is tracked but quality is compromised (e.g., the headset has moved after calibration)
• 3 – Eye is tracked

Gaze data should be polled frequently as all frames older than 500ms are discarded. Gaze data is polled using the following methods:

// Returns the latest gaze data frame.
VarjoEyeTracking.GetGaze();

// Writes all gaze data frames since the last call to this method in GazeData list.
// Returns the number of retrieved gaze data frames.
VarjoEyeTracking.GetGazeList(out gazeDataList);

Data stream options

The Varjo XR plugin provides several options for the eye tracking data stream. You can choose between smoothed (default) and unfiltered gaze data and select a gaze output frequency from 100Hz (default), 200Hz and the maximum supported frequency. See the general Eye tracking documentation for frequencies supported by different headsets.

You can set the options using the following methods:

// Sets GazeOutputFilterType
VarjoEyeTracking.SetGazeOutputFilterType(outputFilterType);

// Sets GazeOutputFrequency
VarjoEyeTracking.SetGazeOutputFrequency(outputFrequency);

Eye tracking examples

Examples of using eye tracking can be found in the Eye tracking sample of the Varjo Unity XR Plugin.