Eye tracking with Varjo headset

Note: you are currently viewing documentation for a beta or an older version of Varjo

Varjo headset comes with the 20/20 Eye Tracker, our integrated eye tracking functionality. You can use eye tracking in your application, and you can log the information about gaze for analytics. Eye tracking can also be used for interacting with content; you can use it for selecting objects or prompting additional information about a specific object by simply looking at it.

WHAT IS GAZE?

Gaze starts from the gaze origin (the eye), and follows the gaze vector (the direction of the gaze). Normalized gaze vector is calculated. It is important to understand this concept in order to process eye tracking data while developing for the Varjo headset.

EYE TRACKER SPECIFICATIONS

Varjo headsets contain two eye tracking cameras – one for each eye.

  • IPD range – 61–73 mm
  • Gaze camera resolution – 1280 x 800 px per camera.
  • Gaze tracking frequency – 100 Hz
  • Eye tracking accuracy – 1°
  • Eye tracking precision – 0.2°

USING THE EYE TRACKING DATA

You can get following eye tracking data from Varjo API:

  • _leftEye _[varjo_ray]
  • _rightEye _[varjo_ray]
  • _gaze _[varjo_ray]
  • focusDistance
  • stability
  • captureTime
  • leftStatus
  • rightStatus
  • status
  • frameNumber
  • leftPupilSize
  • rightPupilSize

varjo_ray

varjo_ray contains information about eye position coordinates origin (x, y, z) and the direction of vector forward (x, y, z). varjo_ray info is recorded for left eye (leftEye), right eye_(rightEye),_ and combined value (gaze). Information about gaze is presented in the left-hand coordinate system and is relative to the head pose.

focusDistance is the distance between the eye and the focus point. It is a value between 0 and 2 meters.

stability is a value between 0.0 and 1.0 specifying the stability of the user’s focus. 0.0 means not stable and 1.0 means stable.

captureTime is the timestamp of when the data was recorded in nanoseconds.

leftStatus and rightStatus is a value for each eye where:

  • 0 – eye is not tracked and not visible (e.g. eye is shut)
  • 1 – eye is visible but not reliably tracked (e.g. saccade or blink)
  • 2 – eye is tracked but quality compromised (e.g. headset has moved after calibration)
  • 3 – eye is tracked

status is a value representing the status of eye tracking in the Varjo headsets, where:

  • 0 – data is not available, user is not wearing the device or eyes cannot be found
  • 1 – user is wearing the device, but gaze tracking is being calibrated
  • 2 – data is valid

frameNumber is a unique identifier of the frame when the data was recorded.

leftPupilSize and rightPupilSize are the values for the size of the pupil between 0 and 1 calculated according to the pupil size range detected by the Varjo headset.

DEVELOPING WITH 20/20 EYE TRACKER

Before trying to launch a demo with eye tracking, make sure to enable Allow eye tracking in Varjo Base from the System tab.

While developing, keep in mind that every time the headset is taken off and put back on, it must be re-calibrated even if the same person is using it. This is needed because the position of the headset on the user’s face may not be the same each time the headset is worn. You can manage calibration directly from your application. For detailed instructions, refer to your engine’s documentation.

Fast calibration

Two parameters are passed for tracking. The first parameter, “GazeCalibrationType” can be either “Legacy” or “Fast”. Legacy is the calibration that is currently used by default and has a long calibration time. Fast calibration is a new 5-point calibration which requires much less user time to complete. However, it is currently in the Beta state and its validation is ongoing. Therefore proven results rely on Legacy calibration. The second parameter passed is “OutputFilterType”. It defines whether you want to receive smoothed tracking data which is easier to read, or raw data. For raw data, pass key “None”. For processed data, pass “Standard”.

In order to request specific data, call RequestGazeCalibrationWithParameters and pass GazeCalibrationParameters as a parameter.

parameters = new VarjoPlugin.GazeCalibrationParameters[2];
                parameters[0] = new VarjoPlugin.GazeCalibrationParameters();
                parameters[0].key = "GazeCalibrationType";
                parameters[0].value = calibrationType == CalibrationType.LEGACY ? "Legacy" : "Fast";
                parameters[1] = new VarjoPlugin.GazeCalibrationParameters();
                parameters[1].key = "OutputFilterType";
                parameters[1].value = outputFilterType == OutputFilterType.STANDARD ? "Standard" : "None";

VarjoPlugin.RequestGazeCalibrationWithParameters(parameters);