Eye tracking with Native SDK

USING THE EYE TRACKING DATA

You can acccess the following eye tracking data through the Varjo API:

  • leftEye [varjo_ray]
  • rightEye [varjo_ray]
  • gaze [varjo_ray]
  • focusDistance
  • stability
  • captureTime
  • leftStatus
  • rightStatus
  • status
  • frameNumber
  • leftPupilSize
  • rightPupilSize

varjo_ray

varjo_ray returns data about eye position coordinates [origin (x, y, z)] and vector direction [forward (x, y, z)]. Data is recorded for the left eye (leftEye), right eye (rightEye), and their combined value (gaze). Gaze data is given in the left-hand coordinate system (X points right, Y points top, Z points forward) and relative to head pose. Please note that not just forward, but also origin component of returned varjo_ray must be accounted when computing gaze ray projection to 3D scene.

focusDistance returns the distance between eye and focus point. It has a value between 0.0 and 2.0 meters.

stability returns the stability of the user’s focus. It has a value between 0.0 and 1.0, where 0.0 indicates least stable focus and 1.0 most stable.

captureTime captureTime returns a timestamp, in nanoseconds, of when the eyes video frame was recorded.

leftStatus and rightStatus return a value for each eye as follows:

  • 0 – eye is not tracked and not visible (e.g., the eye is shut)
  • 1 – eye is visible but not reliably tracked (e.g., during a saccade or blink)
  • 2 – eye is tracked but quality is compromised (e.g., the headset has moved after calibration)
  • 3 – eye is tracked

status returns a value for the status of eye tracking in the Varjo headset as follows:

  • 0 – data unavailable; user is not wearing the device or eyes cannot be found
  • 1 – user is wearing the device, but gaze tracking is being calibrated
  • 2 – data is valid

frameNumber returns a unique identifier of the frame when the data was recorded.

leftPupilSize and rightPupilSize return values for the size of the pupil. Pupil size is measured in pixels from eye tracking camera and then normalized to 0.0…1.0 range via division to the maximum tracked pupil size constant.

Data Stream Options

When using the Varjo gaze tracking system via public API, you can choose between smoothed (default) and raw gaze tracking output and between 100Hz (default) and 200Hz tracking data rates.

For the default data stream settings, call the varjo_GazeInit function with no arguments.

For custom stream settings, use the varjo_GazeInitWithParameters function with the varjo_GazeParameters argument filled according to your desired data stream mode:

  • Set OutputFilterType to Standard for smoothed output and None for raw output.
  • Set OutputFrequency to OutputFrequency100Hz, OutputFrequency200Hz, or OutputFrequencyMaximumSupported (for the maximum data rate supported by connected headset).

It is possible to call varjo_GazeInit and varjo_GazeInitWithParameters both before and after headset is connected. Please note that OutputFrequency200Hz is not supported on older devices (VR-1, VR-2, VR-2 Pro, XR-1). Initializing gaze with unsupported data stream frequency can lead to error varjo_Error_GazeUnsupportedParameter, if headset is connected at the time of initialization. In the case when headset is not yet connected, data stream frequency support is checked again when headset is connected, which might result in varjo_Error_GazeUnsupportedParameter error on next call to varjo_GetGaze or varjo_GetGazeArray.

Calibration Options

There are three types of calibration available in the Varjo gaze tracking system: quick one-dot Foveation calibration, Fast and Legacy.

One-dot quick Foveation calibration is launched automatically every time the user puts on VR-3, XR-3 or Aero headset. After successful calibration, eye tracking data is immediately available via the API functions varjo_GetGaze and varjo_GetGazeArray.

The other two advanced gaze calibration algorithms are available via the varjo_RequestGazeCalibrationWithParameters function. Choose between Fast (default) and Legacy calibration by filling the GazeCalibrationType parameter appropriately. Calling varjo_RequestGazeCalibration will result in Fast calibration requested.

Fast calibration is a quick five-dot calibration that uses observations from the current user as well as statistical priors generated from diverse, previously collected and processed observations from other people.

Legacy calibration takes longer and only uses observations from the current user to train the eye tracking algorithm. Legacy calibration can be appropriate, for example, in medical research use cases where the use of statistical priors may not be advisable.

Note, requesting gaze calibration when headset is not connected will result in error varjo_Error_GazeNotConnected.

Privacy Settings

Varjo Base user interface has privacy setting toggle “Allow eye tracking“, which impacts behaviour of Varjo eye tracking native API.

In case “Allow eye tracking” is off, calling varjo_RequestGazeCalibration, varjo_RequestGazeCalibrationWithParameters, varjo_GetGaze or varjo_GetGazeArray will result in error varjo_Error_GazeNotAllowed.

It is possible to query state of this Base setting via varjo_IsGazeAllowed function.

Please note that this privacy setting does not prevent Varjo system to launch quick one-dot Foveation calibration and use eye tracking for foveated rendering.

Example

Please check our ”GazeTrackingExample” application demonstrating usage of Varjo native API for eye tracking.