Eye Tracking

Analytics

A new OpenXR eye tracking analytics extension XR_VARJO_eye_tracking_analytics is available for preview in Varjo Base 4.10 OpenXR runtime. Note that the API might still be changed in an incompatible way in future releases and enabling the extension requires extra steps detailed below.

Enabling the extension

To enable the preview XR_VARJO_eye_tracking_analytics extension in the runtime, create a DWORD registry key HKEY_LOCAL_MACHINE\SOFTWARE\Varjo\OpenXR\EnableEyeTrackingAnalyticsPreview and set its value to 1. This can be achieved by starting PowerShell with administrator privileges and running the following commands:

If (-NOT (Test-Path "HKLM:\SOFTWARE\Varjo\OpenXR")) { New-Item -Path "HKLM:\SOFTWARE\Varjo\OpenXR" -Force }
New-ItemProperty -Path "HKLM:\SOFTWARE\Varjo\OpenXR" -Name "EnableEyeTrackingAnalyticsPreview" -Value "1" -PropertyType DWORD -Force

Obtaining the header

The preview extension is not yet available in the official openxr.h header. To be able to use the extension, download the extension header varjo_eye_tracking_analytics.h, place it in your project and include it after including openxr.h.

Sample application

The HelloXR sample application that is included in the Varjo OpenXR SDK contains example usage of the XR_VARJO_eye_tracking_analytics extension. Steps for testing the sample:

  • Download and extract the Varjo OpenXR SDK archive.

  • Obtain the extension header varjo_eye_tracking_analytics.h as described in Obtaining the header. Place it in subfolder varjo-openxr\helloxr of the extracted SDK archive.

  • Edit the file varjo-openxr\helloxr\main.cpp, adding the line #define USE_EYE_TRACKING_ANALYTICS_API_PREVIEW in the beginning of the file.

  • Build the SDK project defined by the CMake configuration file varjo-openxr\CMakeLists.txt. For example, this can be done by opening the CMakeLists.txt file using Visual Studio CMake integration and building the project, which will generate the HelloXR executable in varjo-openxr\out\build\x64-Debug\HelloXR.exe. Alternatively, you can generate the project files using cmake or cmake-gui.

  • Connect the Varjo headset, ensure that Varjo Base 4.10 is running, access to eye tracking data is allowed through the “Allow eye tracking” toggle in the Privacy section of the Support tab in Varjo Base and the preview extension is enabled using a registry key as described in Enabling the extension.

  • Launch the HelloXR executable. The headset should enter mixed reality view. Whenever you press the E key in the console, the eye analytics data (gaze directions and eye measurements) will be dumped to the console and the headset view should start showing your tracked gaze rays at the time of the key press as lines of cubes superimposed on the mixed reality view.

Extension API

To use the extension, you should enable the extension XR_VARJO_eye_tracking_analytics when creating the OpenXR instance using xrCreateInstance.

You can query the capabilities of the extension by adding a XrSystemEyeTrackingAnalyticsPropertiesVARJO structure with type set to XR_TYPE_SYSTEM_EYE_TRACKING_ANALYTICS_PROPERTIES_VARJO to the next chain of the XrSystemProperties structure when calling xrGetSystemProperties. This will initialize the following fields in the structure:

  • XrBool32 supportsEyeTrackingAnalytics: XR_TRUE if eye tracking analytics is supported by the runtime.
  • uint32_t eyeTrackerSampleFrequency: The eye tracking sample rate in Hz. For all current supported Varjo headsets, this is 200 Hz.

After creating an OpenXR session, you should create an eye tracker handle using the xrCreateEyeTrackerVARJO function, with the following arguments:

  • XrSession session: The OpenXR session to create the eye tracker for.
  • const XrEyeTrackerCreateInfoVARJO* createInfo: Pointer to a structure with type XR_TYPE_EYE_TRACKER_CREATE_INFO_VARJO. Currently the structure does not contain any data fields.
  • XrEyeTrackerVARJO* eyeTracker: Pointer to the output handle that will be written if the eye tracker is created successfully.

Note that eye tracker creation may fail with error XR_ERROR_PERMISSION_DENIED_VARJO if access to eye tracking data is currently not allowed in Varjo Base.

The eye tracker handle can be destroyed by passing it to xrDestroyEyeTrackerVARJO.

You can then query the eye tracking data samples using either xrGetEyeTrackingAnalyticsDataVARJO (for getting the latest sample) or xrGetEyeTrackingAnalyticsDataArrayVARJO (for reading the full stream of samples at most 0.5 seconds old). These functions take the following arguments:

  • XrEyeTrackerVARJO eyeTracker: The eye tracker handle.
  • const XrEyeTrackerDataInfoVARJO* info: Pointer to a structure with type XR_TYPE_EYE_TRACKER_DATA_INFO_VARJO. It contains a field XrSpace baseSpace that specifies which OpenXR space the gaze poses should be relative to.
  • XrEyeTrackingAnalyticsDataVARJO* data:
    • For xrGetEyeTrackingAnalyticsDataVARJO, a pointer to a single output structure with type XR_TYPE_EYE_TRACKING_ANALYTICS_DATA_VARJO that will be filled with the latest eye analytics data sample if the call is successful.
    • For xrGetEyeTrackingAnalyticsDataArrayVARJO, a pointer to a buffer of output structures with type XR_TYPE_EYE_TRACKING_ANALYTICS_DATA_VARJO. This buffer will be filled with the available analytics data samples that have not yet been read before. The buffer capacity and output size are communicated using the additional parameters:
      • uint32_t dataArrayCapacity: The number of output structures available to be filled in data.
      • uint32_t* dataArraySize: Pointer to the output number of data samples actually filled in data, written if the call is successful.

Note that these functions may also fail with error XR_ERROR_PERMISSION_DENIED_VARJO if access to eye tracking data was turned off in Varjo Base after the eye tracker handle was created.

The XrEyeTrackingAnalyticsDataVARJO structure contains the following fields:

  • XrTime captureTime: Timestamp from when the sample was captured, or 0 when not available.
  • uint64_t frameNumber: Unique frame identifier number from when the sample was captured, or 0 when not available.
  • XrEyeGazeDataVARJO eyeGaze[XR_EYE_GAZE_COUNT_VARJO]: Gaze pose information for combined eyes in array index XR_EYE_GAZE_COMBINED_VARJO and individual eyes in array indices XR_EYE_GAZE_LEFT_VARJO and XR_EYE_GAZE_RIGHT_VARJO.
  • XrEyeMeasurementsDataVARJO eyeMeasurements[XR_EYE_MEASUREMENT_COUNT_VARJO]: Eye measurements for each individual eye in array indices XR_EYE_MEASUREMENT_LEFT_VARJO and XR_EYE_MEASUREMENT_RIGHT_VARJO.
  • XrEyeCombinedDataVARJO combinedMeasurements: Eye tracking measurements that concern both eyes.

The structure types XrEyeGazeDataVARJO, XrEyeMeasurementsDataVARJO and XrEyeCombinedDataVARJO contain eye tracking data fields, each of which has a corresponding status flags field with type XrEyeTrackedStateFlagsVARJO. This status field contains bit XR_EYE_TRACKED_STATE_VALID_BIT_VARJO if the corresponding data field contains a valid value. The available data fields are as follows:

  • XrEyeGazeDataVARJO:
    • XrPosef gaze: The gaze direction pose.
  • XrEyeMeasurementsDataVARJO:
    • float pupilIrisDiameterRatio: The ratio between pupil and iris diameters.
    • float pupilDiameterMM: The pupil diameter in millimeters.
    • float irisDiameterMM: The iris diameter in millimeters.
    • float eyeOpenness: Relative eye openness. Value 1 corresponds to a fully open eye, 0 to a fully closed eye.
  • XrEyeCombinedDataVARJO:
    • float interPupillaryDistanceMM: Interpupillary distance of the user in millimeters.
    • float focusDistance: Distance between the user’s eyes and the focus point in meters. The value is between 0 and 2 meters.