Third-party tracking

Developing a tracking plugin for Varjo
Tracking Plugin API
     Tracking Plugin API for user configuration
System definitions
     Coordinate system
     HMD tracking point
Important information
     Critical HMD components
Using a Tracking Plugin
     Tracking plugin discovery
     Tracking plugin installation
     Tracking plugin registration
     Tracking plugin selection UI
Enabling Varjo Controllers

Developing a Tracking Plugin for Varjo

To use a third-party tracking system, you will need to use the Varjo Tracking Plugin SDK available on the Downloads page. For installation instructions, see Tracking plugin installation.

Tracking Plugin API

The following scheme describes the flow of the Varjo Plugin API (the Plugin API is located below Varjo SW).

Before you begin implementing support for tracking plugins, please read through this page for a description of all the processes and functions that can be accessed through the Varjo API.

Initialize your tracking plugin with varjo_PluginInit.

Use varjo_TrackerStart and varjo_TrackerStop to start and end tracker processing.

Get the current tracker pose for HMD with varjo_GetHMDPose.

To end the session, de-initialize your tracking plugin with varjo_PluginShutdown.

You can find functions that affect your tracking plugin behavior with Varjo API in varjo_PluginUtilityAPI.

The following structure describes tracked device pose:

    typedef struct varjo_DevicePose { 
       varjo_Nanoseconds timeStamp;     //!< Pose timestamp (ns) 
       int64_t frameNo;                 //!< Frame number, increases monotonically. 
       varjo_Vector3D position;         //!< Device position (m) 
       varjo_Quaternion rotation;       //!< Device rotation 
       varjo_Vector3D velocity;         //!< Device linear velocity (m/s) 
       varjo_Vector3D angularVelocity;  //!< Device angular velocity (radians/s) 
       varjo_Vector3D acceleration;     //!< Device acceleration (m/s^2) 
       varjo_PoseFlags poseFlags;       //!< Bit field value describing pose 
       double confidence;               //!< Tracker confidence 
    } varjo_DevicePose;

The following definitions describe the types of data stored in the pose:

    // Tracked device position
    // Distance unit is meter.
    struct varjo_Vector3D { 
        double x;  //!< X coordinate. 
        double y;  //!< Y coordinate. 
        double z;  //!< Z coordinate. 
    }; 

    // Double precision quaternion for tracked device orientation
    struct varjo_Quaternion { 
        double w; 
        double x; 
        double y; 
        double z; 
    }; 

    // Tracker pose timestamp
    // Time unit is nanoseconds. Absolute times are relative to an epoch which is
    // constant during execution of the program.
    typedef int64_t varjo_Nanoseconds;

Note: To change pose values, you will need to set poseFlags.

    pose->poseFlags = varjo_PoseFlags_Ok | varjo_PoseFlags_HasPosition;

All possible pose flags are documented in the API docs.

Tracking Plugin API for user configuration

You can specify persistent configuration options that the user can edit in Varjo Base. Add this feature to your plugin as follows:

  1. Modify your plugin.manifest and add configurationOptions to specify what the user can edit. For example:
    "configurationOptions": [ 
        { 
            "id": "serverAddress",
            "label": "Server IP address",
            "type": "text",
            "defaultValue": "192.168.0.1"
        }
    ]

  1. Use PluginUtilityAPI’s GetConfigurationString, SetConfigurationString and SyncConfiguration C++ API to access the configuration. See the Varjo Plugin SDK for more detailed information.

  2. After you install the plugin and restart Varjo Base, you should see the configuration options in the System tab in Varjo Base. You can then select your plugin, change the configuration, and select Submit to submit the values to your plugin.

  3. You can use PluginUtilityAPI’s SetPluginSystemState to communicate the plugin state to Varjo Base. The plugin state is displayed above the configuration options.

System definitions

Coordinate system

The coordinate system for the HMD is right-handed: positive X goes right, positive Y goes up, and negative Z goes forward from the HMD user’s perspective.

HMD tracking point

The tracking point of the HMD is located between the lenses as shown below.

Important information

Critical HMD components

For tracking systems that require marker objects to be placed on the HMD, it is important to minimize interference with critical HMD components (cameras, sensors, ventilation, etc.) as shown below.

Using a Tracking Plugin

Tracking plugin discovery

On startup, the VarjoTracking process will enumerate all manifest files in the subdirectories of the %ProgramData%\Varjo\VarjoTracking\Plugins folder. The manifests are validated, and plugins that pass validation can communicate with the Varjo software. Varjo Base will display all available tracking systems in the UI, where the user can choose which tracking system to use.

Tracking plugin installation

Install the plugin in the location specified by the tracking system vendor.

Tracking plugin registration

After the plugin is installed, it will need to be made available to the Varjo software stack. You can do this by writing a JSON file known as the plugin manifest file, which contains a description of the plugin. 

Place the file in the following location:

%ProgramData%\Varjo\VarjoTracking\Plugins\<vendor_name>\<tracking_product_name_plugin>.manifest

Notes:

  • The name of the manifest file can be arbitrary
  • The file must have a .manifest file name extension

The plugin manifest file is a JSON file containing the following information: 

    {
        "manifestFileVersion": "1.0.0.0",
        "ID": "<GUID>",
        "vendorName": "<Tracker system vendor name>",
        "productName": "<Tracker product name>",
        "path": "<absolute path to the tracker plugin DLL>",
        "configurationOptions": []
    }

For example:

    {
        "manifestFileVersion": "1.0.0.0",
        "ID": "{E6F31B4E-015F-4719-BDDB-8758E400A6BC}",
        "vendorName": "Vendor",
        "productName": "Advanced Tracker",
        "path": "C:\\Program Files\\Vendor\\Product\\Plugins\\Varjo\\Plugin.dll",
        "configurationOptions": [
            {
                "id": "serverAddress",
                "label": "Server IP address",
                "type": "text",
                "defaultValue": "192.168.0.1"
            }
        ]
    }

Notes: 

  • All fields should be UTF-8 encoded
  • The productName field can contain an empty string
  • The ID field is an RFC-4122 UUID in Microsoft GUID format, such as {123e4567-e89b-12d3-a456-426655440000}
  • Configuration options listed in the configurationOptions array are displayed in the Varjo Base UI

Tracking plugin selection UI

Enabling Varjo Controllers

Varjo controllers are not enabled with third-party tracking by default. You can enable them with the following configuration option.

    "configurationOptions": [
        {
            "id": "useVarjoControllers",
            "label": "Use Varjo Controllers",
            "type": "text",
            "defaultValue": "false"
        }
    ]

This adds a new configuration option to Varjo Base System tab when the tracking plugin is selected.

To use the Varjo controllers with your plugin:
1) Set Use Varjo Controllers to true
2) Press Submit
3) Restart Varjo system from Support tab
Varjo Base restart is required every time this value is changed.

When Varjo controllers are enabled, the tracking cameras on the Varjo XR-4 are turned on. Do not cover the tracking cameras with your custom tracking system hardware.

Note: Varjo Home will not display Varjo controllers when third party plugin is in use and Varjo Base will not have pairing button visible. Please pair the controllers using inside-out tracking option and use your own or third party applications for testing their functions.