Hand tracking with Varjo headset

Varjo’s VR-2 Pro is features built-in hand tracking technology from Ultraleap. It allows you to use your hands without any physical controller, and to reach out with your hands into VR world. Motions like pinching, grabbing, and interaction with objects accommodates a new level of immersion for your applications.

Setup

In order to enable the hand tracking module, download Ultraleap SDK for Varjo from our Downloads page. It is important that you download UltraleapSDK from Varjo website as we provide a software version that is specifically tailored for VR-2 Pro and stripped down of all unnecessary components.

When both the Varjo and Ultraleap SDK are installed, and the VR-2 Pro is correctly connected, you should see the Ultraleap icon turning green in Windows dashboard:

Developing with hand tracking

You can develop hand tracking support for Unreal engine, Unity, or natively. Refer to the Ultraleap documentation to follow up-to-date guides on integration and best practices of using hand tracking in VR.

When you start developing hand tracking, make sure to set offset for the hand position first. Since the head tracking point for the headset and hand tracking point for Ultraleap are not the same, the offset needs to be defined. The offset is the following (note: depending on your engine of choice, you may have a different scale):

Y: -0.025734 m
Z: 0.068423 m
X tilt: 5.0°
OR
Y: -25.734 mm
Z: 68.423 mm
X tilt: 5.0°

UNITY ULTRALEAP PLUGIN INSTALLATION

If you want to add hand tracking to your Unity project follow these steps:

  • Close your Unity project
  • Download Varjo VR-2 Pro Ultraleap SDK for Unity from Github
  • Copy the Assets folder from Ultraleap SDK for Unity to your project’s folder
  • Start your Unity project
  • Select VarjoCamera and Add a new component to it called “Leap XR Service Provider”

  • Set the Device Offset Mode to Transform and drag the HandTrackingOffset game object to Device Origin (HandTrackingOffset can be found under VarjoCamera)

  • (Optional) Enabling “Update Hand In Precull” may improve latency in some cases

For adding 3d models of the hands refer to the Ultraleap documentation.

UNREAL ULTRALEAP PLUGIN INSTALLATION

To add hand tracking to your Unreal project follow the instructions on the Varjo VR-2 Pro Leap Motion SDK for Unreal Engine Github page.