Mixed Reality with Varjo XR plugin

Developing mixed reality applications for the Varjo XR-3 headset is quick and easy with the Varjo XR plugin. Follow these instructions to get started.

First set up the Varjo XR plugin (see Getting Started with Varjo XR plugin for Unity for details).

Once set up, the Varjo XR plugin settings can be found in the Project Settings dialog. Under the XR Plug-in Management tab, select Varjo and disable the Opaque option. You can also do this with a script using the provided runtime functions. See Rendering settings for additional information.

To enable or disable rendering the image from the video pass-through cameras, use the following methods:

// Start rendering the video see-through image
VarjoMixedReality.StartRender();

// Stop rendering the video see-through image
VarjoMixedReality.StopRender();

When enabled, you will see the image from the video pass-through cameras when your application renders 0 in the color buffer. You can do this either by using a stencil mask or by making the camera clear to 0. For instructions on how to set up a stencil mask, see Masking. To make the camera clear to 0, set the camera’s Clear Flags to Solid Color and set RGBA(0,0,0,0) as the background color.

You can find instructions for High Definition Render Pipeline and Universal Render Pipeline below.

Using mixed reality with High Definition Render Pipeline

When using HDRP, make sure to use a color buffer format with an alpha channel. In the Project Settings dialog, select the Quality tab and under HDRP settings select the HD Render Pipeline Asset. Set the Color Buffer Format dropdown to R16G16B16A16. You will need to do this for all HDRP assets in use.

To make the camera clear to 0, set the camera’s Background Type to Color and set RGBA(0,0,0,0) as the background color.

Using mixed reality with Universal Render Pipeline

When using URP, you need to modify the camera settings to be able to write into the alpha channel of the color buffer. For video pass-through to work, make sure the camera’s Post Processing and HDR settings are disabled.

To make the camera clear to 0, set the camera’s Background Type to Solid Color and set RGBA(0,0,0,0) as the background color.

Environment reflections

Varjo XR Plugin can use the feed from the video pass-through cameras to create lighting conditions and reflections that match those in your current real world location. This is useful when blending your virtual content with the real world, making the scene appear more natural and realistic.

Shadow catcher

To cast shadows on the ground under your virtual objects, add a shadow catcher material to a flat plane set at ground level relative to the XR rig. For dynamic content, this can be done with the following graph. The graph will only show real-time shadows.