Masking with Varjo XR plugin

When developing for XR-1 Developer Edition you can mask specific objects to act as a “window” to the real world. This is especially useful if you want to see real world elements such as instruments or controller devices in your virtual environment.

Varjo Compositor uses the alpha value of the layer to blend between the image from the client application and the image from the video see-through (VST) cameras. Every pixel in the color buffer with alpha value smaller than 1.0 will be blended with the VST image if VST rendering is enabled and the layer is not flagged as opaque. Only VST image is visible if the color of the pixel in the color buffer is RGBA(0,0,0,0).

Follow these simple steps to add a VST mask in your Unity project.

Masking with High Definition Render Pipeline

Enable mixed reality for Varjo XR Plug-in (see the document Mixed Reality With Varjo XR Plug-in).

Start by adding a mesh in your scene. This mesh will be the mask for the VST image. The mesh can be any shape you like. In this example we use a simple Quad placed in the doorway.

Create a new material for the mask. For this material, you need to assign a shader that is able to export alpha, but also writes in the depth buffer. You can use HDRP’s default Unlit shader or create a custom one. If using HDRP/Unlit, make sure the Surface Type is ‘Opaque’ and set the Color to RGBA(0,0,0,0). As Varjo layers use premultiplied alpha, having just the Alpha channel 0 is not enough, but you also need to make sure the RGB color is black with the default Unlit shader.

Assign the created material for the mask mesh.

Your VST mask is now ready. Click Play and you should be able to see into the real world through the mesh you added.

Please note that the VST image is not visible in the Unity Editor. You can see it inside the headset and in the Headset view in Varjo Base.