Mixed Reality
Note: you are currently viewing documentation for a beta or an older version of Varjo
Mixed reality support
Enable mixed reality
For Varjo XR-3, you can enable mixed reality by enabling blend mode. This can be done with the XR_ENVIRONMENT_BLEND_MODE_ALPHA_BLEND
flag. Detailed specifications can be found in the OpenXR specifications.
XrFrameEndInfo frameEndInfo{XR_TYPE_FRAME_END_INFO};
frameEndInfo.displayTime = frameState.predictedDisplayTime;
frameEndInfo.environmentBlendMode = mrEnabled ? XR_ENVIRONMENT_BLEND_MODE_ALPHA_BLEND : XR_ENVIRONMENT_BLEND_MODE_OPAQUE;
frameEndInfo.layerCount = 1;
frameEndInfo.layers = layers;
Toggle depth estimation
Once the mixed reality cameras are enabled, Varjo runtime can estimate depth for incoming video frames. A typical use case is hand occlusion. To use depth estimation, enable the XR_VARJO_environment_depth_estimation
extension and call the following function:
xrSetEnvironmentDepthEstimationVARJO(session, XR_TRUE);
Depth estimation requires that the application submit a depth buffer as well. More information about this extension is available in the OpenXR specification.
Limit depth testing range
In most cases, mixing real-world and VR content should happen in a limited range. For example, an application may want to show hands on top of VR content but does not want to show the surrounding walls. In this case XR_VARJO_composition_layer_depth_test
can be used to enable depth testing only at a certain range.
The following example is taken from the specification and enables depth testing at a distance of 0 to 1 meter:
XrCompositionLayerProjection layer{XR_TYPE_COMPOSITION_LAYER_PROJECTION};
layer.space = ...;
layer.viewCount = ...;
layer.views = ...;
layer.layerFlags = ...;
XrCompositionLayerDepthTestVARJO depthTest{XR_TYPE_COMPOSITION_LAYER_DEPTH_TEST_VARJO, layer.next};
depthTest.depthTestRangeNearZ = 0.0f; // in meters
depthTest.depthTestRangeFarZ = 1.0f; // in meters
layer.next = &depthTest
In this example, between 0 and 1 meter the compositor will take depth into account from both the video layer and the application layer. From 1 meter on, it will ignore depth completely and VR content will overwrite pixels from the video stream.
Marker tracking
The Varjo XR-3 supports marker tracking. The Varjo OpenXR runtime fully implements the XR_VARJO_marker_tracking
extension (since Varjo Base 3.3). See the official specification for details.