Point cloud

The Varjo XR plugin features experimental support for generating point clouds using the LiDAR in the Varjo XR-3 headset.

Using point clouds with Varjo XR plugin

The point cloud support in the Varjo XR plugin is provided as an XR Meshing subsystem. You can find more general information on using XR Meshing subsystems in Unity’s documentation.

The easiest way to add support for generating point clouds in your application is by using the provided VarjoReconstruction MonoBehaviour:

• Add VarjoReconstruction to one of the active Game Objects in the scene and set Mesh Parent and Mesh Prefab.
• Use the prefab to set the desired material for rendering the point cloud. The prefab should have at least the Mesh Filter and Mesh Renderer components.
• A prefab called PointMesh is provided with the plugin and can be used for simple point mesh rendering.

VarjoReconstruction starts generating a point cloud of the environment immediately when the component is enabled. The point cloud is provided as a mesh with MeshTopology.Points.

In addition to positions and colors, some additional mesh data arrays are used for other types of data. The following data is provided for each mesh:

• Positions - A tracking space position for each point.
• Colors - 32-bit color for each point.
• Normals - A normal direction for each point.
• UVs - The x-component gives a radius for each point in meters. The y-component gives a normalized confidence value for each point.

It is not required that you use the provided VarjoReconstruction component. You can also implement your own C# component following the XR SDK Meshing documentation and call the XR Meshing subsystem methods directly.