Chroma keying

Chroma keying is a video technique where a predefined color is replaced with virtual content. Usually the color to be replaced is bright green or bright blue, as those colors contrast the most with human skin.

See Chroma key for more general information about the feature. Follow the steps below to use chroma keying in your project.

Using chroma keying

Enable the mixed reality cameras by following the instructions in Mixed reality with Varjo XR Plugin.

To enable or disable chroma keying, use a method in the VarjoChromaKey class:

// Check if chroma key is enabled.
VarjoChromaKey.IsChromaKeyEnabled();

// Enable or disable chroma keying.
VarjoChromaKey.EnableChromaKey(bool enabled);

Configurations

Chroma key configurations are indexed to support multiple configurations. All configurations are combined into a single mask.

Each chroma key configuration index has VarjoChromaKeyConfigType, which can be used to determine if the slot in the current index is being used. Active configurations have their parameters defined in HSV format as VarjoChromaKeyParams.

VarjoChromaKeyConfigType is an enum that determines the type of the configuration:

  • 0 – Disabled: The chroma key config in the index is not in use.
  • 1 – HSV: The chroma key config in the index is of type, HSV.

VarjoChromaKeyParams contains HSV parameters for the chroma key configuration:

  • float hue - Chroma key color tone (0.0 - 1.0).
  • Vector3 hsvTolerance - HSV tolerances (0.0 - 1.0). x: Tolerance for color variation (H), y: Tolerance for bright and pale areas (S), z: Tolerance for dark and shaded areas (V).

Use the following methods to get available configuration slots and to read the parameters of active configurations:

// Get the number of possible configuration slots.
VarjoChromaKey.GetChromaKeyConfigCount();

// Get VarjoChromaKeyConfigType of the configuration in the given index.
VarjoChromaKey.GetChromaKeyConfigType(int index);

// Get VarjoChromaKeyParams of the configuration in the given index.
VarjoChromaKey.GetChromaKeyParameters(int index);

As chroma keying is a global feature shared with all active clients, you will need to acquire a lock to modify any chroma key configurations. Locking and unlocking configurations is described below.

When a lock is acquired, you can modify the configurations in each index using the following methods:

// Set chroma key parameters for the given configuration index.
VarjoChromaKey.SetChromaKeyParameters(int index, VarjoChromaKeyParams parameters);

// Disable chroma key configuration in the given index.
VarjoChromaKey.DisableChromaKeyConfig(int index);

Varjo Base chroma key UI

The chroma key UI in Varjo Base provides an easy way to set chroma key parameters.

You can use the UI to find the optimal parameters and save them in JSON format. Your application can then read the saved values and set the appropriate chroma key parameters.

The values in the JSON file map to VarjoChromaKeyParams as follows:

varjoChromaKeyParams.hue = json.config.targetColor[0];
varjoChromaKeyParams.hsvTolerance = new Vector3(json.config.tolerances[0], json.config.tolerances[1], json.config.tolerances[2]);