Migrating from Legacy API to Layer API

This migration guide describes the necessary steps to convert applications from using Legacy API (present in Varjo SDK versions <2.1) to Layer API (in Varjo SDK since release 2.0). The latter is the new, extendable Varjo API and the only Varjo integration method since release 2.1. Please refer to Rendering to Varjo headsets for more complete description of the Layer API.

Applications written on top of Legacy API will continue to work, but the preferred integration path is to migrate to Layer API to take any new functionality into use. For example, depth buffer submission is only supported with Layer API.

Initialization

During initialization, applications set up data structures that are required in the rendering loop. This mainly consists of graphics initialization and render texture creation.

Legacy API

With Legacy API, the graphics subsystem is initialized with the following call:

// DX11
graphicsInfo = varjo_D3D11Init(session, device, varjo_TextureFormat_R8B8G8A8_SRGB, nullptr);
    
// OpenGL
graphicsInfo = varjo_GLInit(session, varjo_TextureFormat_R8B8G8A8_SRGB, nullptr);

The returned graphicsInfo contains the textures and information about the atlas dimensions, which includes four views for current Varjo devices: two for context screens and two for focus screens. These textures are always returned, even if the application will render into a separate backbuffer.

Layer API

With Layer API, the desired view configuration must be chosen first. After this, the recommended texture sizes can be queried with

void varjo_GetTextureSize(struct varjo_Session* session, varjo_TextureSize_Type type, int32_t viewIndex, int32_t* width, int32_t* height)

Next step is to create swapchain(s). In short, swapchains are textures that the application can render to and submit to Varjo compositor. Applications have the freedom to choose how they organize the texture data: it is possible to create one swapchain which will contain all views (atlas), or separate swapchains for each view. Separate swapchains are a little bit easier due to trivial viewport calculations. If one atlas (or any combination thereof) is desirable, the application must layout views in such an atlas. The same textures formats are supported with Layers API as well.

Legacy API lays the views in the following pattern:

Left Context Right Context
Left Focus Right Focus

With Layer API applications no longer need to be tied to exactly 4 views but can render for example only 2 views.

A swapchain is created with following call:

// D3D11
struct varjo_SwapChain* varjo_D3D11CreateSwapChain(struct varjo_Session* session, struct ID3D11Device* dev, struct varjo_SwapChainConfig2* config)
// OpenGL
struct varjo_SwapChain* varjo_GLCreateSwapChain(struct varjo_Session* session, struct varjo_SwapChainConfig2* config)

The same call is used to create different types of swapchains, e.g. color and depth swapchains.

Note: Legacy API allowed applications to submit textures owned by the application. However, with Layer API one must always submit the swapchains created by the calls above. Applications that render into their own textures need to copy into Varjo swapchains with each frame (this operation was done under the hood with Legacy API). The preferred method is to create Varjo swapchains and directly render into them to avoid the extra copy.

Data structures

Legacy API

To create varjo_FrameInfo and varjo_SubmitInfo, the following functions are used:

struct varjo_FrameInfo* varjo_CreateFrameInfo(struct varjo_Session* session)
struct varjo_SubmitInfo* varjo_CreateSubmitInfo(struct varjo_Session* session)

Layer API

varjo_SubmitInfo is not needed anymore. varjo_FrameInfo works the same as in Legacy API.

Rendering

Frame setup

Applications must call varjo_WaitSync to wait for the perfect moment to start rendering (computed from the rendering times (CPU and GPU) of the previous frames). Waiting can be done in a separate thread. This part works the same for both Legacy and Layer APIs.

After varjo_WaitSync returns, application can start rendering.

Legacy API

Applications should call

void varjo_BeginFrame(struct varjo_Session* session, struct varjo_SubmitInfo* submitInfo)

when they begin the rendering work for the given frame.

Layer API

Since there is no need for varjo_SubmitInfo, there is another version of varjo_BeginFrame to be used with Layer API:

void varjo_BeginFrameWithLayers(struct varjo_Session* session)

Similar to Legacy API, this function should be called just before the rendering work for the active frame begins.

Rendering and submitting the frame

There are two main usage patterns:

  1. Application renders into its own texture Legacy API, Layers API
  2. Application renders into a Varjo swapchain texture directly Legacy API, Layers API

Legacy API supported both cases. However, the first option induced a transparent copy behind the scenes from the application texture to Varjo swapchain texture. Since Layer API only allows Varjo swapchain textures to be submitted, applications must now execute the copy themselves.

Legacy API

With Legacy API, applications can directly render the frame after varjo_BeginFrame and can submit it with varjo_EndFrame.

Application renders into own texture (or textures)

varjo_GraphicsInfo* graphicsInfo = varjo_D3D11Init(session, device, varjo_TextureFormat_R8B8G8A8_SRGB, nullptr);

varjo_SwapChainConfig config = varjo_GetDefaultSwapChainConfig(session);
Renderer::Target tgt = renderer.InitRenderTarget(config.textureWidth, config.textureHeight);

for (int i = 0; i < graphicsInfo->viewCount; ++i) {
   // Application will render into it's own texture, and runtime will copy it to Varjo swapchain
   submitInfo->textures[i] = varjo_FromD3D11Texture(renderer.GetBackBuffer());
}

varjo_Nanoseconds startTime = varjo_GetCurrentTime(session);
while (!gotKey()) {
  varjo_WaitSync(session, frameInfo);
  float time = (frameInfo->displayTime - startTime) / 1000000000.0f;

  varjo_BeginFrame(session, submitInfo);

  for (int i = 0; i < graphicsInfo->viewCount; i++) {
    varjo_Viewport viewport = submitInfo->viewports[i];
    varjo_ViewInfo info = frameInfo->views[i];
    tgt.RenderView(viewport.x, viewport.y, viewport.width, viewport.height,
      info.projectionMatrix, info.viewMatrix, time);
  }

  varjo_EndFrame(session, frameInfo, submitInfo);
}

Application renders into swapchain

varjo_GraphicsInfo* graphicsInfo = varjo_D3D11Init(session, device, varjo_TextureFormat_R8B8G8A8_SRGB, nullptr);

varjo_SwapChainConfig config = varjo_GetDefaultSwapChainConfig(session);
std::vector<Renderer::Target> renderTargets;
renderTargets.resize(config.numberOfTextures);

// Create as many render targets as there are swapchain textures
for (int32_t i = 0; i < graphicsInfo->swapChainTextureCount; i++) {
    varjo_Texture swapChainTexture = graphicsInfo->swapChainTextures[i];
    renderTargets[swapChainIndex] = renderer.InitRenderTarget(
        config.textureWidth, config.textureHeight, varjo_ToD3D11Texture(swapChainTexture));
}

varjo_Nanoseconds startTime = varjo_GetCurrentTime(session);
while (!gotKey()) {
  varjo_WaitSync(session, frameInfo);
  float time = (frameInfo->displayTime - startTime) / 1000000000.0f;

  varjo_BeginFrame(session, submitInfo);

  int swapChainIndex = varjo_GetSwapChainCurrentIndex(session);
  for (int i = 0; i < graphicsInfo->viewCount; i++) {
    varjo_Viewport viewport = submitInfo->viewports[i];
    varjo_ViewInfo info = frameInfo->views[i];
    renderTargets[swapChainIndex].RenderView(viewport.x, viewport.y, viewport.width, viewport.height,
      info.projectionMatrix, info.viewMatrix, time);

    submitInfo->viewports[i] = viewport;
    submitInfo->textures[i] = graphicsInfo->swapChainTextures[i];
  }

  varjo_EndFrame(session, frameInfo, submitInfo);

Layer API

Before a swap chain image is used (as a render target or a target in a copy routine) it has to be acquired with varjo_AcquireSwapChainImage(). Function returns the index of the texture to which application should render/copy frame.

After rendering is completed, swapchain image has to be released with varjo_ReleaseSwapChainImage().

After the application has rendered the frame into Varjo swapchains (and released these swapchains), the swapchain textures (or parts of them) can be submitted as layers to Varjo compositor.

You can read more about layers in Varjo rendering documentation. To match Legacy API behavior, applications must submit one layer of type varjo_LayerMultiProj, which in turn must be filled with four views, each denoted with varjo_LayerMultiProjView. This structure contains information about projection matrix, view matrix, and the source viewport for the view. Viewport (varjo_SwapChainViewport) defines an area on the swap chain from which compositor should sample the view data.

Layer API adds a possibility to submit a depth buffer with the color buffer. Applications should always submit depth when possible as this enables positional timewarp and improves image quality when native frame rates cannot be attained. Refer to Varjo rendering documentation on how to submit a depth buffer.

Application renders into own texture (or textures).

Note: When application is rendering into its own texture, one texture is enough.

const std::vector<varjo_Viewport> viewports = calculateViewports(session);

varjo_SwapChainConfig2 config2{};
config2.numberOfTextures = 3;
config2.textureArraySize = 1;
config2.textureFormat = varjo_TextureFormat_R8G8B8A8_SRGB;
config2.textureWidth = getTotalWidth(viewports);
config2.textureHeight = getTotalHeight(viewports);
varjo_SwapChain* swapchain = varjo_D3D11CreateSwapChain(session, renderer.GetD3DDevice(), &config2);

// Since we are not providing any textures to our render target we need just one render target
// and our implementation will create one
Renderer::Target renderTarget;
renderTarget.InitRenderTarget(config2.textureWidth, config2.textureHeight);

const int32_t viewCount = varjo_GetViewCount(session);
std::vector<varjo_LayerMultiProjView> views;
views.resize(viewCount);

varjo_Nanoseconds startTime = varjo_GetCurrentTime(session);
while (!gotKey()) {
   varjo_WaitSync(session, frameInfo);

   float time = (frameInfo->displayTime - startTime) / 1000000000.0f;

   varjo_BeginFrameWithLayers(session);
   renderer.Clear(renderTarget);

   for (int i = 0; i < viewCount; i++) {
      varjo_Viewport viewport = viewports[i];
      varjo_ViewInfo info = frameInfo->views[i];
      renderTarget.RenderView(viewport.x, viewport.y, viewport.width, viewport.height, info.projectionMatrix, info.viewMatrix, time);
   }

   int swapChainIndex;
   varjo_AcquireSwapChainImage(swapchain, &swapChainIndex);
   varjo_Texture swapchainTexture = varjo_GetSwapChainImage(session, swapChainIndex);
   // Copy texture into swapchain, on DirectX it can use CopyResource(), on OpenGL glCopyImageSubData()
   renderTarget.copy(swapchainTexture);
   varjo_ReleaseSwapChainImage(swapchain);

   for (int i = 0; i < viewCount; i++) {
      const varjo_ViewInfo viewInfo = frameInfo->views[i];
      const varjo_Viewport viewport = viewports[i];
      views[i].viewport = varjo_SwapChainViewport{swapchain, viewport.x, viewport.y, viewport.width, viewport.height, 0};
      memcpy(views[i].projection.value, viewInfo.projectionMatrix, sizeof(double) * 16);
      memcpy(views[i].view.value, viewInfo.viewMatrix, sizeof(double) * 16);
   }


   varjo_LayerMultiProj projLayer{{varjo_LayerMultiProjType, 0}, 0, viewCount, views.data()}; 
   varjo_LayerHeader* layers[] = {&projLayer.header};
   varjo_SubmitInfoLayers submitInfoWithLayers{frameInfo->frameNumber, 0, 1, layers};

   varjo_EndFrameWithLayers(session, &submitInfoWithLayers);
}

Application renders into swapchain

const std::vector<varjo_Viewport> viewports = calculateViewports(session);

varjo_SwapChainConfig2 config2{};
config2.numberOfTextures = 3;
config2.textureArraySize = 1;
config2.textureFormat = varjo_TextureFormat_R8G8B8A8_SRGB;
config2.textureWidth = getTotalWidth(viewports);
config2.textureHeight = getTotalHeight(viewports);
varjo_SwapChain* swapchain = varjo_D3D11CreateSwapChain(session, renderer.GetD3DDevice(), &config2);

// Setup render targets
std::vector<Renderer::Target> renderTargets(config2.numberOfTextures);
for (int swapChainIndex = 0; swapChainIndex < config2.numberOfTextures; swapChainIndex++) {
    varjo_Texture swapChainTexture = varjo_GetSwapChainImage(swapchain, swapChainIndex);
    renderTargets[swapChainIndex] = renderer.InitRenderTarget(
        config2.textureWidth, config2.textureHeight, varjo_ToD3D11Texture(swapChainTexture));
}

const int32_t viewCount = varjo_GetViewCount(session);
std::vector<varjo_LayerMultiProjView> views(viewCount);

varjo_Nanoseconds startTime = varjo_GetCurrentTime(session);
while (!gotKey()) {
  varjo_WaitSync(session, frameInfo);

   float time = (frameInfo->displayTime - startTime) / 1000000000.0f;

   varjo_BeginFrameWithLayers(session);

   int swapChainIndex;
   varjo_AcquireSwapChainImage(swapchain, &swapChainIndex);
   renderer.Clear(renderTargets[swapChainIndex]);

   for (int i = 0; i < viewCount; i++) {
      varjo_Viewport viewport = viewports[i];
      varjo_ViewInfo info = frameInfo->views[i];
      renderTargets[swapChainIndex].RenderView(viewport.x, viewport.y, viewport.width, viewport.height, info.projectionMatrix, info.viewMatrix, time);
   }

   varjo_ReleaseSwapChainImage(swapchain);

   for (int i = 0; i < viewCount; i++) {
      const varjo_ViewInfo viewInfo = frameInfo->views[i];
      const varjo_Viewport viewport = viewports[i];
      views[i].viewport = varjo_SwapChainViewport{swapchain, viewport.x, viewport.y, viewport.width, viewport.height, 0};
      memcpy(views[i].projection.value, viewInfo.projectionMatrix, sizeof(double) * 16);
      memcpy(views[i].view.value, viewInfo.viewMatrix, sizeof(double) * 16);
   }

   varjo_LayerMultiProj projLayer{{varjo_LayerMultiProjType, 0}, 0, viewCount, views.data()};
   varjo_LayerHeader* layers[] = {&projLayer.header};
   varjo_SubmitInfoLayers submitInfoWithLayers{frameInfo->frameNumber, 0, 1, layers};

   varjo_EndFrameWithLayers(session, &submitInfoWithLayers);
}

Shutdown

Shutdown Legacy API

void varjo_D3D11ShutDown(struct varjo_Session* session)

or

void varjo_GLShutDown(struct varjo_Session* session)



void varjo_FreeFrameInfo(struct varjo_FrameInfo* frameInfo)
void varjo_FreeSubmitInfo(struct varjo_SubmitInfo* submitInfo)
void varjo_SessionShutDown(struct varjo_Session* session)

Shutdown Layers API

void varjo_FreeSwapChain(struct varjo_SwapChain* swapChain)
void varjo_FreeFrameInfo(struct varjo_FrameInfo* frameInfo)
void varjo_SessionShutDown(struct varjo_Session* session)