Operational Control API

• 1: Overview of JSON protocol
• 2: Rendering Engine components
• 3: Control API reference

1 - Overview of JSON protocol

This is a description of the network protocol used to control, monitor and explore the components in a running Ateliere Live production.

The JSON protocol is used when connected to the Websocket Control Panel (acl-websocketcontrolpanel) and has multi-client support, making it possible for clients that are connected to the same production to mirror each other’s controls.

The JSON protocol sits on top of an application-independent platform protocol which transports the JSON payload between the clients and the production.

Control Panel implementation

Note: This section is mainly needed if you plan on implementing a new control panel for the Atelier Live Rendering Engine using the C++ SDK.

Most of the JSON messages described in this document are sent and received by the Websocket Control Panel using the ControlDataSender interface. The interface encapsulates the packing and unpacking of messages to and from the application-agnostic platform protocol. The platform protocol includes the addresses for the sender and receiver(s) of the messages but is unaware of the contents of the packages.

The ControlDataSender interface defines the callbacks that should be used for handling messages coming from the production. Response and status messages are received from the production by setting the mResponseCallback and mStatusMessageCallback, respectively. The data received in these callbacks are complete JSON messages conforming to the specification in this document and can be forwarded without changes to the connected clients. There are however some exceptions to this.

Events

Messages of the type event currently signals that a connection was opened or closed in the production. Event messages originate in the control panel meaning that implementing support for this message type is part of making a control panel using the C++ SDK.

The control panel gets the events from the underlying platform through the ControlDataSender::Settings::mConnectionEventCallback. By inspecting the ConnectionEvent provided in the callback, clients connected to the control panel can be informed about the event. This can be used by other control panels, implemented using the C++ SDK.

Please see the section about the event message type under “Message types” for information about the message syntax.

Hop count

Messages of different types need to propagate to different depth of the production. A hop count in each message determines if it will be relayed from one Rendering Engine to a possibly chained one behind it.

When receiving a JSON message from a client, the Websocket Control Panel will select a hop count based on the content of the message, before sending the message to the production.

The hop count is marked in the platform layer of the protocol when sending a message using ControlDataSender::sendRequestToReceivers. It is not visible in the JSON message.

For each type of message, the hop count used by the Websocket Control Panel is specified in a “Hop count” section within “Message types” below. This hop count should be used when implementing a new C++ control panel.

The state tree

The control API is based around a state tree that makes up the hierarchy of parameters and entities in the Rendering Engine that can be controlled or monitored. The tree is expressed as a JSON structure.

An entity in the tree is addressed by a path consisting of the names of each enclosing object surrounding the entity.

For instance, in the structure below, the path to the g-field of the rgb_filter would be /video/nodes/rgb_filter/g:

{
    "video": {
        "nodes": {
            "rgb_filter": {
                "r": 0.5,
                "g": 0.4,
                "b": -0.2
            }
        }
    }
}

Message format

JSON syntax is used for all messages. The type and resource fields are present in all messages and define the type of message and what resource that is affected by this message.

{
    "type": "...",
    "resource": "..."
}

Message types

All changes that can be made to a production are available in the JSON API through messages of type set or command.

A parameter that can be changed immediately, without side effects, and that will keep its state until the next change can be accessed directly with a set message. In other cases a command is necessary to initiate the change. A set message might for instance be used for controlling the layer opacity in an alpha-over node or the gain for an input to the audio mixer whereas a command is needed to initiate the loading of a media file or a transition effect spanning over a period of time.

To get the current state of a resource, a get message can be used. A resource can also be monitored for changes using a subscribe message. When a sub-resource is added or removed from the monitored resource, a state-add or state-remove message is sent to the subscriber. Other changes result in a state-change message.

Parameters that change continuously without any user input, such as the play position of a media player or the loudness in an audio output, are referred to as streaming parameters. Streaming parameters are monitored by sending a sampling-start message to the resource path of the streaming parameter and providing a time interval stating how often sampling-update messages should be sent.

Table of message types

All response messages contain the field timestamp. This is the time when the request message was handled, i.e. the timestamp of the frame processed on that time.

Each type of message is described in detail below.

get and get-response

A message of type get is used to retrieve the current state of a given resource within the production.

Required fields for get messages:

For instance:

{
    "type": "get",
    "resource": "/video/nodes"
}

The response is a get-response. A successful response contains a body object with the current state of the resource:

{
    "type": "get-response",
    "resource": "/video/nodes",
    "timestamp": 1731073800720000,
    "body": {
        ...
    }
}

The body has its root where the resource URI points.

An unsuccessful get has no body and contains an error message, for instance:

{
    "type": "get-response",
    "resource": "/video/nodes",
    "error": "No such resource: /video/nodes",
    "timestamp": 1731073800720000
}

NOTE: Repeatedly calling get is a very inefficient way of monitoring the state of a production. Please see the subscribe and state-change message sections for a better way.

Hop count

In a proxy-editing setup, i.e. when there is one pipeline behind another one, get messages will stop at the first one and the state for that pipeline will be returned. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to 1 (one) for get messages.

set and set-response

A message of type set is used to change the state of one or more components located at a specified resource path in the production. This can for instance be to change the currently selected video source of a transition node or to change the gain of an audio strip.

set messages are generally only available for changes which are immediate. Please see the command message section for details about changes that occur over a period of time.

Required parameters for set messages:

The body of a set request contains the new values of the fields to change within the resource path. The structure of the body does not need to be complete - only the fields that should be updated need to be included.

Example of a set request that changes program and preview within a node named my_transition_node:

{
    "type": "set",
    "resource": "/video/nodes/my_transition_node",
    "body": {
        "preview": 3,
        "program": 8
    }
}

The body does not need to be an object when setting a single item, just the value. Here is an example:

{
    "type": "set",
    "resource": "/audio/strips/1/filters/gain/value",
    "body": 8.6
}

The set-response does not contain a body with the set values - instead changes in the production need to be monitored by subscribing to the affected resources. Please see the subscribe message section for details about subscriptions.

If there is an error, a set-response is returned containing an error message. In this case none of the parameters in the body of the set message will be applied. If the set was successful the response will contain a result ok key/value.

Hop count

In a proxy-editing setup, i.e. when there is one pipeline behind another one, set messages will propagate all the way to the last pipeline while following the configured alignment delay of each step. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to -1 (negative one) for set messages.

command and command-response

Commands are used to initiate events in the production which are not immediate or that do not have a direct relationship to one specific field of the production state tree. For instance, when starting playback of a media file there might be a play command. This will start the actual playback but also change the player state (e.g. from paused to playing) and also start to periodically update the current playback position field.

Required parameters for command messages:

Example of a command that starts a fade command on a transition node:

{
    "type": "command",
    "resource": "/video/nodes/my_transition_node",
    "body": {
        "command": "fade",
        "parameters": {
          "duration_ms": 120
        }
    }
}

Example of a command that takes no parameters:

{
    "type": "command",
    "resource": "/video/nodes/my_transition_node",
    "body": {
        "command": "cut"
    }
}

As for set-response messages, the command-response has no body - instead changes in the production need to be monitored by subscribing to one or more resources. Please see the subscribe message section for details about subscriptions.

If there is an error, a command-response is returned containing an error message. If the command was successful the response will contain a result ok key/value.

Hop count

In a proxy-editing setup, i.e. when there is one pipeline behind another one, command messages will propagate all the way to the last pipeline while following the configured alignment delay of each step. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to -1 (negative one) for command messages.

subscribe and subscribe-response

Subscriptions are used to monitor changes in the production. This can e.g. be useful in order to mirror the controls of different control surfaces that are connected to the same production, detect when new resources appear, or to otherwise visualize different parts of the system.

Required parameters for subscribe messages:

Example of a subscribe message for tracking changes of the resource /audio/strips/3/compressor:

{
    "type": "subscribe",
    "resource": "/audio/strips/3/compressor"
}

The response is a subscribe-response. A successful response contains a body object with the current state of the resource, for instance:

{
    "type": "subscribe-response",
    "resource": "/audio/strips/3/compressor",
    "body": {
        "attack": 30,
        "gain": 1,
        "knee": 3.5,
        "ratio": 3.3,
        "release": 2000,
        "threshold": -24,
        "type": "compressor"
    },
    "timestamp": 1731073800720000
}

If there is an error, a subscribe-response is returned containing an error message instead of a body.

Whenever a monitored resource changes, a state-change, state-add or state-remove message is sent to all subscribers of that resource. Please see the state-change, state-add and state-remove message sections for details about this.

Websocket Control Panel

In a proxy-editing setup, i.e. when there is one pipeline behind another one, subscribe messages will stop at the first one and it is from this pipeline that state changes will be reported. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to 1 for subscribe messages.

unsubscribe and unsubscribe-response

To stop receiving updates when a monitored resource changes, an unsubscribe message can be used.

Required parameters for unsubscribe messages:

If the unsubscribe message is successful, an unsubscribe-response is returned:

{
    "type": "unsubscribe-response",
    "resource": "/audio/strips/3/compressor",
    "timestamp": 1731073800720000
}

If there is an error, an unsubscribe-response is returned containing an error message instead of a body.

Hop count

In a proxy-editing setup, i.e. when there is one pipeline behind another one, unsubscribe messages will stop at the first receiver of the message. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to 1 for unsubscribe messages.

state-change

A state-change message describes the new state for a number of fields within a specified resource. state-change messages are sent from the production side, i.e. they are only received, never sent, by clients.

state-change messages are sent whenever the internal state of the production changes, for instance when:

• a client has issued a set message
• a command that changes the state is being executed
• an automation changes the state over a period of time
• a component, such as a metering device, has new data to report

To avoid loops or unnecessary updates on the client side, the message includes an actor field stating whether the receiver of the message or some other entity caused the change to happen.

Format of the state-change message:

Below is an example state-change message where the client receiving the message also caused the change to happen:

{
    "type": "state-change",
    "resource": "/audio/strips/3/compressor",
    "actor": "self",
    "body": {
        "gain": 2.5,
        "ratio": 4.0
    }
}

state-add and state-remove

The state-add and state-remove messages are similar to the state-change message. The top level resource field is the resource the client subscribes to. In the body of the message there is another resource field telling what sub-resource has been added or removed.

Examples:

{
  "type": "state-add",
  "resource": "/audio",
  "actor": "system",
  "body": {
    "resource": "/strips/2"
  }
}

{
  "type": "state-remove",
  "resource": "/audio",
  "actor": "system",
  "body": {
    "resource": "/strips/2"
  }
}

subscription-list and subscription-list-response

The subscription-list message is used to list own subscriptions. The response message shows the client’s subscriptions, located under a given resource point in the tree.

Required fields for subscription-list messages:

For instance:

{
    "type": "subscription-list",
    "resource": "/video"
}

The response is a subscription-list-response. A successful response contains a body object with the current state of the resource:

{
  "type": "subscription-list-response",
  "resource": "/video",
  "timestamp": 1731414140040000,
  "body": {
    "subscriptions": [
      "/nodes/alpha_combine",
      "/nodes/alpha_over"
    ]
  }
}

The body has its root where the resource URI points.

An unsuccessful request has no body and contains an error message, for instance:

{
  "type": "subscription-list-response",
  "resource": "/video/nodes/over",
  "timestamp": 1731414382900000,
  "error": "Failed to find resource with name 'over'"
}

Hop count

In a proxy-editing setup, i.e. when there is one pipeline behind another one, subscription-list messages will stop at the first one, and the subscriptions for that pipeline will be returned. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to 1 for subscription-list messages.

describe and describe-response

The purpose of these messages is to get a description of some resource in the state tree, like a node in the video mixer component for example. The descriptions will list all commands and parameters which are valid for the resource.

Required fields for describe messages:

This is the description for /video/nodes/fade_to_black:

{
  "type": "describe-response",
  "resource": "/video/nodes/fade_to_black",
  "timestamp": 1731480819600000,
  "body": {
    "children": [],
    "commands": [
      {
        "command": "fade_from",
        "optional_parameters": [],
        "required_parameters": [...],
        ...
      },
      ...
    ],
    "parameters": [
    ...
    ]
  }
}

Hop count

In a proxy-editing setup, i.e. when there is one pipeline behind another one, describe messages will stop at the first receiver of the message. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to 1 for describe messages.

sampling-start and sampling-start-response

The sampling-start message is used to start sampling a streaming parameter. If successful, sampling-update messages will be sent to the subscriber periodically at the specified time inteval.

Required parameters for sampling-start messages:

An asterisk (*) can be used at any level of the resource expression to indicate “all components at this level”. If an asterisk is used, no other characters are allowed at that level of the path. For instance, "resource": "/audio/mixes/*/input_meter/*" can be used to start sampling all streaming parameters under input_meter for all audio mixes.

The interval_ms parameter must be greater than or equal to 40.

Example:

{
  "type": "sampling-start",
  "resource": "/audio/mixes/*/input_meter/*",
  "body": {
    "interval_ms": 100
  }
}

If there is an error, a sampling-start-response is returned containing an error message. If the sampling-start was successful the response will contain a result ok key/value.

sampling-stop and sampling-stop-response

The sampling-stop message is used to stop sampling a streaming parameter.

Required parameters for sampling-stop messages:

The resource parameter must match one used to start sampling exactly.

Example:

{
  "type": "sampling-stop",
  "resource": "/audio/mixes/*/input_meter/*"
}

If there is an error, a sampling-stop-response is returned containing an error message. If the sampling-stop was successful the response will contain a result ok key/value.

sampling-list and sampling-list-response

The sampling-list message is used to list the streaming parameter subscriptions that are registered to the client sending the message.

Required parameters for sampling-stop messages:

Example:

{
  "type": "sampling-list",
  "resource": "/"
}

If there is an error, a sampling-list-response is returned containing an error message. If the sampling-stop was successful the response will contain a list of all the sender’s streaming parameter subscriptions.

Example:

{
    "type": "sampling-list-response",
    "resource": "/",
    "body": {
        "samplings": [
            "/audio/strips/1/pre_filter_meter/*",
            "/audio/mixes/0/input_meter/*"
        ]
    },
    "timestamp": 1736860857880000
}

sampling-update

A sampling-update message describes the latest sampled state of one or more streaming parameters matching a subscription. The message is sent periodically to clients that have started subscriptions using sampling-start.

The message contains the resource expression used in sampling-start. If necessary this can be used to identify the subscription in the client.

The body of the message contains the updated state for all of the streaming parameters matching the subscription. Note that the body starts at the root of the state tree, which is different from e.g. get responses or subscriptions for non-streaming parameters.

Example:

{
    "type": "sampling-update",
    "resource": "/audio/strips/*/pre_filter_meter/*",
    "body": {
        "audio": {
            "strips": {
                "1": {
                    "pre_filter_meter": {
                        "peak": -0.439775225995602234
                    }
                },
                "2": {
                    "pre_filter_meter": {
                        "peak": -0.982362873895602837
                    }
                }
            }
        }
    },
    "timestamp": 1736861180080000
}

event

When connected to a Websocket Control Panel it will act as a proxy in front of the Rendering Engine. This introduces a message of type event, which is only used between the Websocket Control Panel and its clients. Messages of type event contain an event field specifying the type of event.

All clients connected to the Websocket Control Panel will receive events of type connect whenever a new connection is established between the Websocket Control Panel and a Rendering Engine. Such a message might look like:

{
    "type": "event",
    "connected_node": "<uuid>",
    "address": "<uuid>:<uuid>:...",
    "event": "connect"
}

where connected_node is the UUID of the newly connected node (Rendering Engine) and address is a colon separated list of UUIDs, which is the address to the node that discovered the connection. For connect events this is currently only the Websocket Control Panel itself, meaning the address is always the Websocket Control Panel’s.

When a connection between a client and a Websocket Control Panel is established, the client will receive connect events for all Rendering Engines that where already connected to the Websocket Control Panel, to let the client know it is connected to a production.

If the connection between the Websocket Control Panel and the Rendering Engine breaks, whether it is disconnected on purpose or gets disconnected due to a network outage, the Websocket Control Panel will send an event message to all its clients.

{
    "type": "event",
    "disconnected_node": "<uuid>",
    "address": "<uuid>:<uuid>:...",
    "event": "disconnect"
}

A similar message is also sent to the clients in case the connection between two Rendering Engines/Pipelines is torn down or lost, such as between a Low Delay and a High Quality Pipeline. In that case the address parameter will be the UUID of the Websocket Control Panel followed by the UUID of the LD Pipeline and the disconnected_node is the UUID of the HQ Pipeline.

2 - Rendering Engine components

This page describes the parameters and commands that is used for controlling the video mixer, audio mixer, HTML renderers and media players in the Ateliere Live Rendering Engine. This topic is closely related to this page on how to configure the rendering engine at startup.

Control command protocol

All commands to the Ateliere Live Rendering Engine are sent as human readable JSON objects and are listed below.

Each subsystem of the rendering engine has their own set of parameters and commands, and can be reached using:

• /video for the video mixer
• /audio for the audio mixer (if using the built-in mixer)
• /ndi_audio for the NDI audio mixer
• /html for the html renderer instances
• /media for the media playback instances

Video mixer resources

Background

The video mixer is built as a tree graph of processing nodes, please see this page for further information on the video mixer node graph. The names of the nodes defined in the node graph are used as part of the resources: /video/nodes/{node_name} and they all have the parameter type which describes which type of video node it is.

There are 10 different node types.

• The Transition node is used to pick which input slots to use for the program and preview output. The node also supports making transitions between the program and preview.
• The Select node simply forwards one of the available sources to its output.
• The Alpha combine node is used to pick two input slots (or the same input slot) to copy the color from one input and combine it with some information from the other input to form the alpha. The color will just be copied from the color input frame, but there are several modes that can be used to produce the alpha channel in the output frame in different ways. This is known as “key & fill” in broadcasting.
• The Alpha over node is used to perform an “Alpha Over” operation, that is to put the overlay video stream on top of the background video stream, and let the background be seen through the overlay depending on the alpha of the overlay. The node also features fading the graphics in and out by multiplying the alpha channel by a constant factor.
• The Transform node takes one input stream and transforms it (scaling and translating) to one output stream.
• The Chroma Key node takes one input stream, and by setting appropriate parameters for the keying, it will remove areas with the key color from the incoming video stream both affecting the alpha and color channels
• The Fade to black node takes one input stream, which it can fade to or from black gradually, and then outputs that stream.
• The Output node has one input stream and will output that stream out from the video mixer, back to the Rendering Engine. It has no control commands
• The Crop node takes one input stream and can crop a video stream to a new size
• The Video delay node takes one input stream and will hold the frames for a configured amount of time before forwarding the frames, causing a delay of the video output

To reset the runtime configuration of all video nodes to their default state use the reset command of the /video resource.

Transition

The transition node picks a program and a preview video source from the input slots and forward these to other nodes. The node also features auto transitions between the program and the preview sources. Some transition commands last over a duration of time, for example wipes. These can be performed either automatically or manually. The automatic mode works by the operator first selecting the type of transition, for instance a fade, setting the preview to the input slot to fade to and then trigger the transition at the right time with a auto command with the duration for the transition. In manual mode the exact position of the transition is set by the control panel by setting the factor parameter. This is used for implementing T-bars, where the T-bar repeatedly sends the current position of the bar. In the manual mode, the transition type is set before the transition begins, just as in the automatic mode. Note that an automatic transition will be overridden in case the transition position/factor is manually set, by interrupting the automatic transition and jumping to the manually set position.

resource: /video/nodes/{node name} type: transition

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "factor": 0.0,
        "mode": "fade",
        "preview": 0,
        "program": 0
    }
}

Commands

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "auto",
       "parameters": {
           "duration_ms": <uint32>
       }
   }
}

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "cut"
   }
}

Select

A node to select a video source from the input slots and send it on to the next node.

resource: /video/nodes/{node name} type: select

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "input": 0
    }
}

Alpha combine

A node to combine the color channels of one video stream with the alpha from another. This node is useful for video sources where the alpha channel is provided as a separate black and white video source that must be combined with the color source. The node supports multiple modes of obtaining the alpha, either by copying a specific color or alpha channel of some input slot, or by taking the average of the R, G and B channels of the video from some input slot.

resource: /video/nodes/{node name} type: alpha_combine

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "alpha": 0,
        "color": 0,
        "mode": "average-rgb"
    }
}

Alpha over

A node to combine two video streams using alpha over compositing, overlaying the foreground stream on the background stream. The node will keep the transparency of both layers. The overlay stream can be faded in and out of the background stream.

resource: /video/nodes/{node name} type: alpha_over

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "factor": 0.0
    }
}

Commands

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "fade_from",
       "parameters": {
           "duration_ms": <uint32>
       }
   }
}

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "fade_to",
       "parameters": {
           "duration_ms": <uint32>
       }
   }
}

Transform

A node to transform an incoming video stream, by scaling and transposing it. The canvas size of the input will be kept and all surrounding area in case the source video is shrunk, is filled with transparent black.

resource: /video/nodes/{node name} type: transform

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "scale": 1.0,
        "x": 0.0,
        "y": 0.0
    }
}

Chroma key

A node to perform chroma keying on an incoming video stream. The output video stream will have the alpha and possibly the color channels modified, according to the parameter values in this node. To remove a color from the incoming video stream, first enable the node and then select the key color to remove. The key color can be selected in two ways, either by manually setting the color with the R, G and B channel values, or by using the color picker. When using the color picker, the color picker command will define the position and size of the color picker square to sample the incoming video stream. The R, G and B color parameters will be updated according to the average color of the area when the command was received by the Rendering Engine. The currently selected color can be shown in the upper left hand corner in the output video stream of the node by setting the parameter show_key_color to true. Also, the latest sampled color picker area can be drawn in the node’s output by setting show_color_picker to true. When a suitable color has been chosen, adjust the distance and falloffparameters to get a clear mask. To aid the tweaking of the parameters, set the show_alpha parameter to true. This will make the node output the black and white mask instead of the keyed result, which makes it easier to see which parts are masked away and not. Remember to turn this off before going on air. As a last step, any remaining fringes of the key color around the subject can be desaturated with the color_spill parameter. But remember this will desaturate colors close to the key color even in parts of the frame fully visible.

resource: /video/nodes/{node name} type: chroma_key

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "color_spill": 0.1,
        "distance": 0.1,
        "enabled": false,
        "falloff": 0.08,
        "show_alpha": false,
        "show_color_picker": false,
        "show_key_color": false
    }
}

Commands

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "pick_color",
       "parameters": {
           "size": <uint32>,
           "x": <float>,
           "y": <float>
       }
   }
}

Chroma key / Key color

The key color

resource: /video/nodes/{node name}/key_color type: chroma_key

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}/key_color",
    "body": {
        "b": 0.0,
        "g": 0.0,
        "r": 0.0
    }
}

Fade to black

A node to fade the incoming video stream to and from black.

resource: /video/nodes/{node name} type: fade_to_black

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "factor": 0.0
    }
}

Commands

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "fade_from",
       "parameters": {
           "duration_ms": <uint32>
       }
   }
}

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "fade_to",
       "parameters": {
           "duration_ms": <uint32>
       }
   }
}

Crop

A node to crop the incoming video stream. The node can crop the left, right, top and bottom edge of the incoming video stream. The areas outside of the cropped area will be transparent in the output.

resource: /video/nodes/{node name} type: crop

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "bottom": 1.0,
        "left": 0.0,
        "right": 1.0,
        "top": 0.0
    }
}

Video delay

A node to delay the video stream a given number of frames.

resource: /video/nodes/{node name} type: video_delay

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "delay": 0
    }
}

Audio mixer resources

Overview

The audio mixer has the following data path:

The three main elements of the audio mixer are the Input strips, the Mixes, and the Outputs.

Input strips

Audio enters the audio mixer through the input strips. When a strip has been created, the audio for the strip is selected among the available channels in the Rendering Engine’s input slots.

In the example picture above, three sources are connected to the input slots. The slots have four, two, and six channels respectively. In the illustration we can see that the first strip uses a single channel from the first slot, the second strip uses channel three and four from the second slot and so on. Several input strips may select the same input slot channels.

Each input strip has two stereo output connection points (), one before the volume fader (pre fader) and one after the volume fader (post fader). When a mix or an output is connected to the output of a strip, which of these two locations to take the audio from can be selected using the origin parameter.

Mixes

Mixes are used to combine the output from input strips, or other mixes, and also allows for further filtering of the audio signal. When a mix has been created, the input strips and other mixes contributing to the mix can be controlled in the /inputs/strips and /inputs/mixes sections of the mix.

Just like an input strip a mix has two stereo output connection points () that can be used when connecting the output of the mix to another mix or an output.

Outputs

Each output corresponds to an audio output in the Rendering Engine configuration. An output takes audio from a single location, either directly from an input strip or from a mix. In both cases it can be specified whether to take the audio pre_fader or post_fader.

Note: The audio mixer is completely separate from the video mixer, so switching image in the video mixer will not change the audio mixer in any way.

Audio mixer root

The internal audio mixer consists of input strips (/strips), mixes (/mixes) and output buses (/outputs). The strips are the inputs of the audio mixer, taking audio from the inputs of the Rendering Engine. The mixes mix audio from strips and other mixes. Finally the output buses are the outputs of the audio mixer, taking audio from a single strip or a mix.

resource: /audio

Commands

{
   "type": "command",
   "resource": "/audio",
   "body": {
       "command": "reset"
   }
}

To reset the audio mixer to the default state, including removal of all configured input strips, use the reset command of the /audio resource.

Input strips

An input strip in the audio mixer, which takes audio from the input slots of the Rendering Engine (/input). Audio is either taken as a mono channel or as a stereo pair. The audio is sent through a filter chain (/filters). After the filter chain the output loudness of the strip is controlled by a main fader. There are peak meters placed before (/pre_fader_meter) and after (/post_fader_meter) the fader, as well as before the filter chain, measuring the raw audio input (/input_meter).

resource: /audio/strips/{strip index}

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}",
    "body": {
        "label": ""
    }
}

Input strip input configuration

Input settings for a strip.

resource: /audio/strips/{strip index}/input

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/input",
    "body": {
        "first_channel": 0,
        "input_slot": 0,
        "is_stereo": false,
        "second_channel": 1
    }
}

Input loudness meter

Peak loudness meter for incoming audio, measured before the filters of this input strip. This meter will either have one streaming parameter called ‘peak’ in case the strip is in mono mode, else two streaming parameters ‘peak_left’ and ‘peak_right’ will replace the ‘peak’ parameter when in stereo mode.

resource: /audio/strips/{strip index}/input_meter

Streaming parameters

Mid side stereo

Filter for controlling the mid and side amount of an audio signal. The input can either be Mid-Side (MS) or Left-Right (LR) stereo. Mono input will be passed through unaltered. If the input is LR, it is converted to MS with the mid channel being the average of the input channels and the side channel being half the difference of the channels. With the signal in MS format the mid and the side amount can be controlled using the mid_amount, side_amount, and invert_polarity parameters.

resource: /audio/strips/{strip index}/filters/mid_side

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/filters/mid_side",
    "body": {
        "enabled": false,
        "input_format": "lr_stereo",
        "invert_polarity": true,
        "mid_amount": 1.0,
        "side_amount": 1.0
    }
}

Pre-gain

Gain filter

resource: /audio/strips/{strip index}/filters/gain

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/filters/gain",
    "body": {
        "value": 0.0
    }
}

Parametric equalizer

Equalizer filter

resource: /audio/strips/{strip index}/filters/eq

Commands

{
   "type": "command",
   "resource": "/audio/strips/{strip index}/filters/eq",
   "body": {
       "command": "reset"
   }
}

Parametric equalizer bands

A filter/band in the equalizer

resource: /audio/strips/{strip index}/filters/eq/bands/{band index}

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/filters/eq/bands/{band index}",
    "body": {
        "freq": 1000.0,
        "gain": 0.0,
        "q": 0.707,
        "type": "none"
    }
}

Dynamic range compressor

Dynamic range compressor

resource: /audio/strips/{strip index}/filters/compressor

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/filters/compressor",
    "body": {
        "attack": 50.0,
        "gain": 0.0,
        "knee": 0.0,
        "ratio": 1.0,
        "release": 200.0,
        "threshold": 0.0
    }
}

Panning

Panning filter

resource: /audio/strips/{strip index}/filters/pan

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/filters/pan",
    "body": {
        "value": 0.0
    }
}

Pre-fader loudness meter

Peak loudness meter for filtered audio, measured after the filters of this input strip, but before the fader. This meter will always have two streaming parameters called ‘peak_left’ and ‘peak_right’, since the output of the strip is stereo.

resource: /audio/strips/{strip index}/pre_fader_meter

Streaming parameters

Input strip fader

Volume fader controlling the output loudness of this strip

resource: /audio/strips/{strip index}/fader

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/fader",
    "body": {
        "muted": false,
        "volume": 0.0
    }
}

Commands

{
   "type": "command",
   "resource": "/audio/strips/{strip index}/fader",
   "body": {
       "command": "fade",
       "parameters": {
           "volume": <float>,
           "duration_ms": <uint32>
       }
   }
}

Post-fader loudness meter

Peak loudness meter for filtered audio, measured after the fader and after the filters of this input strip. This meter will always have two streaming parameters called ‘peak_left’ and ‘peak_right’, since the output of the strip is stereo.

resource: /audio/strips/{strip index}/post_fader_meter

Streaming parameters

Mixes

A mix in the audio mixer, which takes audio from selected input strips and possibly other mixes to mix to a single stereo pair. After mixing the inputs (/inputs) of the mix, the audio is sent through a filter chain (/filters), similar to that of an input strip. After the filter chain the output loudness of the mix is controlled by a main fader. The inputs of a mix can either be taken pre- or post-fader from the strips and other mixes included in the mix. There are peak meters placed before (/pre_fader_meter) and after (/post_fader_meter) the fader, as well as before the filter chain, after the inputs has been mixed down to a single stereo pair (/input_meter).

resource: /audio/mixes/{mix index}

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}",
    "body": {
        "label": ""
    }
}

Mixes inputs

The inputs to this mix.

resource: /audio/mixes/{mix index}/inputs

Mixes inputs from input strips

Volume fader for controlling the contribution of the result of the other mix to this mix

resource: /audio/mixes/{mix index}/inputs/strips/{strip index}

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/inputs/strips/{strip index}",
    "body": {
        "muted": false,
        "origin": "post_fader",
        "volume": 0.0
    }
}

Commands

{
   "type": "command",
   "resource": "/audio/mixes/{mix index}/inputs/strips/{strip index}",
   "body": {
       "command": "fade",
       "parameters": {
           "volume": <float>,
           "duration_ms": <uint32>
       }
   }
}

Mixes inputs from other mixes

Volume fader for controlling the contribution of the result of the other mix to this mix

resource: /audio/mixes/{mix index}/inputs/mixes/{mix index}

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/inputs/mixes/{mix index}",
    "body": {
        "muted": false,
        "origin": "post_fader",
        "volume": 0.0
    }
}

Commands

{
   "type": "command",
   "resource": "/audio/mixes/{mix index}/inputs/mixes/{mix index}",
   "body": {
       "command": "fade",
       "parameters": {
           "volume": <float>,
           "duration_ms": <uint32>
       }
   }
}

Input loudness meter

Peak loudness meter for incoming audio, measured before the filters of this mix. This meter will always have two streaming parameters ‘peak_left’ and ‘peak_right’, since the input to the mix is always stereo.

resource: /audio/mixes/{mix index}/input_meter

Streaming parameters

Pre-gain

Gain filter

resource: /audio/mixes/{mix index}/filters/gain

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/filters/gain",
    "body": {
        "value": 0.0
    }
}

Parametric equalizer

Equalizer filter

resource: /audio/mixes/{mix index}/filters/eq

Commands

{
   "type": "command",
   "resource": "/audio/mixes/{mix index}/filters/eq",
   "body": {
       "command": "reset"
   }
}

Parametric equalizer bands

A filter/band in the equalizer

resource: /audio/mixes/{mix index}/filters/eq/bands/{band index}

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/filters/eq/bands/{band index}",
    "body": {
        "freq": 1000.0,
        "gain": 0.0,
        "q": 0.707,
        "type": "none"
    }
}

Dynamic range compressor

Dynamic range compressor

resource: /audio/mixes/{mix index}/filters/compressor

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/filters/compressor",
    "body": {
        "attack": 50.0,
        "gain": 0.0,
        "knee": 0.0,
        "ratio": 1.0,
        "release": 200.0,
        "threshold": 0.0
    }
}

Panning

Panning filter

resource: /audio/mixes/{mix index}/filters/pan

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/filters/pan",
    "body": {
        "value": 0.0
    }
}

Pre-fader loudness meter

Peak loudness meter for filtered audio, measured after the filters of this mix, but before the fader. This meter will always have two streaming parameters called ‘peak_left’ and ‘peak_right’, since the output of the mix is stereo.

resource: /audio/mixes/{mix index}/pre_fader_meter

Streaming parameters

Mix output fader

Volume fader controlling the output loudness of this mix

resource: /audio/mixes/{mix index}/fader

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/fader",
    "body": {
        "muted": false,
        "volume": 0.0
    }
}

Commands

{
   "type": "command",
   "resource": "/audio/mixes/{mix index}/fader",
   "body": {
       "command": "fade",
       "parameters": {
           "volume": <float>,
           "duration_ms": <uint32>
       }
   }
}

Post-fader loudness meter

Peak loudness meter for filtered audio, measured after the fader and after the filters of this mix. This meter will always have two streaming parameters called ‘peak_left’ and ‘peak_right’, since the output of the strip is stereo.

resource: /audio/mixes/{mix index}/post_fader_meter

Streaming parameters

Output bus

Audio mixer output bus to select which input strip or mix should be sent to the output of the audio mixer. The outputs are populated from the Rendering Engine config. Audio from a strip or a mix can be selected for the output. The selected source of this output can either be taken pre- or post-fader from the referenced strip or mix.This makes it possible for outputs meant for pre-listening to listen to a mix or strip without touching its fader. Such outputs can also jump freely between mixes to listen to what is being output from the mixer or strips to listen to the incoming components to tweak the audio filters. The selected audio is fed through a loudness meter.

resource: /audio/outputs/{output name}

Parameters

{
    "type": "set",
    "resource": "/audio/outputs/{output name}",
    "body": {
        "label": ""
    }
}

Output bus input

The source of an output bus.

resource: /audio/outputs/{output name}/input

Parameters

{
    "type": "set",
    "resource": "/audio/outputs/{output name}/input",
    "body": {
        "index": 0,
        "origin": "post_fader",
        "source": "mix"
    }
}

Output bus loudness meter

Loudness meters for the output bus. Used to monitor the loudness of the outgoing audio from the mixer.This meter will always have two streaming parameters called ‘peak_left’ and ‘peak_right’. When the parameter ’enable_ebu_meters’ is set to true, three additional streaming parameters will be available, called ’ebu_m’, ’ebu_s’ and ’ebu_i’, which measure loudness according to the EBU R 128 standard.

resource: /audio/outputs/{output name}/meters

Parameters

{
    "type": "set",
    "resource": "/audio/outputs/{output name}/meters",
    "body": {
        "enable_ebu_meters": false
    }
}

Streaming parameters

Commands

{
   "type": "command",
   "resource": "/audio/outputs/{output name}/meters",
   "body": {
       "command": "reset"
   }
}

External audio mixing via NDI

The internal audio mixer can be replaced with an external audio mixer, where NDI is used for communicating with the external mixer. To enable the external NDI audio mixer the Rendering Engine application should be started with the environment variable ACL_AUDIO_MIXER set to external_ndi.

The external NDI audio mixer bridge

resource: /ndi_audio

Sending audio to external mixer

All streams to send to the external audio mixer

resource: /ndi_audio/send_streams

Commands

{
   "type": "command",
   "resource": "/ndi_audio/send_streams",
   "body": {
       "command": "add_stream",
       "parameters": {
           "name": <string>
       }
   }
}

{
   "type": "command",
   "resource": "/ndi_audio/send_streams",
   "body": {
       "command": "remove_stream",
       "parameters": {
           "name": <string>
       }
   }
}

A stream to send to the external audio mixer

resource: /ndi_audio/send_streams/{send stream index}

Parameters

Commands

{
   "type": "command",
   "resource": "/ndi_audio/send_streams/{send stream index}",
   "body": {
       "command": "disable_channel",
       "parameters": {
           "index": <uint32>
       }
   }
}

{
   "type": "command",
   "resource": "/ndi_audio/send_streams/{send stream index}",
   "body": {
       "command": "enable_channel",
       "parameters": {
           "index": <uint32>,
           "input_slot": uint32,
           "channel": uint32
       }
   }
}

List of audio channels to send

resource: /ndi_audio/send_streams/{send stream index}/channels

A channel to send to the external audio mixer

resource: /ndi_audio/send_streams/{send stream index}/channels/{channel index}

Parameters

{
    "type": "set",
    "resource": "/ndi_audio/send_streams/{send stream index}/channels/{channel index}",
    "body": {
        "channel": 0,
        "input_slot": 0
    }
}

Receiving audio in return from the external mixer

All streams that returns audio from the external audio mixer

resource: /ndi_audio/return_streams

Commands

{
   "type": "command",
   "resource": "/ndi_audio/return_streams",
   "body": {
       "command": "add_stream",
       "parameters": {
           "name": <string>
       }
   }
}

{
   "type": "command",
   "resource": "/ndi_audio/return_streams",
   "body": {
       "command": "remove_stream",
       "parameters": {
           "name": <string>
       }
   }
}

A stream which returns audio from the external audio mixer

resource: /ndi_audio/return_streams/{return stream index}

Parameters

Mapping received audio to outputs

All audio outputs from this component

resource: /ndi_audio/outputs

An output from this component consisting of audio from one or more NDI return streams

resource: /ndi_audio/outputs/{output bus}

Parameters

The list of channels for one output

resource: /ndi_audio/outputs/{output bus}/channels

An output channel

resource: /ndi_audio/outputs/{output bus}/channels/1

Parameters

{
    "type": "set",
    "resource": "/ndi_audio/outputs/{output bus}/channels/1",
    "body": {
        "return_channel": 0,
        "return_stream": 0
    }
}

HTML rendering

The Rendering Engine features a built-in HTML renderer which uses the Chromium web browser engine to render HTML pages. This resource can create and close HTML renderers.

resource: /html

Commands

{
   "type": "command",
   "resource": "/html",
   "body": {
       "command": "close",
       "parameters": {
           "input_slot": <uint32>
       }
   }
}

{
   "type": "command",
   "resource": "/html",
   "body": {
       "command": "create",
       "parameters": {
           "input_slot": <uint32>,
           "width": <uint32>,
           "height": <uint32>,
           "url": string
       }
   }
}

{
   "type": "command",
   "resource": "/html",
   "body": {
       "command": "reset"
   }
}

HTML renderer

This resource controls an HTML renderer instance and allows loading of new URLs and executing JavaScript snippets.

resource: /html/{input slot}

Parameters

Commands

{
   "type": "command",
   "resource": "/html/{input slot}",
   "body": {
       "command": "execute",
       "parameters": {
           "javascript": <string>
       }
   }
}

{
   "type": "command",
   "resource": "/html/{input slot}",
   "body": {
       "command": "load",
       "parameters": {
           "url": <string>
       }
   }
}

Media players

The Rendering Engine can create media player instances to play video and audio files from the hard drive of the machine running the Rendering Engine. It is up to the user of the API to ensure the files are uploaded to the machines running the Rendering Engine(s) before trying to run them. The media players use the FFmpeg library to demux and decode the media files, so most files supported by FFmpeg should work. This resource can create and close media players.

resource: /media

Commands

{
   "type": "command",
   "resource": "/media",
   "body": {
       "command": "close",
       "parameters": {
           "input_slot": <uint32>
       }
   }
}

{
   "type": "command",
   "resource": "/media",
   "body": {
       "command": "create",
       "parameters": {
           "input_slot": <uint32>,
           "path": string
       }
   }
}

{
   "type": "command",
   "resource": "/media",
   "body": {
       "command": "reset"
   }
}

Media player

This resource controls a media player instance. The media players have three parameters that can be set to control if only a portion of the file should be played, and if the playback should loop once it reaches the end. See the section_start_ms, section_duration_ms and loop parameters below.

resource: /media/{input slot}

Parameters

{
    "type": "set",
    "resource": "/media/{input slot}",
    "body": {
        "loop": false,
        "section_duration_ms": 4294967295,
        "section_start_ms": 0
    }
}

Streaming parameters

Commands

{
   "type": "command",
   "resource": "/media/{input slot}",
   "body": {
       "command": "load",
       "parameters": {
           "path": <string>
       }
   }
}

{
   "type": "command",
   "resource": "/media/{input slot}",
   "body": {
       "command": "pause"
   }
}

{
   "type": "command",
   "resource": "/media/{input slot}",
   "body": {
       "command": "play"
   }
}

{
   "type": "command",
   "resource": "/media/{input slot}",
   "body": {
       "command": "seek",
       "parameters": {
           "time_ms": <int32>
       }
   }
}

3 - Control API reference

All available controllable resources have a unique resource path and are listed below. The resources may contain some or all of the following sections:

• Parameters
• Streaming parameters
• Commands

resource: /

{
   "type": "command",
   "resource": "/",
   "body": {
       "command": "reset"
   }
}

resource: /audio

{
   "type": "command",
   "resource": "/audio",
   "body": {
       "command": "reset"
   }
}

resource: /audio/mixes

{
   "type": "command",
   "resource": "/audio/mixes",
   "body": {
       "command": "add_mix",
       "parameters": {
           "index": <uint32>
       }
   }
}

{
   "type": "command",
   "resource": "/audio/mixes",
   "body": {
       "command": "remove_mix",
       "parameters": {
           "index": <uint32>
       }
   }
}

resource: /audio/mixes/{mix index}

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}",
    "body": {
        "label": ""
    }
}

resource: /audio/mixes/{mix index}/fader

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/fader",
    "body": {
        "muted": false,
        "volume": 0.0
    }
}

{
   "type": "command",
   "resource": "/audio/mixes/{mix index}/fader",
   "body": {
       "command": "fade",
       "parameters": {
           "volume": <float>,
           "duration_ms": <uint32>
       }
   }
}

resource: /audio/mixes/{mix index}/filters

resource: /audio/mixes/{mix index}/filters/compressor

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/filters/compressor",
    "body": {
        "attack": 50.0,
        "gain": 0.0,
        "knee": 0.0,
        "ratio": 1.0,
        "release": 200.0,
        "threshold": 0.0
    }
}

resource: /audio/mixes/{mix index}/filters/eq

{
   "type": "command",
   "resource": "/audio/mixes/{mix index}/filters/eq",
   "body": {
       "command": "reset"
   }
}

resource: /audio/mixes/{mix index}/filters/eq/bands

{
   "type": "command",
   "resource": "/audio/mixes/{mix index}/filters/eq/bands",
   "body": {
       "command": "add_band",
       "parameters": {
           "index": <uint32>
       }
   }
}

{
   "type": "command",
   "resource": "/audio/mixes/{mix index}/filters/eq/bands",
   "body": {
       "command": "remove_band",
       "parameters": {
           "index": <uint32>
       }
   }
}

resource: /audio/mixes/{mix index}/filters/eq/bands/{band index}

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/filters/eq/bands/{band index}",
    "body": {
        "freq": 1000.0,
        "gain": 0.0,
        "q": 0.707,
        "type": "none"
    }
}

resource: /audio/mixes/{mix index}/filters/gain

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/filters/gain",
    "body": {
        "value": 0.0
    }
}

resource: /audio/mixes/{mix index}/filters/pan

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/filters/pan",
    "body": {
        "value": 0.0
    }
}

resource: /audio/mixes/{mix index}/input_meter

resource: /audio/mixes/{mix index}/inputs