Reference

• 1: Ateliere Live 9.0.0 developer reference
• 1.1: System controller config
• 1.2: REST API v3
• 1.3: Rendering Engine config
• 1.4: Operational Control API
• 1.4.1: Overview of JSON protocol
• 1.4.2: Rendering Engine components
• 1.4.3: Control API reference
• 1.5: C++ SDK
• 1.5.1: Classes
• 1.5.1.1: Acl::AclLog::FileLocationFormatterFlag
• 1.5.1.2: Acl::AclLog::ThreadNameFormatterFlag
• 1.5.1.3: Acl::ControlDataAddress
• 1.5.1.4: Acl::ControlDataCommon::ConnectionEvent
• 1.5.1.5: Acl::ControlDataCommon::Response
• 1.5.1.6: Acl::ControlDataCommon::StatusMessage
• 1.5.1.7: Acl::ControlDataSender
• 1.5.1.8: Acl::ControlDataSender::Settings
• 1.5.1.9: Acl::ISystemControllerInterface
• 1.5.1.10: Acl::ISystemControllerInterface::Callbacks
• 1.5.1.11: Acl::ISystemControllerInterface::Response
• 1.5.1.12: Acl::SystemControllerConnection
• 1.5.1.13: Acl::SystemControllerConnection::Settings
• 1.5.1.14: Acl::UUID
• 1.5.1.15: fmt::formatter< Acl::ControlDataAddress >
• 1.5.1.16: fmt::formatter< Acl::ControlDataCommon::EventType >
• 1.5.1.17: fmt::formatter< Acl::ISystemControllerInterface::StatusCode >
• 1.5.1.18: fmt::formatter< Acl::SystemControllerConnection::ComponentType >
• 1.5.1.19: fmt::formatter< Acl::UUID >
• 1.5.2: Files
• 1.5.2.1: include/AclLog.h
• 1.5.2.2: include/ControlDataAddress.h
• 1.5.2.3: include/ControlDataCommon.h
• 1.5.2.4: include/ControlDataSender.h
• 1.5.2.5: include/ISystemControllerInterface.h
• 1.5.2.6: include/SystemControllerConnection.h
• 1.5.2.7: include/UUID.h
• 1.5.3: Namespaces
• 1.5.3.1: Acl
• 1.5.3.2: Acl::AclLog
• 1.5.3.3: Acl::ControlDataCommon
• 1.5.3.4: fmt
• 1.5.3.5: spdlog
• 2: Ateliere Live 8.0.0 developer reference
• 2.1: System controller config
• 2.2: REST API v3
• 2.3: Rendering Engine config
• 2.4: Operational Control API
• 2.4.1: Overview of JSON protocol
• 2.4.2: Rendering Engine components
• 2.4.3: Control API reference
• 2.5: C++ SDK
• 2.5.1: Classes
• 2.5.1.1: Acl::AclLog::FileLocationFormatterFlag
• 2.5.1.2: Acl::AclLog::ThreadNameFormatterFlag
• 2.5.1.3: Acl::AlignedAudioFrame
• 2.5.1.4: Acl::AlignedFrame
• 2.5.1.5: Acl::ControlDataAddress
• 2.5.1.6: Acl::ControlDataCommon::ConnectionEvent
• 2.5.1.7: Acl::ControlDataCommon::Response
• 2.5.1.8: Acl::ControlDataCommon::StatusMessage
• 2.5.1.9: Acl::ControlDataSender
• 2.5.1.10: Acl::ControlDataSender::Settings
• 2.5.1.11: Acl::DeviceMemory
• 2.5.1.12: Acl::IControlDataReceiver
• 2.5.1.13: Acl::IControlDataReceiver::IRequest
• 2.5.1.14: Acl::IControlDataReceiver::RequestData
• 2.5.1.15: Acl::IControlDataReceiver::Settings
• 2.5.1.16: Acl::IMediaStreamer
• 2.5.1.17: Acl::IMediaStreamer::Configuration
• 2.5.1.18: Acl::IMediaStreamer::Settings
• 2.5.1.19: Acl::IngestApplication
• 2.5.1.20: Acl::IngestApplication::Settings
• 2.5.1.21: Acl::ISystemControllerInterface
• 2.5.1.22: Acl::ISystemControllerInterface::Callbacks
• 2.5.1.23: Acl::ISystemControllerInterface::Response
• 2.5.1.24: Acl::MediaReceiver
• 2.5.1.25: Acl::MediaReceiver::CustomSystemControllerCallResponse
• 2.5.1.26: Acl::MediaReceiver::NewStreamParameters
• 2.5.1.27: Acl::MediaReceiver::Settings
• 2.5.1.28: Acl::SystemControllerConnection
• 2.5.1.29: Acl::SystemControllerConnection::Settings
• 2.5.1.30: Acl::TimeCommon::TAIStatus
• 2.5.1.31: Acl::TimeCommon::TimeStructure
• 2.5.1.32: Acl::UUID
• 2.5.1.33: fmt::formatter< Acl::AudioChannelLayout >
• 2.5.1.34: fmt::formatter< Acl::ControlDataAddress >
• 2.5.1.35: fmt::formatter< Acl::ControlDataCommon::EventType >
• 2.5.1.36: fmt::formatter< Acl::FieldOrder >
• 2.5.1.37: fmt::formatter< Acl::ISystemControllerInterface::StatusCode >
• 2.5.1.38: fmt::formatter< Acl::SystemControllerConnection::ComponentType >
• 2.5.1.39: fmt::formatter< Acl::UUID >
• 2.5.2: Files
• 2.5.2.1: include/AclLog.h
• 2.5.2.2: include/AlignedFrame.h
• 2.5.2.3: include/Base64.h
• 2.5.2.4: include/ControlDataAddress.h
• 2.5.2.5: include/ControlDataCommon.h
• 2.5.2.6: include/ControlDataSender.h
• 2.5.2.7: include/DeviceMemory.h
• 2.5.2.8: include/IControlDataReceiver.h
• 2.5.2.9: include/IMediaStreamer.h
• 2.5.2.10: include/IngestApplication.h
• 2.5.2.11: include/IngestUtils.h
• 2.5.2.12: include/ISystemControllerInterface.h
• 2.5.2.13: include/MediaEnumerations.h
• 2.5.2.14: include/MediaReceiver.h
• 2.5.2.15: include/SystemControllerConnection.h
• 2.5.2.16: include/TimeCommon.h
• 2.5.2.17: include/UUID.h
• 2.5.3: Namespaces
• 2.5.3.1: Acl
• 2.5.3.2: Acl::AclLog
• 2.5.3.3: Acl::ControlDataCommon
• 2.5.3.4: Acl::IngestUtils
• 2.5.3.5: Acl::TimeCommon
• 2.5.3.6: fmt
• 2.5.3.7: spdlog
• 3: Ateliere Live 7.0.0 developer reference
• 3.1: REST API v3
• 3.2: Ateliere Live Rendering Engine configuration documentation
• 3.3: Ateliere Live Rendering Engine command documentation
• 3.4: C++ SDK
• 3.4.1: Classes
• 3.4.1.1: AclLog::CommandLogFormatter
• 3.4.1.2: AclLog::FileLocationFormatterFlag
• 3.4.1.3: AclLog::ThreadNameFormatterFlag
• 3.4.1.4: AlignedAudioFrame
• 3.4.1.5: AlignedFrame
• 3.4.1.6: ControlDataCommon::ConnectionStatus
• 3.4.1.7: ControlDataCommon::Response
• 3.4.1.8: ControlDataCommon::StatusMessage
• 3.4.1.9: ControlDataSender
• 3.4.1.10: ControlDataSender::Settings
• 3.4.1.11: DeviceMemory
• 3.4.1.12: IControlDataReceiver
• 3.4.1.13: IControlDataReceiver::IncomingRequest
• 3.4.1.14: IControlDataReceiver::ReceiverResponse
• 3.4.1.15: IControlDataReceiver::Settings
• 3.4.1.16: IMediaStreamer
• 3.4.1.17: IMediaStreamer::Configuration
• 3.4.1.18: IMediaStreamer::Settings
• 3.4.1.19: IngestApplication
• 3.4.1.20: IngestApplication::Settings
• 3.4.1.21: ISystemControllerInterface
• 3.4.1.22: ISystemControllerInterface::Callbacks
• 3.4.1.23: ISystemControllerInterface::Response
• 3.4.1.24: MediaReceiver
• 3.4.1.25: MediaReceiver::NewStreamParameters
• 3.4.1.26: MediaReceiver::Settings
• 3.4.1.27: SystemControllerConnection
• 3.4.1.28: SystemControllerConnection::Settings
• 3.4.1.29: TimeCommon::TAIStatus
• 3.4.1.30: TimeCommon::TimeStructure
• 3.4.2: Files
• 3.4.2.1: include/AclLog.h
• 3.4.2.2: include/AlignedFrame.h
• 3.4.2.3: include/Base64.h
• 3.4.2.4: include/ControlDataCommon.h
• 3.4.2.5: include/ControlDataSender.h
• 3.4.2.6: include/DeviceMemory.h
• 3.4.2.7: include/IControlDataReceiver.h
• 3.4.2.8: include/IMediaStreamer.h
• 3.4.2.9: include/IngestApplication.h
• 3.4.2.10: include/IngestUtils.h
• 3.4.2.11: include/ISystemControllerInterface.h
• 3.4.2.12: include/MediaReceiver.h
• 3.4.2.13: include/PixelFormat.h
• 3.4.2.14: include/SystemControllerConnection.h
• 3.4.2.15: include/TimeCommon.h
• 3.4.2.16: include/UUIDUtils.h
• 3.4.3: Namespaces
• 3.4.3.1: ACL
• 3.4.3.2: AclLog
• 3.4.3.3: ControlDataCommon
• 3.4.3.4: IngestUtils
• 3.4.3.5: spdlog
• 3.4.3.6: TimeCommon
• 3.4.3.7: UUIDUtils
• 3.5: System controller config

1 - Ateliere Live 9.0.0 developer reference

1.1 - System controller config

This page describes the configuration settings possible to set via the acl_sc_settings.json file.

1.2 - REST API v3

The following is a rendering of the OpenAPI for the Ateliere Live System Controller using Swagger. It does not have a backend server, so it cannot be run interactively.

This API is available from the server at the base URL /api/v3.

1.3 - Rendering Engine config

This page describes how to configure the Rendering Engine. This topic is closely related to this page on the control command protocol for the video and audio mixers.

Rendering Engine components

The Rendering Engine is an application that uses the Production Pipeline library in the base platform for transport, and adds to that media file playback, HTML rendering, a full video mixer and an audio mixer. The figure below shows a schematic of the different components and an example of how streams may transition through it.

HTML Renderers

Multiple HTML renderers can be instantiated in runtime by control panels, and in each an HTML page can be opened. Each HTML renderer produces a video stream, but no audio.

Media Players

Multiple media players can be instantiated in runtime by control panels, and in each a media file can be opened. Each media player produces a stream containing a video stream and all audio streams from the file.

Video Mixer

The Video mixer receives all video inputs into the system, i.e. streams from ingests, from HTML renderers and from media players. It outputs one or more named video output streams. The internal structure of the video mixer is defined at startup (as described in detail below), and controlling the video mixer is done in runtime by control panels.

Audio Mixer

The audio mixer takes a number of mono or stereo pair streams as inputs that we call input strips. It outputs a number of named output streams in stereo. The outputs of the audio mixer are defined at startup, and the audio mixer is controlled in runtime by control panels.

Combine outputs

The combination of video and audio outputs into full output streams from the Rendering Engine is configured at startup.

Configuring the Rendering Engine

Some aspects of the Rendering Engine can be configured. An initial configuration is provided at startup and can later be changed using the REST API when a running Rendering Engine is about to be used in another production with different configuration requirements. The initial configuration uses a configuration file in JSON format referenced by the ACL_RENDERING_ENGINE_CONFIG environment variable. Exactly how the configuration file is specified at startup is covered in this guide. The REST API takes a JSON payload with the exact same format. Specifically, the video mixer node graph, the audio mixer outputs and the combination of video and audio outputs can be configured.

As an example, here is the contents of such a JSON configuration file that we will refer to throughout this guide:

{
  "version": "1.0",
  "video": {
    "nodes": {
      "transition": {
        "type": "transition"
      },
      "chroma_key_select": {
        "type": "select"
      },
      "chroma_key": {
        "type": "chroma_key"
      },
      "chroma_key_alpha_over": {
        "type": "alpha_over"
      },
      "fade_to_black": {
        "type": "fade_to_black"
      },
      "program": {
        "type": "output"
      },
      "chroma_key_preview": {
        "type": "output"
      },
      "preview": {
        "type": "output"
      }
    },
    "links": [
      {
        "from_node": "transition",
        "from_socket": 0,
        "to_node": "chroma_key_alpha_over",
        "to_socket": 1
      },
      {
        "from_node": "transition",
        "from_socket": 1,
        "to_node": "preview",
        "to_socket": 0
      },
      {
        "from_node": "chroma_key_select",
        "from_socket": 0,
        "to_node": "chroma_key",
        "to_socket": 0
      },
      {
        "from_node": "chroma_key",
        "from_socket": 0,
        "to_node": "chroma_key_alpha_over",
        "to_socket": 0
      },
      {
        "from_node": "chroma_key_alpha_over",
        "from_socket": 0,
        "to_node": "fade_to_black",
        "to_socket": 0
      },
      {
        "from_node": "fade_to_black",
        "from_socket": 0,
        "to_node": "program",
        "to_socket": 0
      },
      {
        "from_node": "chroma_key",
        "from_socket": 0,
        "to_node": "chroma_key_preview",
        "to_socket": 0
      }
    ]
  },
  "audio": {
    "outputs": [
      {
        "name": "main",
        "channels": 2
      },
      {
        "name": "aux1",
        "channels": 2
      },
      {
        "name": "aux2",
        "channels": 2
      }
    ]
  },
  "output_mapping": [
    {
      "name": "program",
      "video_output": "program",
      "audio_output": "main",
      "feedback_input_slot": 1001,
      "stream": true
    },
    {
      "name": "preview",
      "video_output": "preview",
      "audio_output": "",
      "feedback_input_slot": 1002,
      "stream": false
    },
    {
      "name": "aux1",
      "video_output": "program",
      "audio_output": "aux1",
      "feedback_input_slot": 0,
      "stream": true
    },
    {
      "name": "chroma_key_preview",
      "video_output": "chroma_key_preview",
      "audio_output": "aux2",
      "feedback_input_slot": 100,
      "stream": true
    }
  ]
}

Video Mixer node graph

The video mixer is defined as a tree graph of nodes that work together in sequence to produce one or several video outputs. Each node performs a specific action, and has zero or more input sockets and zero or more output sockets. Links connect output sockets on one node to input sockets on other nodes. Each node is named and the name is used to control the node in runtime. The node tree configuration is specified in the video section of the JSON file, which contains two parts: the list of nodes and the list of links connecting the nodes.

The following is a graphical representation of the video mixer node graph configuration in the JSON example file above, with two input nodes, three processing nodes and three output nodes, and links in between:

Video nodes block

The nodes object of the video block is a JSON object where the names/keys in the object is the unique name of the node. The name is used to refer to the node when operating the mixer later on, so names that are easy to understand what they are supposed to be used for in the production is recommended. The name can only contain lower case letters, digits, _ and -. Space or other special characters are not allowed.

Each node is an JSON object with parameters defining the node properties. Here the parameter type defines which type of node it is. The supported types are listed below. They can be divided into three groups depending on if they provide input to the graph, output from the graph or is a processing node, placed in the middle of the graph.

Input nodes

The input nodes take their input from the input slots of the Rendering Engine, which contains the latest frame for all connected sources (this includes connected cameras, HTML graphics, media players etc.) The nodes in this group does not have any input sockets, as they take the input frames from the input slots of the Rendering Engine. Which slot to take the frames from is dynamically controlled during runtime.

Alpha combine node (alpha_combine)
Input sockets: 0 (Sources are taken from the input slots of the mixer)
Output sockets: 1
The alpha combine node takes two inputs and combines the color from one of them with the alpha from the other one. The node features multiple modes that can be set during runtime to either pick the alpha channel of the second video input, or to take any of the other R, G and B channels, or to average the RGB channels and use as alpha for the output. This node is useful in case videos with alpha is provided from SDI sources, where the alpha must be sent as a separate video feed and then combined with the fill/color source in the mixer.

Select node (select)
Input sockets: 0 (Source is taken from the input slots of the mixer)
Output sockets: 1
The select node simply selects an input slot from the Rendering Engine and forwards that video stream. Which input slot to forward is set during runtime. The node is a variant of the transition node, but does not support the transitions supported by the transition node.

Transition node (transition)
Input sockets: 0 (Sources are taken from the input slots of the mixer)
Output sockets: 2 (One with the program output and one with the preview output)
The transition node takes two video streams from the Rendering Engine’s input slots, one to use as the program and one as the preview output stream. These are output through the two output sockets. During runtime this node can be used to make transitions such as wipes and fades between the selected program and preview.

Output nodes

This group only contains a single node, used to mark a point where processed video can exit the graph and be used in output mappings.

Output node (output)
Input sockets: 1
Output sockets: 0 (Output is sent out of the video mixer)
The output node marks an output point in the video mixer’s graph, where the video stream can be used by the output mapping to be included in the Rendering Engine’s Output streams, or as streams to view in the multi-viewer. The node takes a single input and makes it possible to use that video feed outside the video mixer. Output nodes can be used both to output the program and preview feeds of a video mixer, but also to mark auxiliary outputs, as in the example above, where chroma_key_preview is output to be included in the graph to be able to view the result of the chroma keying, without the effect being keyed on to the program output.

Processing nodes

The processing nodes take their input from another node’s output and outputs a result that is sent to another node’s input. They are therefore placed in the middle of the graph, after nodes from the input group and before output nodes.

Alpha over node (alpha_over)
Input sockets: 2 (Index 0 for overlay video and index 1 for background video)
Output sockets: 1
The alpha over node composites the overlay video input on top of the background video input. The alpha of the overlay video input is taken into consideration. During runtime, this node can be controlled to show or not to show the overlay, and to fade the overlay in or out. This node is useful to composite things such as graphics or chroma keyed video onto a background video.

Chroma key node (chroma_key)
Input sockets: 1
Output sockets: 1
The chroma key node takes an input video stream and performs chroma keying on it based on parameters set during runtime. The video output will have the alpha channel (and in some cases also the color channels) altered. The result of this node can then be composited on top of a background using an Alpha over node.

Crop node (crop)
Input sockets: 1
Output sockets: 1
The crop node takes an input video stream and crops the video based on parameters set during runtime. The parts of the video outside of the cropped area will be fully transparent.

Fade to black node (fade_to_black)
Input sockets: 1
Output sockets: 1
The fade to black node takes a single input video stream and can fade that video stream to and from black. This node is normally used as the last node before the main program output node, to be able to fade to and from black at the beginning and end of the broadcast.

Transform node (transform)
Input sockets: 1
Output sockets: 1
The transform node takes an input video stream and transform the result inside the visible canvas. This node can be configured during runtime to scale and move the input video. This node is useful for picture-in-picture effects, or to move a chroma key node output to the lower corner of the frame.

Video delay node (video_delay)
Input sockets: 1
Output sockets: 1
The video delay node is used to delay the video by a given number of frames. The number of frames to delay is controlled during runtime. This node is useful whenever the need for dynamically delay a video stream arises, for example in case an external audio mixer is used, which comes with a delay of some frames.

Video links block

The links array in the video block is a list of links between video nodes. The video frames are fed in one direction from node to node via these links. An input socket on a node can only have one connected link. The output sockets on a node can have multiple connected links.

Each block contains the following keys:

• from_node - The name of the node in the nodes object, from which the link is receiving frames from
• from_socket - The index of the output socket in the node from which this link originates
• to_node - The name of the node in the nodes object, to which the link is sending frames to
• to_socket - The index of the input socket in the node to which this link connects

Some examples

The simplest video mixer node graph imaginable would be a select node feeding an output node. This mixer would only be able to select one of the inputs and output it unaltered, like a video router would do:



The JSON file section for this is:

"video": {
  "nodes": {
    "input_select": {
      "type": "select"
    },
    "program": {
      "type": "output"
    }
  },
  "links": [
    {
      "from_node": "input_select",
      "from_socket": 0,
      "to_node": "program",
      "to_socket": 0
    }
  ]
}

A slightly more advanced node graph would be to use a transition node and two outputs, one for the program out and one for the preview:



The JSON file section for this is:

"video": {
  "nodes": {
    "transition": {
      "type": "transition"
    },
    "program": {
      "type": "output"
    },
    "preview": {
      "type": "output"
    }
  },
  "links": [
      {
        "from_node": "transition",
        "from_socket": 0,
        "to_node": "program",
        "to_socket": 0
      },
      {
        "from_node": "transition",
        "from_socket": 1,
        "to_node": "preview",
        "to_socket": 0
      },
  ]
}

Audio outputs

The audio block of the configuration file defines the properties of the audio mixer. The JSON object contains a list called outputs which lists the outputs of the audio mixer and the configuration of each output. Each output has these parameters:

• name - The unique name of this audio output, used to refer to if from the output_mapping block
• channels - The number of audio channels of this output

The input routing and configuration of the strips is made via the control command API.

Combine outputs

The output_mapping block is used to define outputs of the entire Rendering Engine, by combining outputs from the video and audio mixers and configuring where those should be sent. The output mapping is an array of output mappings. Each mapping has the following parameters:

• name - The unique name of the output mapping. This will be displayed in the REST API both for sources being streamed and sources with feedback streams that can be included in the multi-view
• video_output - The name of the video mixer node of type output to get the video stream from. Leave empty, or omit the key to create an audio-only output
• audio_output - The name of the audio output to get the audio stream from. Leave empty, or omit the key to create a video-only output
• feedback_input_slot The input slot to use to feed this output back to the multi-view. The REST API may then refer to this input slot to include this output in a multi-view view and the video mixer might refer to this input slot to use the output as an input to another part of the video mixer graph (as long as this does not create an reference loop in the video mixer graph, in which case the input node making the reference that creates the loop will return an black frame). Note that these are the same input slots that ingested sources, media players and HTML browsers use. Use 0, or omit the key to disable feedback of this stream.
• stream - Boolean value to tell if the output should be streamable and visible as an output in the REST API. If set to false the output will not turn up as an Pipeline Output in the REST API.

The following is a graphical visualisation of the output mappings in the JSON example configuration file above:

The default Rendering Engine config

If no Rendering Engine configuration file has been supplied by the user, a default configuration will be used. The following is the video node graph in the default configuration:



The full JSON for this default configuration is:

{
  "version": "1.0",
  "video" : {
    "nodes":{
      "transition": {"type":"transition"},
      "alpha_combine": {"type":"alpha_combine"},
      "chroma_key_select": {"type":"select"},
      "chroma_key": {"type":"chroma_key"},
      "chroma_key_transform": {"type":"transform"},
      "pip1_select": {"type":"select"},
      "pip1_transform": {"type":"transform"},
      "pip2_select": {"type":"select"},
      "pip2_transform": {"type":"transform"},
      "html_select": {"type":"select"},
      "alpha_over": {"type":"alpha_over"},
      "chroma_key_alpha_over": {"type":"alpha_over"},
      "pip1_alpha_over": {"type":"alpha_over"},
      "pip2_alpha_over": {"type":"alpha_over"},
      "html_alpha_over": {"type":"alpha_over"},
      "fade_to_black": {"type":"fade_to_black"},
      "program": {"type":"output"},
      "preview": {"type":"output"}
    },
    "links":[
      {"from_node":"transition", "from_socket":0, "to_node":"alpha_over", "to_socket":1},
      {"from_node":"alpha_combine", "from_socket":0, "to_node":"alpha_over", "to_socket":0},
      {"from_node":"alpha_over", "from_socket":0, "to_node":"chroma_key_alpha_over", "to_socket":1},
      {"from_node":"chroma_key_select", "from_socket":0, "to_node":"chroma_key", "to_socket":0},
      {"from_node":"chroma_key", "from_socket":0, "to_node":"chroma_key_transform", "to_socket":0},
      {"from_node":"chroma_key_transform", "from_socket":0, "to_node":"chroma_key_alpha_over", "to_socket":0},
      {"from_node":"chroma_key_alpha_over", "from_socket":0, "to_node":"pip1_alpha_over", "to_socket":1},
      {"from_node":"pip1_select", "from_socket":0, "to_node":"pip1_transform", "to_socket":0},
      {"from_node":"pip1_transform", "from_socket":0, "to_node":"pip1_alpha_over", "to_socket":0},
      {"from_node":"pip1_alpha_over", "from_socket":0, "to_node":"pip2_alpha_over", "to_socket":1},
      {"from_node":"pip2_select", "from_socket":0, "to_node":"pip2_transform", "to_socket":0},
      {"from_node":"pip2_transform", "from_socket":0, "to_node":"pip2_alpha_over", "to_socket":0},
      {"from_node":"pip2_alpha_over", "from_socket":0, "to_node":"html_alpha_over", "to_socket":1},
      {"from_node":"html_select", "from_socket":0, "to_node":"html_alpha_over", "to_socket":0},
      {"from_node":"html_alpha_over", "from_socket":0, "to_node":"fade_to_black", "to_socket":0},
      {"from_node":"fade_to_black", "from_socket":0, "to_node":"program", "to_socket":0},
      {"from_node":"transition", "from_socket":1, "to_node":"preview", "to_socket":0}
    ]
  },
  "audio" : {
    "outputs": [
      {"name": "main", "channels": 2}
    ]
  },
  "output_mapping" : [
    {"name":"program", "video_output":"program", "audio_output":"main", "feedback_input_slot":1001, "stream":true},
    {"name":"preview", "video_output":"preview", "feedback_input_slot":1002, "stream":false}
  ]
}

1.4 - Operational Control API

1.4.1 - Overview of JSON protocol

This is a description of the network protocol used to control, monitor and explore the components in a running Ateliere Live production.

The JSON protocol is used when connected to the Websocket Control Panel (acl-websocketcontrolpanel) and has multi-client support, making it possible for clients that are connected to the same production to mirror each other’s controls.

The JSON protocol sits on top of an application-independent platform protocol which transports the JSON payload between the clients and the production.

Control Panel implementation

Note: This section is mainly needed if you plan on implementing a new control panel for the Atelier Live Rendering Engine using the C++ SDK.

Most of the JSON messages described in this document are sent and received by the Websocket Control Panel using the ControlDataSender interface. The interface encapsulates the packing and unpacking of messages to and from the application-agnostic platform protocol. The platform protocol includes the addresses for the sender and receiver(s) of the messages but is unaware of the contents of the packages.

The ControlDataSender interface defines the callbacks that should be used for handling messages coming from the production. Response and status messages are received from the production by setting the mResponseCallback and mStatusMessageCallback, respectively. The data received in these callbacks are complete JSON messages conforming to the specification in this document and can be forwarded without changes to the connected clients. There are however some exceptions to this.

Events

Messages of the type event currently signals that a connection was opened or closed in the production. Event messages originate in the control panel meaning that implementing support for this message type is part of making a control panel using the C++ SDK.

The control panel gets the events from the underlying platform through the ControlDataSender::Settings::mConnectionEventCallback. By inspecting the ConnectionEvent provided in the callback, clients connected to the control panel can be informed about the event. This can be used by other control panels, implemented using the C++ SDK.

Please see the section about the event message type under “Message types” for information about the message syntax.

Hop count

Messages of different types need to propagate to different depth of the production. A hop count in each message determines if it will be relayed from one Rendering Engine to a possibly chained one behind it.

When receiving a JSON message from a client, the Websocket Control Panel will select a hop count based on the content of the message, before sending the message to the production.

The hop count is marked in the platform layer of the protocol when sending a message using ControlDataSender::sendRequestToReceivers. It is not visible in the JSON message.

For each type of message, the hop count used by the Websocket Control Panel is specified in a “Hop count” section within “Message types” below. This hop count should be used when implementing a new C++ control panel.

The state tree

The control API is based around a state tree that makes up the hierarchy of parameters and entities in the Rendering Engine that can be controlled or monitored. The tree is expressed as a JSON structure.

An entity in the tree is addressed by a path consisting of the names of each enclosing object surrounding the entity.

For instance, in the structure below, the path to the g-field of the rgb_filter would be /video/nodes/rgb_filter/g:

{
    "video": {
        "nodes": {
            "rgb_filter": {
                "r": 0.5,
                "g": 0.4,
                "b": -0.2
            }
        }
    }
}

Message format

JSON syntax is used for all messages. The type and resource fields are present in all messages and define the type of message and what resource that is affected by this message.

{
    "type": "...",
    "resource": "..."
}

Message types

All changes that can be made to a production are available in the JSON API through messages of type set or command.

A parameter that can be changed immediately, without side effects, and that will keep its state until the next change can be accessed directly with a set message. In other cases a command is necessary to initiate the change. A set message might for instance be used for controlling the layer opacity in an alpha-over node or the gain for an input to the audio mixer whereas a command is needed to initiate the loading of a media file or a transition effect spanning over a period of time.

To get the current state of a resource, a get message can be used. A resource can also be monitored for changes using a subscribe message. When a sub-resource is added or removed from the monitored resource, a state-add or state-remove message is sent to the subscriber. Other changes result in a state-change message.

Parameters that change continuously without any user input, such as the play position of a media player or the loudness in an audio output, are referred to as streaming parameters. Streaming parameters are monitored by sending a sampling-start message to the resource path of the streaming parameter and providing a time interval stating how often sampling-update messages should be sent.

Table of message types

All response messages contain the field timestamp. This is the time when the request message was handled, i.e. the timestamp of the frame processed on that time.

Each type of message is described in detail below.

get and get-response

A message of type get is used to retrieve the current state of a given resource within the production.

Required fields for get messages:

For instance:

{
    "type": "get",
    "resource": "/video/nodes"
}

The response is a get-response. A successful response contains a body object with the current state of the resource:

{
    "type": "get-response",
    "resource": "/video/nodes",
    "timestamp": 1731073800720000,
    "body": {
        ...
    }
}

The body has its root where the resource URI points.

An unsuccessful get has no body and contains an error message, for instance:

{
    "type": "get-response",
    "resource": "/video/nodes",
    "error": "No such resource: /video/nodes",
    "timestamp": 1731073800720000
}

NOTE: Repeatedly calling get is a very inefficient way of monitoring the state of a production. Please see the subscribe and state-change message sections for a better way.

Hop count

In a proxy-editing setup, i.e. when there is one pipeline behind another one, get messages will stop at the first one and the state for that pipeline will be returned. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to 1 (one) for get messages.

set and set-response

A message of type set is used to change the state of one or more components located at a specified resource path in the production. This can for instance be to change the currently selected video source of a transition node or to change the gain of an audio strip.

set messages are generally only available for changes which are immediate. Please see the command message section for details about changes that occur over a period of time.

Required parameters for set messages:

The body of a set request contains the new values of the fields to change within the resource path. The structure of the body does not need to be complete - only the fields that should be updated need to be included.

Example of a set request that changes program and preview within a node named my_transition_node:

{
    "type": "set",
    "resource": "/video/nodes/my_transition_node",
    "body": {
        "preview": 3,
        "program": 8
    }
}

The body does not need to be an object when setting a single item, just the value. Here is an example:

{
    "type": "set",
    "resource": "/audio/strips/1/filters/gain/value",
    "body": 8.6
}

The set-response does not contain a body with the set values - instead changes in the production need to be monitored by subscribing to the affected resources. Please see the subscribe message section for details about subscriptions.

If there is an error, a set-response is returned containing an error message. In this case none of the parameters in the body of the set message will be applied. If the set was successful the response will contain a result ok key/value.

Hop count

In a proxy-editing setup, i.e. when there is one pipeline behind another one, set messages will propagate all the way to the last pipeline while following the configured alignment delay of each step. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to -1 (negative one) for set messages.

command and command-response

Commands are used to initiate events in the production which are not immediate or that do not have a direct relationship to one specific field of the production state tree. For instance, when starting playback of a media file there might be a play command. This will start the actual playback but also change the player state (e.g. from paused to playing) and also start to periodically update the current playback position field.

Required parameters for command messages:

Example of a command that starts a fade command on a transition node:

{
    "type": "command",
    "resource": "/video/nodes/my_transition_node",
    "body": {
        "command": "fade",
        "parameters": {
          "duration_ms": 120
        }
    }
}

Example of a command that takes no parameters:

{
    "type": "command",
    "resource": "/video/nodes/my_transition_node",
    "body": {
        "command": "cut"
    }
}

As for set-response messages, the command-response has no body - instead changes in the production need to be monitored by subscribing to one or more resources. Please see the subscribe message section for details about subscriptions.

If there is an error, a command-response is returned containing an error message. If the command was successful the response will contain a result ok key/value.

Hop count

In a proxy-editing setup, i.e. when there is one pipeline behind another one, command messages will propagate all the way to the last pipeline while following the configured alignment delay of each step. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to -1 (negative one) for command messages.

subscribe and subscribe-response

Subscriptions are used to monitor changes in the production. This can e.g. be useful in order to mirror the controls of different control surfaces that are connected to the same production, detect when new resources appear, or to otherwise visualize different parts of the system.

Required parameters for subscribe messages:

Example of a subscribe message for tracking changes of the resource /audio/strips/3/compressor:

{
    "type": "subscribe",
    "resource": "/audio/strips/3/compressor"
}

The response is a subscribe-response. A successful response contains a body object with the current state of the resource, for instance:

{
    "type": "subscribe-response",
    "resource": "/audio/strips/3/compressor",
    "body": {
        "attack": 30,
        "gain": 1,
        "knee": 3.5,
        "ratio": 3.3,
        "release": 2000,
        "threshold": -24,
        "type": "compressor"
    },
    "timestamp": 1731073800720000
}

If there is an error, a subscribe-response is returned containing an error message instead of a body.

Whenever a monitored resource changes, a state-change, state-add or state-remove message is sent to all subscribers of that resource. Please see the state-change, state-add and state-remove message sections for details about this.

Websocket Control Panel

In a proxy-editing setup, i.e. when there is one pipeline behind another one, subscribe messages will stop at the first one and it is from this pipeline that state changes will be reported. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to 1 for subscribe messages.

unsubscribe and unsubscribe-response

To stop receiving updates when a monitored resource changes, an unsubscribe message can be used.

Required parameters for unsubscribe messages:

If the unsubscribe message is successful, an unsubscribe-response is returned:

{
    "type": "unsubscribe-response",
    "resource": "/audio/strips/3/compressor",
    "timestamp": 1731073800720000
}

If there is an error, an unsubscribe-response is returned containing an error message instead of a body.

Hop count

In a proxy-editing setup, i.e. when there is one pipeline behind another one, unsubscribe messages will stop at the first receiver of the message. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to 1 for unsubscribe messages.

state-change

A state-change message describes the new state for a number of fields within a specified resource. state-change messages are sent from the production side, i.e. they are only received, never sent, by clients.

state-change messages are sent whenever the internal state of the production changes, for instance when:

• a client has issued a set message
• a command that changes the state is being executed
• an automation changes the state over a period of time
• a component, such as a metering device, has new data to report

To avoid loops or unnecessary updates on the client side, the message includes an actor field stating whether the receiver of the message or some other entity caused the change to happen.

Format of the state-change message:

Below is an example state-change message where the client receiving the message also caused the change to happen:

{
    "type": "state-change",
    "resource": "/audio/strips/3/compressor",
    "actor": "self",
    "body": {
        "gain": 2.5,
        "ratio": 4.0
    }
}

state-add and state-remove

The state-add and state-remove messages are similar to the state-change message. The top level resource field is the resource the client subscribes to. In the body of the message there is another resource field telling what sub-resource has been added or removed.

Examples:

{
  "type": "state-add",
  "resource": "/audio",
  "actor": "system",
  "body": {
    "resource": "/strips/2"
  }
}

{
  "type": "state-remove",
  "resource": "/audio",
  "actor": "system",
  "body": {
    "resource": "/strips/2"
  }
}

subscription-list and subscription-list-response

The subscription-list message is used to list own subscriptions. The response message shows the client’s subscriptions, located under a given resource point in the tree.

Required fields for subscription-list messages:

For instance:

{
    "type": "subscription-list",
    "resource": "/video"
}

The response is a subscription-list-response. A successful response contains a body object with the current state of the resource:

{
  "type": "subscription-list-response",
  "resource": "/video",
  "timestamp": 1731414140040000,
  "body": {
    "subscriptions": [
      "/nodes/alpha_combine",
      "/nodes/alpha_over"
    ]
  }
}

The body has its root where the resource URI points.

An unsuccessful request has no body and contains an error message, for instance:

{
  "type": "subscription-list-response",
  "resource": "/video/nodes/over",
  "timestamp": 1731414382900000,
  "error": "Failed to find resource with name 'over'"
}

Hop count

In a proxy-editing setup, i.e. when there is one pipeline behind another one, subscription-list messages will stop at the first one, and the subscriptions for that pipeline will be returned. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to 1 for subscription-list messages.

describe and describe-response

The purpose of these messages is to get a description of some resource in the state tree, like a node in the video mixer component for example. The descriptions will list all commands and parameters which are valid for the resource.

Required fields for describe messages:

This is the description for /video/nodes/fade_to_black:

{
  "type": "describe-response",
  "resource": "/video/nodes/fade_to_black",
  "timestamp": 1731480819600000,
  "body": {
    "children": [],
    "commands": [
      {
        "command": "fade_from",
        "optional_parameters": [],
        "required_parameters": [...],
        ...
      },
      ...
    ],
    "parameters": [
    ...
    ]
  }
}

Hop count

In a proxy-editing setup, i.e. when there is one pipeline behind another one, describe messages will stop at the first receiver of the message. This is applicable when using a Websocket Control Panel. When implementing a new control panel using the C++ SDK, set the hop count to 1 for describe messages.

sampling-start and sampling-start-response

The sampling-start message is used to start sampling a streaming parameter. If successful, sampling-update messages will be sent to the subscriber periodically at the specified time inteval.

Required parameters for sampling-start messages:

An asterisk (*) can be used at any level of the resource expression to indicate “all components at this level”. If an asterisk is used, no other characters are allowed at that level of the path. For instance, "resource": "/audio/mixes/*/input_meter/*" can be used to start sampling all streaming parameters under input_meter for all audio mixes.

The interval_ms parameter must be greater than or equal to 40.

Example:

{
  "type": "sampling-start",
  "resource": "/audio/mixes/*/input_meter/*",
  "body": {
    "interval_ms": 100
  }
}

If there is an error, a sampling-start-response is returned containing an error message. If the sampling-start was successful the response will contain a result ok key/value.

sampling-stop and sampling-stop-response

The sampling-stop message is used to stop sampling a streaming parameter.

Required parameters for sampling-stop messages:

The resource parameter must match one used to start sampling exactly.

Example:

{
  "type": "sampling-stop",
  "resource": "/audio/mixes/*/input_meter/*"
}

If there is an error, a sampling-stop-response is returned containing an error message. If the sampling-stop was successful the response will contain a result ok key/value.

sampling-list and sampling-list-response

The sampling-list message is used to list the streaming parameter subscriptions that are registered to the client sending the message.

Required parameters for sampling-stop messages:

Example:

{
  "type": "sampling-list",
  "resource": "/"
}

If there is an error, a sampling-list-response is returned containing an error message. If the sampling-stop was successful the response will contain a list of all the sender’s streaming parameter subscriptions.

Example:

{
    "type": "sampling-list-response",
    "resource": "/",
    "body": {
        "samplings": [
            "/audio/strips/1/pre_filter_meter/*",
            "/audio/mixes/0/input_meter/*"
        ]
    },
    "timestamp": 1736860857880000
}

sampling-update

A sampling-update message describes the latest sampled state of one or more streaming parameters matching a subscription. The message is sent periodically to clients that have started subscriptions using sampling-start.

The message contains the resource expression used in sampling-start. If necessary this can be used to identify the subscription in the client.

The body of the message contains the updated state for all of the streaming parameters matching the subscription. Note that the body starts at the root of the state tree, which is different from e.g. get responses or subscriptions for non-streaming parameters.

Example:

{
    "type": "sampling-update",
    "resource": "/audio/strips/*/pre_filter_meter/*",
    "body": {
        "audio": {
            "strips": {
                "1": {
                    "pre_filter_meter": {
                        "peak": -0.439775225995602234
                    }
                },
                "2": {
                    "pre_filter_meter": {
                        "peak": -0.982362873895602837
                    }
                }
            }
        }
    },
    "timestamp": 1736861180080000
}

event

When connected to a Websocket Control Panel it will act as a proxy in front of the Rendering Engine. This introduces a message of type event, which is only used between the Websocket Control Panel and its clients. Messages of type event contain an event field specifying the type of event.

All clients connected to the Websocket Control Panel will receive events of type connect whenever a new connection is established between the Websocket Control Panel and a Rendering Engine. Such a message might look like:

{
    "type": "event",
    "connected_node": "<uuid>",
    "address": "<uuid>:<uuid>:...",
    "event": "connect"
}

where connected_node is the UUID of the newly connected node (Rendering Engine) and address is a colon separated list of UUIDs, which is the address to the node that discovered the connection. For connect events this is currently only the Websocket Control Panel itself, meaning the address is always the Websocket Control Panel’s.

When a connection between a client and a Websocket Control Panel is established, the client will receive connect events for all Rendering Engines that where already connected to the Websocket Control Panel, to let the client know it is connected to a production.

If the connection between the Websocket Control Panel and the Rendering Engine breaks, whether it is disconnected on purpose or gets disconnected due to a network outage, the Websocket Control Panel will send an event message to all its clients.

{
    "type": "event",
    "disconnected_node": "<uuid>",
    "address": "<uuid>:<uuid>:...",
    "event": "disconnect"
}

A similar message is also sent to the clients in case the connection between two Rendering Engines/Pipelines is torn down or lost, such as between a Low Delay and a High Quality Pipeline. In that case the address parameter will be the UUID of the Websocket Control Panel followed by the UUID of the LD Pipeline and the disconnected_node is the UUID of the HQ Pipeline.

1.4.2 - Rendering Engine components

This page describes the parameters and commands that is used for controlling the video mixer, audio mixer, HTML renderers and media players in the Ateliere Live Rendering Engine. This topic is closely related to this page on how to configure the rendering engine at startup.

Control command protocol

All commands to the Ateliere Live Rendering Engine are sent as human readable JSON objects and are listed below.

Each subsystem of the rendering engine has their own set of parameters and commands, and can be reached using:

• /video for the video mixer
• /audio for the audio mixer (if using the built-in mixer)
• /ndi_audio for the NDI audio mixer
• /html for the html renderer instances
• /media for the media playback instances

Video mixer resources

Background

The video mixer is built as a tree graph of processing nodes, please see this page for further information on the video mixer node graph. The names of the nodes defined in the node graph are used as part of the resources: /video/nodes/{node_name} and they all have the parameter type which describes which type of video node it is.

There are 10 different node types.

• The Transition node is used to pick which input slots to use for the program and preview output. The node also supports making transitions between the program and preview.
• The Select node simply forwards one of the available sources to its output.
• The Alpha combine node is used to pick two input slots (or the same input slot) to copy the color from one input and combine it with some information from the other input to form the alpha. The color will just be copied from the color input frame, but there are several modes that can be used to produce the alpha channel in the output frame in different ways. This is known as “key & fill” in broadcasting.
• The Alpha over node is used to perform an “Alpha Over” operation, that is to put the overlay video stream on top of the background video stream, and let the background be seen through the overlay depending on the alpha of the overlay. The node also features fading the graphics in and out by multiplying the alpha channel by a constant factor.
• The Transform node takes one input stream and transforms it (scaling and translating) to one output stream.
• The Chroma Key node takes one input stream, and by setting appropriate parameters for the keying, it will remove areas with the key color from the incoming video stream both affecting the alpha and color channels
• The Fade to black node takes one input stream, which it can fade to or from black gradually, and then outputs that stream.
• The Output node has one input stream and will output that stream out from the video mixer, back to the Rendering Engine. It has no control commands
• The Crop node takes one input stream and can crop a video stream to a new size
• The Video delay node takes one input stream and will hold the frames for a configured amount of time before forwarding the frames, causing a delay of the video output

To reset the runtime configuration of all video nodes to their default state use the reset command of the /video resource.

Transition

The transition node picks a program and a preview video source from the input slots and forward these to other nodes. The node also features auto transitions between the program and the preview sources. Some transition commands last over a duration of time, for example wipes. These can be performed either automatically or manually. The automatic mode works by the operator first selecting the type of transition, for instance a fade, setting the preview to the input slot to fade to and then trigger the transition at the right time with a auto command with the duration for the transition. In manual mode the exact position of the transition is set by the control panel by setting the factor parameter. This is used for implementing T-bars, where the T-bar repeatedly sends the current position of the bar. In the manual mode, the transition type is set before the transition begins, just as in the automatic mode. Note that an automatic transition will be overridden in case the transition position/factor is manually set, by interrupting the automatic transition and jumping to the manually set position.

resource: /video/nodes/{node name} type: transition

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "factor": 0.0,
        "mode": "fade",
        "preview": 0,
        "program": 0
    }
}

Commands

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "auto",
       "parameters": {
           "duration_ms": <uint32>
       }
   }
}

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "cut"
   }
}

Select

A node to select a video source from the input slots and send it on to the next node.

resource: /video/nodes/{node name} type: select

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "input": 0
    }
}

Alpha combine

A node to combine the color channels of one video stream with the alpha from another. This node is useful for video sources where the alpha channel is provided as a separate black and white video source that must be combined with the color source. The node supports multiple modes of obtaining the alpha, either by copying a specific color or alpha channel of some input slot, or by taking the average of the R, G and B channels of the video from some input slot.

resource: /video/nodes/{node name} type: alpha_combine

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "alpha": 0,
        "color": 0,
        "mode": "average-rgb"
    }
}

Alpha over

A node to combine two video streams using alpha over compositing, overlaying the foreground stream on the background stream. The node will keep the transparency of both layers. The overlay stream can be faded in and out of the background stream.

resource: /video/nodes/{node name} type: alpha_over

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "factor": 0.0
    }
}

Commands

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "fade_from",
       "parameters": {
           "duration_ms": <uint32>
       }
   }
}

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "fade_to",
       "parameters": {
           "duration_ms": <uint32>
       }
   }
}

Transform

A node to transform an incoming video stream, by scaling and transposing it. The canvas size of the input will be kept and all surrounding area in case the source video is shrunk, is filled with transparent black.

resource: /video/nodes/{node name} type: transform

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "scale": 1.0,
        "x": 0.0,
        "y": 0.0
    }
}

Chroma key

A node to perform chroma keying on an incoming video stream. The output video stream will have the alpha and possibly the color channels modified, according to the parameter values in this node. To remove a color from the incoming video stream, first enable the node and then select the key color to remove. The key color can be selected in two ways, either by manually setting the color with the R, G and B channel values, or by using the color picker. When using the color picker, the color picker command will define the position and size of the color picker square to sample the incoming video stream. The R, G and B color parameters will be updated according to the average color of the area when the command was received by the Rendering Engine. The currently selected color can be shown in the upper left hand corner in the output video stream of the node by setting the parameter show_key_color to true. Also, the latest sampled color picker area can be drawn in the node’s output by setting show_color_picker to true. When a suitable color has been chosen, adjust the distance and falloffparameters to get a clear mask. To aid the tweaking of the parameters, set the show_alpha parameter to true. This will make the node output the black and white mask instead of the keyed result, which makes it easier to see which parts are masked away and not. Remember to turn this off before going on air. As a last step, any remaining fringes of the key color around the subject can be desaturated with the color_spill parameter. But remember this will desaturate colors close to the key color even in parts of the frame fully visible.

resource: /video/nodes/{node name} type: chroma_key

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "color_spill": 0.1,
        "distance": 0.1,
        "enabled": false,
        "falloff": 0.08,
        "show_alpha": false,
        "show_color_picker": false,
        "show_key_color": false
    }
}

Commands

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "pick_color",
       "parameters": {
           "size": <uint32>,
           "x": <float>,
           "y": <float>
       }
   }
}

Chroma key / Key color

The key color

resource: /video/nodes/{node name}/key_color type: chroma_key

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}/key_color",
    "body": {
        "b": 0.0,
        "g": 0.0,
        "r": 0.0
    }
}

Fade to black

A node to fade the incoming video stream to and from black.

resource: /video/nodes/{node name} type: fade_to_black

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "factor": 0.0
    }
}

Commands

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "fade_from",
       "parameters": {
           "duration_ms": <uint32>
       }
   }
}

{
   "type": "command",
   "resource": "/video/nodes/{node name}",
   "body": {
       "command": "fade_to",
       "parameters": {
           "duration_ms": <uint32>
       }
   }
}

Crop

A node to crop the incoming video stream. The node can crop the left, right, top and bottom edge of the incoming video stream. The areas outside of the cropped area will be transparent in the output.

resource: /video/nodes/{node name} type: crop

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "bottom": 1.0,
        "left": 0.0,
        "right": 1.0,
        "top": 0.0
    }
}

Video delay

A node to delay the video stream a given number of frames.

resource: /video/nodes/{node name} type: video_delay

Parameters

{
    "type": "set",
    "resource": "/video/nodes/{node name}",
    "body": {
        "delay": 0
    }
}

Audio mixer resources

Overview

The audio mixer has the following data path:

The three main elements of the audio mixer are the Input strips, the Mixes, and the Outputs.

Input strips

Audio enters the audio mixer through the input strips. When a strip has been created, the audio for the strip is selected among the available channels in the Rendering Engine’s input slots.

In the example picture above, three sources are connected to the input slots. The slots have four, two, and six channels respectively. In the illustration we can see that the first strip uses a single channel from the first slot, the second strip uses channel three and four from the second slot and so on. Several input strips may select the same input slot channels.

Each input strip has two stereo output connection points (), one before the volume fader (pre fader) and one after the volume fader (post fader). When a mix or an output is connected to the output of a strip, which of these two locations to take the audio from can be selected using the origin parameter.

Mixes

Mixes are used to combine the output from input strips, or other mixes, and also allows for further filtering of the audio signal. When a mix has been created, the input strips and other mixes contributing to the mix can be controlled in the /inputs/strips and /inputs/mixes sections of the mix.

Just like an input strip a mix has two stereo output connection points () that can be used when connecting the output of the mix to another mix or an output.

Outputs

Each output corresponds to an audio output in the Rendering Engine configuration. An output takes audio from a single location, either directly from an input strip or from a mix. In both cases it can be specified whether to take the audio pre_fader or post_fader.

Note: The audio mixer is completely separate from the video mixer, so switching image in the video mixer will not change the audio mixer in any way.

Audio mixer root

The internal audio mixer consists of input strips (/strips), mixes (/mixes) and output buses (/outputs). The strips are the inputs of the audio mixer, taking audio from the inputs of the Rendering Engine. The mixes mix audio from strips and other mixes. Finally the output buses are the outputs of the audio mixer, taking audio from a single strip or a mix.

resource: /audio

Commands

{
   "type": "command",
   "resource": "/audio",
   "body": {
       "command": "reset"
   }
}

To reset the audio mixer to the default state, including removal of all configured input strips, use the reset command of the /audio resource.

Input strips

An input strip in the audio mixer, which takes audio from the input slots of the Rendering Engine (/input). Audio is either taken as a mono channel or as a stereo pair. The audio is sent through a filter chain (/filters). After the filter chain the output loudness of the strip is controlled by a main fader. There are peak meters placed before (/pre_fader_meter) and after (/post_fader_meter) the fader, as well as before the filter chain, measuring the raw audio input (/input_meter).

resource: /audio/strips/{strip index}

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}",
    "body": {
        "label": ""
    }
}

Input strip input configuration

Input settings for a strip.

resource: /audio/strips/{strip index}/input

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/input",
    "body": {
        "first_channel": 0,
        "input_slot": 0,
        "is_stereo": false,
        "second_channel": 1
    }
}

Input loudness meter

Peak loudness meter for incoming audio, measured before the filters of this input strip. This meter will either have one streaming parameter called ‘peak’ in case the strip is in mono mode, else two streaming parameters ‘peak_left’ and ‘peak_right’ will replace the ‘peak’ parameter when in stereo mode.

resource: /audio/strips/{strip index}/input_meter

Streaming parameters

Mid side stereo

Filter for controlling the mid and side amount of an audio signal. The input can either be Mid-Side (MS) or Left-Right (LR) stereo. Mono input will be passed through unaltered. If the input is LR, it is converted to MS with the mid channel being the average of the input channels and the side channel being half the difference of the channels. With the signal in MS format the mid and the side amount can be controlled using the mid_amount, side_amount, and invert_polarity parameters.

resource: /audio/strips/{strip index}/filters/mid_side

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/filters/mid_side",
    "body": {
        "enabled": false,
        "input_format": "lr_stereo",
        "invert_polarity": true,
        "mid_amount": 1.0,
        "side_amount": 1.0
    }
}

Pre-gain

Gain filter

resource: /audio/strips/{strip index}/filters/gain

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/filters/gain",
    "body": {
        "value": 0.0
    }
}

Parametric equalizer

Equalizer filter

resource: /audio/strips/{strip index}/filters/eq

Commands

{
   "type": "command",
   "resource": "/audio/strips/{strip index}/filters/eq",
   "body": {
       "command": "reset"
   }
}

Parametric equalizer bands

A filter/band in the equalizer

resource: /audio/strips/{strip index}/filters/eq/bands/{band index}

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/filters/eq/bands/{band index}",
    "body": {
        "freq": 1000.0,
        "gain": 0.0,
        "q": 0.707,
        "type": "none"
    }
}

Dynamic range compressor

Dynamic range compressor

resource: /audio/strips/{strip index}/filters/compressor

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/filters/compressor",
    "body": {
        "attack": 50.0,
        "gain": 0.0,
        "knee": 0.0,
        "ratio": 1.0,
        "release": 200.0,
        "threshold": 0.0
    }
}

Panning

Panning filter

resource: /audio/strips/{strip index}/filters/pan

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/filters/pan",
    "body": {
        "value": 0.0
    }
}

Pre-fader loudness meter

Peak loudness meter for filtered audio, measured after the filters of this input strip, but before the fader. This meter will always have two streaming parameters called ‘peak_left’ and ‘peak_right’, since the output of the strip is stereo.

resource: /audio/strips/{strip index}/pre_fader_meter

Streaming parameters

Input strip fader

Volume fader controlling the output loudness of this strip

resource: /audio/strips/{strip index}/fader

Parameters

{
    "type": "set",
    "resource": "/audio/strips/{strip index}/fader",
    "body": {
        "muted": false,
        "volume": 0.0
    }
}

Commands

{
   "type": "command",
   "resource": "/audio/strips/{strip index}/fader",
   "body": {
       "command": "fade",
       "parameters": {
           "volume": <float>,
           "duration_ms": <uint32>
       }
   }
}

Post-fader loudness meter

Peak loudness meter for filtered audio, measured after the fader and after the filters of this input strip. This meter will always have two streaming parameters called ‘peak_left’ and ‘peak_right’, since the output of the strip is stereo.

resource: /audio/strips/{strip index}/post_fader_meter

Streaming parameters

Mixes

A mix in the audio mixer, which takes audio from selected input strips and possibly other mixes to mix to a single stereo pair. After mixing the inputs (/inputs) of the mix, the audio is sent through a filter chain (/filters), similar to that of an input strip. After the filter chain the output loudness of the mix is controlled by a main fader. The inputs of a mix can either be taken pre- or post-fader from the strips and other mixes included in the mix. There are peak meters placed before (/pre_fader_meter) and after (/post_fader_meter) the fader, as well as before the filter chain, after the inputs has been mixed down to a single stereo pair (/input_meter).

resource: /audio/mixes/{mix index}

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}",
    "body": {
        "label": ""
    }
}

Mixes inputs

The inputs to this mix.

resource: /audio/mixes/{mix index}/inputs

Mixes inputs from input strips

Volume fader for controlling the contribution of the result of the other mix to this mix

resource: /audio/mixes/{mix index}/inputs/strips/{strip index}

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/inputs/strips/{strip index}",
    "body": {
        "muted": false,
        "origin": "post_fader",
        "volume": 0.0
    }
}

Commands

{
   "type": "command",
   "resource": "/audio/mixes/{mix index}/inputs/strips/{strip index}",
   "body": {
       "command": "fade",
       "parameters": {
           "volume": <float>,
           "duration_ms": <uint32>
       }
   }
}

Mixes inputs from other mixes

Volume fader for controlling the contribution of the result of the other mix to this mix

resource: /audio/mixes/{mix index}/inputs/mixes/{mix index}

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/inputs/mixes/{mix index}",
    "body": {
        "muted": false,
        "origin": "post_fader",
        "volume": 0.0
    }
}

Commands

{
   "type": "command",
   "resource": "/audio/mixes/{mix index}/inputs/mixes/{mix index}",
   "body": {
       "command": "fade",
       "parameters": {
           "volume": <float>,
           "duration_ms": <uint32>
       }
   }
}

Input loudness meter

Peak loudness meter for incoming audio, measured before the filters of this mix. This meter will always have two streaming parameters ‘peak_left’ and ‘peak_right’, since the input to the mix is always stereo.

resource: /audio/mixes/{mix index}/input_meter

Streaming parameters

Pre-gain

Gain filter

resource: /audio/mixes/{mix index}/filters/gain

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/filters/gain",
    "body": {
        "value": 0.0
    }
}

Parametric equalizer

Equalizer filter

resource: /audio/mixes/{mix index}/filters/eq

Commands

{
   "type": "command",
   "resource": "/audio/mixes/{mix index}/filters/eq",
   "body": {
       "command": "reset"
   }
}

Parametric equalizer bands

A filter/band in the equalizer

resource: /audio/mixes/{mix index}/filters/eq/bands/{band index}

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/filters/eq/bands/{band index}",
    "body": {
        "freq": 1000.0,
        "gain": 0.0,
        "q": 0.707,
        "type": "none"
    }
}

Dynamic range compressor

Dynamic range compressor

resource: /audio/mixes/{mix index}/filters/compressor

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/filters/compressor",
    "body": {
        "attack": 50.0,
        "gain": 0.0,
        "knee": 0.0,
        "ratio": 1.0,
        "release": 200.0,
        "threshold": 0.0
    }
}

Panning

Panning filter

resource: /audio/mixes/{mix index}/filters/pan

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/filters/pan",
    "body": {
        "value": 0.0
    }
}

Pre-fader loudness meter

Peak loudness meter for filtered audio, measured after the filters of this mix, but before the fader. This meter will always have two streaming parameters called ‘peak_left’ and ‘peak_right’, since the output of the mix is stereo.

resource: /audio/mixes/{mix index}/pre_fader_meter

Streaming parameters

Mix output fader

Volume fader controlling the output loudness of this mix

resource: /audio/mixes/{mix index}/fader

Parameters

{
    "type": "set",
    "resource": "/audio/mixes/{mix index}/fader",
    "body": {
        "muted": false,
        "volume": 0.0
    }
}

Commands

{
   "type": "command",
   "resource": "/audio/mixes/{mix index}/fader",
   "body": {
       "command": "fade",
       "parameters": {
           "volume": <float>,
           "duration_ms": <uint32>
       }
   }
}

Post-fader loudness meter

Peak loudness meter for filtered audio, measured after the fader and after the filters of this mix. This meter will always have two streaming parameters called ‘peak_left’ and ‘peak_right’, since the output of the strip is stereo.

resource: /audio/mixes/{mix index}/post_fader_meter

Streaming parameters

Output bus

Audio mixer output bus to select which input strip or mix should be sent to the output of the audio mixer. The outputs are populated from the Rendering Engine config. Audio from a strip or a mix can be selected for the output. The selected source of this output can either be taken pre- or post-fader from the referenced strip or mix.This makes it possible for outputs meant for pre-listening to listen to a mix or strip without touching its fader. Such outputs can also jump freely between mixes to listen to what is being output from the mixer or strips to listen to the incoming components to tweak the audio filters. The selected audio is fed through a loudness meter.

resource: /audio/outputs/{output name}

Parameters

{
    "type": "set",
    "resource": "/audio/outputs/{output name}",
    "body": {
        "label": ""
    }
}

Output bus input

The source of an output bus.

resource: /audio/outputs/{output name}/input

Parameters

{
    "type": "set",
    "resource": "/audio/outputs/{output name}/input",
    "body": {
        "index": 0,
        "origin": "post_fader",
        "source": "mix"
    }
}

Output bus loudness meter

Loudness meters for the output bus. Used to monitor the loudness of the outgoing audio from the mixer.This meter will always have two streaming parameters called ‘peak_left’ and ‘peak_right’. When the parameter ’enable_ebu_meters’ is set to true, three additional streaming parameters will be available, called ’ebu_m’, ’ebu_s’ and ’ebu_i’, which measure loudness according to the EBU R 128 standard.

resource: /audio/outputs/{output name}/meters

Parameters

{
    "type": "set",
    "resource": "/audio/outputs/{output name}/meters",
    "body": {
        "enable_ebu_meters": false
    }
}

Streaming parameters

Commands

{
   "type": "command",
   "resource": "/audio/outputs/{output name}/meters",
   "body": {
       "command": "reset"
   }
}

External audio mixing via NDI

The internal audio mixer can be replaced with an external audio mixer, where NDI is used for communicating with the external mixer. To enable the external NDI audio mixer the Rendering Engine application should be started with the environment variable ACL_AUDIO_MIXER set to external_ndi.

The external NDI audio mixer bridge

resource: /ndi_audio

Sending audio to external mixer

All streams to send to the external audio mixer

resource: /ndi_audio/send_streams

Commands

{
   "type": "command",
   "resource": "/ndi_audio/send_streams",
   "body": {
       "command": "add_stream",
       "parameters": {
           "name": <string>
       }
   }
}

{
   "type": "command",
   "resource": "/ndi_audio/send_streams",
   "body": {
       "command": "remove_stream",
       "parameters": {
           "name": <string>
       }
   }
}

A stream to send to the external audio mixer

resource: /ndi_audio/send_streams/{send stream index}

Parameters

Commands

{
   "type": "command",
   "resource": "/ndi_audio/send_streams/{send stream index}",
   "body": {
       "command": "disable_channel",
       "parameters": {
           "index": <uint32>
       }
   }
}

{
   "type": "command",
   "resource": "/ndi_audio/send_streams/{send stream index}",
   "body": {
       "command": "enable_channel",
       "parameters": {
           "index": <uint32>,
           "input_slot": uint32,
           "channel": uint32
       }
   }
}

List of audio channels to send

resource: /ndi_audio/send_streams/{send stream index}/channels

A channel to send to the external audio mixer

resource: /ndi_audio/send_streams/{send stream index}/channels/{channel index}

Parameters

{
    "type": "set",
    "resource": "/ndi_audio/send_streams/{send stream index}/channels/{channel index}",
    "body": {
        "channel": 0,
        "input_slot": 0
    }
}

Receiving audio in return from the external mixer

All streams that returns audio from the external audio mixer

resource: /ndi_audio/return_streams

Commands

{
   "type": "command",
   "resource": "/ndi_audio/return_streams",
   "body": {
       "command": "add_stream",
       "parameters": {
           "name": <string>
       }
   }
}

{
   "type": "command",
   "resource": "/ndi_audio/return_streams",
   "body": {
       "command": "remove_stream",
       "parameters": {
           "name": <string>
       }
   }
}

A stream which returns audio from the external audio mixer

resource: /ndi_audio/return_streams/{return stream index}

Parameters

Mapping received audio to outputs

All audio outputs from this component

resource: /ndi_audio/outputs

An output from this component consisting of audio from one or more NDI return streams

resource: /ndi_audio/outputs/{output bus}

Parameters

The list of channels for one output

resource: /ndi_audio/outputs/{output bus}/channels

An output channel

resource: /ndi_audio/outputs/{output bus}/channels/1

Parameters