connect
search

NTP Validations

AVAILABLE STARTING WITH 2022.10

About NTP Validations

This page contains details about the types of warnings and error messages displayed at NTP job-level.

Job pre-validation (at Save job, in Job Analysis section)

Job Status: Valid/Invalid. The job status is invalid if at least one Validation error is present.

Validation errors:

Validation warnings:

Also, pop-up errors can occur during this Save job step.

Pop-up Errors


Scenario:

  • Job template (rendered) is not a valid JSON so cannot be sent to Hybrik/MC

The Failed to Parse Error

Failed to parse JSON template! SyntaxError: Unexpected token <token> in JSON at position <position> ...

Scenario: DRM is enabled in NTP and we want to get the DRM variable name, in order to use it further for resolving ‘<drmVariableName>.<field>’ references in Job template

The reserved keywords are:

  • entity
  • explicitFile
  • sourceFileName
  • sourcePath
  • watermark

The DRM variable name Error

The DRM variable name &lt;drmVariableName&gt; is a reserved keyword and cannot be used

Scenario: DRM is enabled in NTP and the DRM required fields are not completed (nothing new - it existed before, but cannot be reproduced in FE, because FE enforces that all DRM required fields are completed ‘Please complete the required fields before saving’)

The DRM template has not been defined

Validation Errors

Note

A validation error can occur in two scenarios:

1. any of the NTP input file cannot be fetched

  • this is available since the old implementation

2. LiquidJS template engine raises an error during the NTP Job Template rendering

  • this is the first error encountered and raised by the LiquidJS template engine
  • after this error is fixed, another errors can be encountered and raised by the LiquidJS template engine
  • the cause of the error can be one of the displayed Validation warnings, for a value that is undefined due to unresolved values from the application

e.g. Examples of various scenarios - Scenario 5, Scenario 6, Scenario 7

  • this is available since the old implementation, but until now it was in the form of a pop-up error message, as presented in the image below:

The Invalid Transcode JSON Template Error


Scenario: any of the NTP input file cannot be fetched

e.g. for Tag NTP input file

Validation errors:
Error fetching the file associated with the following tag(s) &lt;- e.g. for Tag
* Missing Additional Content 3 Video media item from UPTV Apple Season 1

e.g. for Metadata field NTP input file

Validation errors:
Other issues
* Missing source for ['global']['Attr'] &lt;- e.g. Metadata field
* Missing source for ['global']['Attr2'] &lt;- e.g. Metadata field

e.g. for Explicit NTP input file

Validation errors:
Other issues
* Missing source for 1b7af900-338e-4a8e-a031-e600c80d2129 &lt;- e.g. Explicit

Validations Error

Inputs


Scenario: LiquidJS template engine raises an error during the NTP Job Template rendering

Validation errors:
undefined variable: watermark, line:1, col:1, line:12, col:22

The error message raised by the template engine (LiquidJS) has the following structure:

undefined variable: VARIABLE, line: LINE_VARIABLE, col: COL_VARIABLE, line: LINE_TOKEN, col: COL_TOKEN, where:

  • LINE_VARIABLE - the line where the VARIABLE is situated inside the LiquidJS token
  • COL_VARIABLE - the column where the VARIABLE is situated inside the LiquidJS token
  • LINE_TOKEN - the line where the LiquidJS token (that contains the undefined VARIABLE) is situated inside the template
  • COL_TOKEN - the column where the LiquidJS token (that contains the undefined VARIABLE) is situated inside the template

Examples:

undefined variable: entity, line:1, col:1, line:1, col:1 {{entity.parent.parent.attributes['en_US']['simple'] }}

undefined variable: entity, line:1, col:15, line:1, col:1

{{ "now" | date: entity.parent.parent.attributes['en_US']['simple'] }}

undefined variable: entity, line:2, col:15, line:1, col:1 {{ "now" | date: entity.parent.parent.attributes['en_US']['simple'] }} undefined variable: entity, line:2, col:15, line:2, col:7 {{ "March 14, 2016" | date: "%b %d, %y" }} {{ "now" | date: entity.parent.parent.attributes['en_US']['simple'] }}

  • FE required changes, to integrate this new TemplateValidationError:
{
"type": "TemplateValidationError",
"message": "undefined variable: videoTrack, line:1, col:1, line:11, col:22",
"item": "",
"config": {
"type": "template_engine_error",
"__typename": "TemplateErrorConfig"
},
"entity": null,
"__typename": "TemplateValidationError"
}

Validation warnings

Some of the previously defined errors or warnings (previously defined and used for metadata template or deliverable name format) will appear now under the Validation warnings section, because they are NTP template syntax related:

  • Error fetching the following tag(s)
  • Unable to resolve the following
  • Error solving the following unsupported file properties
  • Unable to resolve the remaining syntax
  • Unable to parse attributes
  • The following attribute(s) have not been defined
  • The following attribute(s) are empty
  • The following locale(s) for attribute are not valid
  • Unable to resolve the following item(s)
  • Error fetching the file of the following Tag attribute(s)
  • Error fetching the file of the following File attribute(s)
  • Unable to resolve the following
  • Error solving the following properties due to file missing
  • Error fetching the file associated with the following tag(s)

Scenario: Using explicitFile[<id>] inside the NTP Job Template

Validation warnings:
Error fetching the file with the following id(s)
* &lt;id&gt;
  • FE required changes, to integrate this new TemplateValidationWarning:
{
"type": "TemplateValidationWarning",
"message": "Error fetching the file with the following id(s)",
"item": "30a159a7-8fa4-451f-ba44-218059ca86fg",
"config": {
"type": "ntp_missing_file",
"__typename": "TemplateErrorConfig"
},
"entity": null,
"__typename": "TemplateValidationWarning"
}

Scenario: Using videoTrack.<property> inside the NTP Job Template

Validation warnings:
Cannot find video track
* videoTrack for 'linkToFile'
  • FE required changes, to integrate this new TemplateValidationWarning → must integrate the Fix modal.
{
"type": "TemplateValidationWarning",
"message": "Cannot find video track",
"item": "videoTrack",
"config": {
"type": "ntp_missing_video_track",
"__typename": "TemplateErrorConfig"
},
"entity": {
"id": "62557ac0-2783-4915-8592-e7d85f655ca1",
"name": "File_meta_image_C_bis.jpeg",
"type": "file",
"__typename": "TemplateErrorEntity"
},
"__typename": "TemplateValidationWarning"
}

Scenario: Using
audioTracks[<tag>][&lt;index&gt;].&glt;property>
timedTextTracks[<tag>][&lt;index&gt;].<property> inside the NTP Job Template

Validation warnings:
Cannot find track or index is out of bounds
* audioTracks[&lt;tag&gt;][&lt;index&gt;] for 'linkToFile'
* timedTextTracks[&lt;tag&gt;][&lt;index&gt;] for 'linkToFile'
  • FE required changes, to integrate this new TemplateValidationWarning → must integrate the Fix modal.
{
"type": "TemplateValidationWarning",
"message": "Cannot find track or index is out of bounds",
"item": "audioTracks['Audio_track'][0]",
"config": {
"type": "ntp_track_out_of_bounds_index",
"__typename": "TemplateErrorConfig"
},
"entity": {
"id": "62557ac0-2783-4915-8592-e7d85f655ca1",
"name": "File_meta_image_C_bis.jpeg",
"type": "file",
"__typename": "TemplateErrorEntity"
},
"__typename": "TemplateValidationWarning"
}

Scenario: Using
audioTracks[<tag>][&lt;index&gt;].<property>
timedTextTracks[<tag>][&lt;index&gt;].<property> inside the NTP Job Template

Validation warnings:
Error fetching the following track tag(s)
* &lt;tagName&gt;
  • FE required changes, to integrate this new TemplateValidationWarning → must integrate the Fix modal.
{
"type": "TemplateValidationWarning",
"message": "Error fetching the following track tag(s)",
"item": "Audio_track_wrong_tag_name",
"config": {
"type": "ntp_missing_tag",
"__typename": "TemplateErrorConfig"
},
"entity": null,
"__typename": "TemplateValidationWarning"
}

Scenario: Any watermark issues, when ‘watermark.<field> is referenced in Job template

Validation warnings:
The watermark provider has not been defined
* watermark.&lt;field&gt;
Validation warnings:
Unable to find the watermark provider
* watermark.&lt;field&gt;
Validation warnings:
Could not generate a unique watermarkValue for the watermark provider
* watermark.&lt;field&gt;
Validation warnings:
Unsupported watermark provider type &lt;watermarkProviderType&gt;
* watermark.&lt;field&gt;
  • FE required changes, to integrate this new TemplateValidationWarning:
general e.g.
{
"type": "TemplateValidationWarning",
"message": "&lt;any of the above messages&gt;",
"item": "watermark.&lt;field&gt;",
"config": {
"type": "message",
"__typename": "TemplateErrorConfig"
},
"entity": null,
"__typename": "TemplateValidationWarning"
}
concrete e.g.
{
"type": "TemplateValidationWarning",
"message": "The watermark provider has not been defined",
"item": "watermark.name",
"config": {
"type": "message",
"__typename": "TemplateErrorConfig"
},
"entity": null,
"__typename": "TemplateValidationWarning"
}

Scenario: Any DRM issues, when <drmVariableName>.<field> is referenced in Job template

Validation warnings:
At least one DRM content key should be defined
* &lt;drmVariableName&gt;.&lt;field&gt;

Validation warnings:
Unable to find the DRM provider
* &lt;drmVariableName&gt;.&lt;field&gt;

Validation warnings:
The DRM encryption information request could not be signed.Please check your DRM provider configuration.
* &lt;drmVariableName&gt;.&lt;field&gt;

Validation warnings:
The signature of the DRM encryption information response could not be correctly verified.Please check your DRM provider configuration.
* &lt;drmVariableName&gt;.&lt;field&gt;

Validation warnings:
The DRM encryption information configuration is invalid.Please check your DRM provider configuration.
* &lt;drmVariableName&gt;.&lt;field&gt;

Validation warnings:
The DRM encryption information could not be requested from your CPIX provider.Please check your DRM provider configuration or consult with your CPIX provider.
* &lt;drmVariableName&gt;.&lt;field&gt;

Validation warnings:
The DRM encryption information response cannot be decrypted, most likely because of an invalid certificate. Please check your DRM provider configuration.
* &lt;drmVariableName&gt;.&lt;field&gt;

Validation warnings:
There is data missing or invalid in the DRM encryption information request.
* &lt;drmVariableName&gt;.&lt;field&gt;
  • FE required changes, to integrate this new TemplateValidationWarning:
general e.g.
{
"type": "TemplateValidationWarning",
"message": "&lt;any of the above messages&gt;",
"item": "&lt;drmVariableName&gt;.&lt;field&gt;",
"config": {
"type": "message",
"__typename": "TemplateErrorConfig"
},
"entity": null,
"__typename": "TemplateValidationWarning"
}
concrete e.g.
{
"type": "TemplateValidationWarning",
"message": "The DRM encryption information could not be requested from your CPIX provider. Please check your DRM provider configuration or consult with your CPIX provider.",
"item": "drm.name",
"config": {
"type": "message",
"__typename": "TemplateErrorConfig"
},
"entity": null,
"__typename": "TemplateValidationWarning"
}

Examples of various scenarios


Scenario 1:

Given:
NTP input file:

  • Type: File
  • Resolution mode: Metadata tag
  • Metadata field: ['en-US']['Amazon_JPG_Thumbnail2_2560x1920_4x3']

and the ‘Amazon_JPG_Thumbnail2_2560x1920_4x3’ attribute is not defined, so the NTP input file cannot be fetched.
Result:

Validation errors:

Other issues

  • Missing source for ['en-US']['Amazon_JPG_Thumbnail2_2560x1920_4x3']

Validation warnings:


Scenario 2:

Given:
NTP input file:

  • Type: File
  • Resolution mode: Metadata file
  • Metadata field: ['en-US']['Amazon_File_2560x1920_4x3']

and the ‘Amazon_File_2560x1920_4x3’ attribute is not defined, so the NTP input file cannot be fetched.
Result:

Validation errors:

Other issues

  • Missing source for ['en-US']['Amazon_File_2560x1920_4x3']

Validation warnings:

Scenario 3:

Given:
NTP input file:

  • Type: File
  • Resolution mode: Tag
  • Tag: ‘Additional Content 3 Caption’

and the ‘Additional Content 3 Caption’ tag is not associated to a file, so the NTP input file cannot be fetched.
Result:

Validation errors:

Error fetching the file associated with the following tag(s)

  • Missing Additional Content 3 Caption media item from UPTV Apple Episode 1

Validation warnings:

Scenario 4:

Given:
NTP input file:

  • Type: File
  • Resolution mode: Explicit
  • Explicit: pick a file, with id: d60a850c-3d42-41fb-8f7e-a3abfab9947e

and in the meantime, the file with id: d60a850c-3d42-41fb-8f7e-a3abfab9947e was deleted/trashed, so the NTP input file cannot be fetched.
Result:

Validation errors:

Other issues

  • Missing source for d60a850c-3d42-41fb-8f7e-a3abfab9947e

Validation warnings:


Scenario 5:

Given:

I use the file ‘{# entity.attributes['en-US']['Amazon_JPG_Thumbnail2_2560x1920_4x3'].locator.basename #}’ inside the NTP Job template

and the ‘Amazon_JPG_Thumbnail2_2560x1920_4x3’ attribute is not defined, so the file referenced in the NTP Job template cannot be fetched.

Here, the cause of the error from Validation errors is the one displayed under Validation warnings, having a value that is undefined due to unresolved values from the application.

Result:

Validation errors:

undefined variable: attributes, line:1, col:2, line:4, col:1

Validation warnings:

The following attribute(s) have not been defined

  • Amazon_JPG_Thumbnail2_2560x1920_4x3 for UPTV Apple Episode 1

Scenario 6:

Given:
I enabled DRM in NTP, with Variable name: drm.
I use '{# drm.wrongfield #}' inside the NTP Job template, when the wrongfield is not a supported drm property
Result:

Validation errors:

undefined variable: drm, line:1, col:1, line:126, col:63

Validation warnings:

Unable to resolve the remaining syntax

  • wrongfield

Scenario 7:

Given:

I use the file ‘{# explicitFile['75165616-d0b6-4472-b588-f7b60fc7754dh'].locator.basename #}’ inside the NTP Job template

and the ‘75165616-d0b6-4472-b588-f7b60fc7754dh’ file id is not pointing to a file, so the file referenced in the NTP Job template cannot be fetched.
Result:

Validation errors:

undefined variable: explicitFile, line:1, col:1, line:11, col:22

Validation warnings:

Error fetching the file with the following id(s)

  • 75165616-d0b6-4472-b588-f7b60fc7754dh

Scenario 8:

Given:

I use a value ‘entity.nodeTypeId’ that is null or undefined, inside the NTP Job template

{
"task_tags":[
"dev"
],
"definitions":{
"profile_name":"mgm_abr_ladder",
"source_path":"{# file1.locator.dirname #}",
"source_name":"{# entity.attributes['en-US']['Amazon_JPG_Thumbnail_2560x1920_4x3'].locator.basename #}{#file1.locator.extname#}",
"transcode_preset":"medium_{# entity.nodeTypeId #}",
"fmp4_segment_duration_sec":4,
"destination_path":"s3://tf-s3-ownzones-char/test/deliverables/Native_outputs/{{output_basename}}",
"output_basename":"1_output_6_1_2021"
},
……
}

Result:

Validation errors:

undefined variable: nodeTypeId, line:1, col:1, line:11, col:34


Scenario 9:

Given:

I use a value ‘entity.nodeTypeId’ that is null or undefined, but inside a if statement, inside the NTP Job template

{
"task_tags": [
"dev"
],
"definitions": {
"profile_name":"mgm_abr_ladder",
"source_path":"{# file1.locator.dirname #}",
"source_name":"{# entity.attributes['en-US']['Amazon_JPG_Thumbnail_2560x1920_4x3'].locator.basename #}{#file1.locator.extname#}",
"transcode_preset":"medium_{% if entity.nodeTypeId %}awesome_transcode_preset{% endif %}",
"fmp4_segment_duration_sec":4,
"destination_path":"s3://tf-s3-ownzones-char/test/deliverables/Native_outputs/{{output_basename}}",
"output_basename":"1_output_6_1_2021"
},
……
}

Result:

Job valid


Scenario 10 (similar with Scenario 7, but inside an if statement):

Given:

I use the file ‘{# explicitFile['75165616-d0b6-4472-b588-f7b60fc7754dh'].locator.url #}’ inside the NTP Job template, and inside an if statement

{% if explicitFile['75165616-d0b6-4472-b588-f7b60fc7754dh'].locator.basename %}

{% endif %}

and the ‘75165616-d0b6-4472-b588-f7b60fc7754dh’ file id is not pointing to a file, so the file referenced in the NTP Job template cannot be fetched.

Note

There will be no Validation errors because the LiquidJs template engine raises an error during the NTP Job Template rendering only for undefined values that are not the condition* to an if, elsif, or unless tag.

*the if/elsif/unless has only one condition that is resolved to an undefined value, without having any other operators inside the if/elsif/unless (by operators, we mean comparison operators: ==, !=, >, <, >=, <= , and logic operators: or, and)

{% if explicitFile['75165616-d0b6-4472-b588-f7b60fc7754dh'].locator.url %}no Validation errors (if has only one condition that is resolved to an undefined value)

...

{% endif %}

{% if explicitFile['75165616-d0b6-4472-b588-f7b60fc7754dh'].locator.url != ' ' %}

ONE Validation errors (if has condition that is resolved to an undefined value + comparison operator ‘!=’)

...

{% endif %}

{% if explicitFile['75165616-d0b6-4472-b588-f7b60fc7754dh'].locator.url and true %}ONE Validation errors (if has condition that is resolved to an undefined value + logic operator 'and')

...

{% endif %}

For more details, see Options | LiquidJS documentation for the options strictVariables and lenientIf that are set on true inside the Connect backend services.

Result:

Validation warnings:

Error fetching the file with the following id(s)

  • 75165616-d0b6-4472-b588-f7b60fc7754dh

Scenario 11:

Given:
I enabled DRM in NTP, with Variable name: drm.

I use '{# drm.name #}' inside the NTP Job template.

NTP is configured to use a DRM provider with an incorrect URL.

Result Job pre-validation (at Save job, in Job Analysis section):

Validation errors:

undefined variable: drm, line:1, col:1, line:11, col:34

Validation warnings:

A valid url must be provided.

  • drm.name

Result Job Runtime/ Quick Transcode/etc:

undefined variable: drm, line:1, col:1, line:11, col:34
A valid url must be provided.

And this message can be seen as in the below examples:

Job Runtime (at Start job, in Build Workflow → Transcode Task section):

Job Runtime Error

OR

Quick Transcode (Media → Workbench tab → Transcode → Run → Pop-up error):

Quick Transcode Error

Quick Transcode Error

Job Runtime

Before this development, if an error that was thrown during runtime, it was caught and shown in the build status of the job/task. This is unchanged.

Here are two types of errors that existed:

  • any LiquidJS error.
  • any errors thrown by Connect

Will focus on DRM / Watermark issues.

Before this development, DRM / Watermark logic was done before parsing the template. Now, this logic is done while parse template.

So now, if a DRM/Watermark error is thrown, a LiquidJS error will be thrown as well. To have backwards compatibility, when this happens, we will show both errors.

Check last scenario with DRM, before this chapter.

UI behavior:

  • in transcode task, if the error is on multiple lines, it appears on multiple lines.
  • in build status, the error is always on one line.

Examples of various scenarios


Scenario: Missing title attribute

  • Have a job with a transcode deliverable.
  • The transcode profile uses entity.attributes['en-US']['Description'] in the template.
  • The title has a value for the above attribute.
  • Save the job. Job should be valid.
  • Delete the above attribute value.
  • Run the job. Job will have the following error.

Error: undefined variable: attributes, line:1, col:1, line:11, col:46

Job

Example job

Arrakis Job

NTP Profile

Example NTP Profile
{
"definitions":{
"profile_name":"AN 11218-1",
"source_path":"{# file1.locator.dirname #}/",
"source_name":"{# file1.locator.basename #}{# file1.locator.extname #}",
"destination_path":"s3://tf-s3-ownzones-arrakis/testarrakis/deliverables/Native_outputs/Copy_NTP/{# 'now' | date: '%Y-%m-%d_%H-%M-%S' #}/",
"credentials_key":"oz-dev-credentials"
},
"name":"{{profile_name}}: {{source_name}} {# entity.attributes['en-US']['Description'] #}",
"payload":{
"elements":[
{
"uid":"sources",
"kind":"source",
"payload":{
"kind":"asset_url",
"payload":{
"storage_provider":"s3",
"url":"{{source_path}}/{{source_name}}"
}
}
},
{
"uid":"copy_task",
"kind":"copy",
"payload":{
"target":{
"location":{
"storage_provider":"s3",
"path":"{{destination_path}}"
},
"existing_files":"rename_new",
"file_pattern":"NTP_copy_{{source_name}}"
}
}
}
],
"connections":[
{
"from":[
{
"element":"sources"
}
],
"to":{
"success":[
{
"element":"copy_task"
}
]
}
}
]
}
}

Scenario: Wrong KID title attribute

  • Have a job with a transcode deliverable.
  • The transcode profile uses DRM and has {# entity.attributes['en-US']['KID'] #} in the KID template.
  • The title has an uuid for the above attribute.
  • Save the job. Job should be valid.
  • Change the above attribute value to a random string.
  • Run the job. Job will have the following error.

Error: undefined variable: drm, line:1, col:1, line:13, col:24

There is data missing or invalid in the DRM encryption information request.

  • field 'KID' of content key #1 (with intended track type value: 'HD') with value 'be3c17': kid must be an UUID

Job

Example job

Arrakis Job 2

NTP Profile

Example NTP Profile
{
"definitions": {
"profile_name": "AN_11212_profile_",
"source_path": "{# file1.locator.dirname #}",
"source_name": "{# file1.locator.basename #}{# file1.locator.extname #}",
"output_basename": "{# entity.name #}",
"destination_path": "s3://tf-s3-ownzones-arrakis/testarrakis/deliverables/Native_outputs/{# 'now' | date: '%Y-%m-%d_%H-%M-%S' #}",
"transcode_preset": "medium",
"fmp4_segment_duration_sec": 6,
"credentials_key": "oz-dev-credentials",
"title_name": "{# drm.name #}"
},
"name": "{{profile_name}}: {{source_name}} - {{output_basename}}",
"priority": 100,
"task_tags": ["dev"],
"payload": {
"elements": [
{
"uid": "source_file",
"kind": "source",
"payload": {
"kind": "asset_url",
"payload": {
"storage_provider": "s3",
"url": "{{source_path}}/{{source_name}}"
}
}
},
{
"uid": "transcode",
"kind": "transcode",
"payload": {
"location": {
"storage_provider": "s3",
"path": "{{destination_path}}/mp4s"
},
"targets": [
{
"file_pattern": "{{output_basename}}_video_6500{default_extension}",
"existing_files": "rename_new",
"container": {
"kind": "fmp4",
"segment_duration": "{{fmp4_segment_duration_sec}}"
},
"video": {
"codec": "h264",
"bitrate_mode": "cbr",
"use_scene_detection": false,
"bitrate_kb": 6500,
"vbv_buffer_size_kb": 6500,
"height": 1080,
"par": "1:1",
"level": "3.1",
"profile": "high"
}
},
{
"file_pattern": "{{output_basename}}_video_5000{default_extension}",
"existing_files": "rename_new",
"container": {
"kind": "fmp4",
"segment_duration": "{{fmp4_segment_duration_sec}}"
},
"video": {
"codec": "h264",
"bitrate_mode": "cbr",
"use_scene_detection": false,
"bitrate_kb": 5000,
"vbv_buffer_size_kb": 5000,
"height": 720,
"par": "1:1",
"level": "3.1",
"profile": "high"
}
},
{
"file_pattern": "{{output_basename}}_audio_128{default_extension}",
"existing_files": "replace",
"container": {
"kind": "fmp4",
"segment_duration": "{{fmp4_segment_duration_sec}}"
},
"audio": [
{
"channels": 2,
"codec": "aac_lc",
"sample_rate": 48000,
"bitrate_kb": 128,
"layer_id": "audio_high"
}
]
}
]
}
},
{
"uid": "package_dash_cmaf",
"kind": "package",
"payload": {
"uid": "hls_and_dash_manifest",
"kind": [
"dash",
"hls"
],
"location": {
"storage_provider": "s3",
"path": "{{destination_path}}/cmaf/dash"
},
"attributes": [
{
"name": "ContentType",
"value": "application/x-mpegURL"
}
],
"dash": {
"file_pattern": "master_manifest.mpd"
},
"hls": {
"file_pattern": "master_manifest.m3u8"
},
"segmentation_mode": "fmp4",
"force_original_media": false,
"encryption": {
"enabled": true,
"schema": "mpeg-cbcs",
"drm": [
"playready",
"widevine"
],
"key_id": "{# drm.contentKeys['HD'].kid | replace: '-', '' #}",
"key": "{# drm.contentKeys['HD'].key | hex #}",
"playready_url": "https://pr-keyos.licensekeyserver.com/core/rightsmanager.asmx",
"playready_pssh": "{# drm.contentKeys['HD'].playreadyPssh | base64 #}",
"widevine_provider": "buydrmkeyos",
"clearkey_pssh_version": 1
}
}
}
],
"connections": [
{
"from": [
{
"element": "source_file"
}
],
"to": {
"success": [
{
"element": "transcode"
}
]
}
},
{
"from": [
{
"element": "transcode"
}
],
"to": {
"success": [
{
"element": "package_dash_cmaf"
}
]
}
}
]
},
"subscription_key": "char"
}

Scenario: Wrong DRM provider URL

  • Have a job with a transcode deliverable.
  • The transcode profile uses DRM.
  • Save the job. Job should be valid.
  • Change DRM provider URL to a random string.
  • Run the job. Job will have the following error.

Error: undefined variable: drm, line:1, col:1, line:13, col:24

A valid DRM URL must be provided.

Job

Example job

Arrakis Job 2

NTP Profile

Example NTP Profile
{
"definitions": {
"profile_name": "AN_11218_profile_",
"source_path": "{# file1.locator.dirname #}",
"source_name": "{# file1.locator.basename #}{# file1.locator.extname #}",
"output_basename": "{# entity.name #}",
"destination_path": "s3://tf-s3-ownzones-arrakis/testarrakis/deliverables/Native_outputs/{# 'now' | date: '%Y-%m-%d_%H-%M-%S' #}",
"transcode_preset": "medium",
"fmp4_segment_duration_sec": 6,
"credentials_key": "oz-dev-credentials",
"title_name": "{# drm.name #}"
},
"name": "{{profile_name}}: {{source_name}} - {{output_basename}}",
"priority": 100,
"payload": {
"elements": [
{
"uid": "source_file",
"kind": "source",
"payload": {
"kind": "asset_url",
"payload": {
"storage_provider": "s3",
"url": "{{source_path}}/{{source_name}}"
}
}
},
{
"uid": "transcode",
"kind": "transcode",
"payload": {
"location": {
"storage_provider": "s3",
"path": "{{destination_path}}/mp4s"
},
"targets": [
{
"file_pattern": "{{output_basename}}_video_6500{default_extension}",
"existing_files": "rename_new",
"container": {
"kind": "fmp4",
"segment_duration": "{{fmp4_segment_duration_sec}}"
},
"video": {
"codec": "h264",
"bitrate_mode": "cbr",
"use_scene_detection": false,
"bitrate_kb": 6500,
"vbv_buffer_size_kb": 6500,
"height": 1080,
"par": "1:1",
"level": "3.1",
"profile": "high"
}
},
{
"file_pattern": "{{output_basename}}_video_5000{default_extension}",
"existing_files": "rename_new",
"container": {
"kind": "fmp4",
"segment_duration": "{{fmp4_segment_duration_sec}}"
},
"video": {
"codec": "h264",
"bitrate_mode": "cbr",
"use_scene_detection": false,
"bitrate_kb": 5000,
"vbv_buffer_size_kb": 5000,
"height": 720,
"par": "1:1",
"level": "3.1",
"profile": "high"
}
},
{
"file_pattern": "{{output_basename}}_audio_128{default_extension}",
"existing_files": "replace",
"container": {
"kind": "fmp4",
"segment_duration": "{{fmp4_segment_duration_sec}}"
},
"audio": [
{
"channels": 2,
"codec": "aac_lc",
"sample_rate": 48000,
"bitrate_kb": 128,
"layer_id": "audio_high"
}
]
}
]
}
},
{
"uid": "package_dash_cmaf",
"kind": "package",
"payload": {
"uid": "hls_and_dash_manifest",
"kind": [
"dash",
"hls"
],
"location": {
"storage_provider": "s3",
"path": "{{destination_path}}/cmaf/dash"
},
"attributes": [
{
"name": "ContentType",
"value": "application/x-mpegURL"
}
],
"dash": {
"file_pattern": "master_manifest.mpd"
},
"hls": {
"file_pattern": "master_manifest.m3u8"
},
"segmentation_mode": "fmp4",
"force_original_media": false,
"encryption": {
"enabled": true,
"schema": "mpeg-cbcs",
"drm": [
"playready",
"widevine"
],
"key_id": "{# drm.contentKeys['HD'].kid | replace: '-', '' #}",
"key": "{# drm.contentKeys['HD'].key | hex #}",
"playready_url": "https://pr-keyos.licensekeyserver.com/core/rightsmanager.asmx",
"playready_pssh": "{# drm.contentKeys['HD'].playreadyPssh | base64 #}",
"widevine_provider": "buydrmkeyos",
"clearkey_pssh_version": 1
}
}
}
],
"connections": [
{
"from": [
{
"element": "source_file"
}
],
"to": {
"success": [
{
"element": "transcode"
}
]
}
},
{
"from": [
{
"element": "transcode"
}
],
"to": {
"success": [
{
"element": "package_dash_cmaf"
}
]
}
}
]
},
"subscription_key": "char"
}

Quick Transcode

Before this development, if an error was thrown when starting transcode, it was caught and displayed in a pop-up. This is unchanged.

Similar scenarios as Job runtime, with the addition of a Watermark scenario.

UI behavior:

  • the errors is always on one line.

Examples of various scenarios


Scenario: Missing title attribute

  • Have a transcode profile that uses entity.attributes['en-US']['Description'] in the template.
  • Choose a media attached to a title that does not have the above attribute value.
  • Run transcode using the transcode profile.
  • A popup will appear with the following error.

undefined variable: entity, line:1, col:1, line:11, col:46

Media

Example Media

Media

NTP Profile

Example NTP Profile
{
"definitions":{
"profile_name":"AN 11218-1",
"source_path":"{# file1.locator.dirname #}/",
"source_name":"{# file1.locator.basename #}{# file1.locator.extname #}",
"destination_path":"s3://tf-s3-ownzones-arrakis/testarrakis/deliverables/Native_outputs/Copy_NTP/{# 'now' | date: '%Y-%m-%d_%H-%M-%S' #}/",
"credentials_key":"oz-dev-credentials"
},
"name":"{{profile_name}}: {{source_name}} {# entity.attributes['en-US']['Description'] #}",
"payload":{
"elements":[
{
"uid":"sources",
"kind":"source",
"payload":{
"kind":"asset_url",
"payload":{
"storage_provider":"s3",
"url":"{{source_path}}/{{source_name}}"
}
}
},
{
"uid":"copy_task",
"kind":"copy",
"payload":{
"target":{
"location":{
"storage_provider":"s3",
"path":"{{destination_path}}"
},
"existing_files":"rename_new",
"file_pattern":"NTP_copy_{{source_name}}"
}
}
}
],
"connections":[
{
"from":[
{
"element":"sources"
}
],
"to":{
"success":[
{
"element":"copy_task"
}
]
}
}
]
}
}

Scenario: Wrong KID title attribute

  • Have a transcode profile that uses DRM and has {# entity.attributes['en-US']['KID'] #} in the KID template.
  • Choose a media attached to a title that does have a random string for the above attribute.
  • Run transcode using the transcode profile.
  • A popup will appear with the following error.

undefined variable: drm, line:1, col:1, line:13, col:24 There is data missing or invalid in the DRM encryption information request. - field 'KID' of content key #1 (with intended track type value: 'HD') with value 'be3c17': kid must be an UUID

Media

Example Media

Media

NTP Profile

Example NTP Profile
{
"definitions": {
"profile_name": "AN_11212_profile_",
"source_path": "{# file1.locator.dirname #}",
"source_name": "{# file1.locator.basename #}{# file1.locator.extname #}",
"output_basename": "{# entity.name #}",
"destination_path": "s3://tf-s3-ownzones-arrakis/testarrakis/deliverables/Native_outputs/{# 'now' | date: '%Y-%m-%d_%H-%M-%S' #}",
"transcode_preset": "medium",
"fmp4_segment_duration_sec": 6,
"credentials_key": "oz-dev-credentials",
"title_name": "{# drm.name #}"
},
"name": "{{profile_name}}: {{source_name}} - {{output_basename}}",
"priority": 100,
"task_tags": ["dev"],
"payload": {
"elements": [
{
"uid": "source_file",
"kind": "source",
"payload": {
"kind": "asset_url",
"payload": {
"storage_provider": "s3",
"url": "{{source_path}}/{{source_name}}"
}
}
},
{
"uid": "transcode",
"kind": "transcode",
"payload": {
"location": {
"storage_provider": "s3",
"path": "{{destination_path}}/mp4s"
},
"targets": [
{
"file_pattern": "{{output_basename}}_video_6500{default_extension}",
"existing_files": "rename_new",
"container": {
"kind": "fmp4",
"segment_duration": "{{fmp4_segment_duration_sec}}"
},
"video": {
"codec": "h264",
"bitrate_mode": "cbr",
"use_scene_detection": false,
"bitrate_kb": 6500,
"vbv_buffer_size_kb": 6500,
"height": 1080,
"par": "1:1",
"level": "3.1",
"profile": "high"
}
},
{
"file_pattern": "{{output_basename}}_video_5000{default_extension}",
"existing_files": "rename_new",
"container": {
"kind": "fmp4",
"segment_duration": "{{fmp4_segment_duration_sec}}"
},
"video": {
"codec": "h264",
"bitrate_mode": "cbr",
"use_scene_detection": false,
"bitrate_kb": 5000,
"vbv_buffer_size_kb": 5000,
"height": 720,
"par": "1:1",
"level": "3.1",
"profile": "high"
}
},
{
"file_pattern": "{{output_basename}}_audio_128{default_extension}",
"existing_files": "replace",
"container": {
"kind": "fmp4",
"segment_duration": "{{fmp4_segment_duration_sec}}"
},
"audio": [
{
"channels": 2,
"codec": "aac_lc",
"sample_rate": 48000,
"bitrate_kb": 128,
"layer_id": "audio_high"
}
]
}
]
}
},
{
"uid": "package_dash_cmaf",
"kind": "package",
"payload": {
"uid": "hls_and_dash_manifest",
"kind": [
"dash",
"hls"
],
"location": {
"storage_provider": "s3",
"path": "{{destination_path}}/cmaf/dash"
},
"attributes": [
{
"name": "ContentType",
"value": "application/x-mpegURL"
}
],
"dash": {
"file_pattern": "master_manifest.mpd"
},
"hls": {
"file_pattern": "master_manifest.m3u8"
},
"segmentation_mode": "fmp4",
"force_original_media": false,
"encryption": {
"enabled": true,
"schema": "mpeg-cbcs",
"drm": [
"playready",
"widevine"
],
"key_id": "{# drm.contentKeys['HD'].kid | replace: '-', '' #}",
"key": "{# drm.contentKeys['HD'].key | hex #}",
"playready_url": "https://pr-keyos.licensekeyserver.com/core/rightsmanager.asmx",
"playready_pssh": "{# drm.contentKeys['HD'].playreadyPssh | base64 #}",
"widevine_provider": "buydrmkeyos",
"clearkey_pssh_version": 1
}
}
}
],
"connections": [
{
"from": [
{
"element": "source_file"
}
],
"to": {
"success": [
{
"element": "transcode"
}
]
}
},
{
"from": [
{
"element": "transcode"
}
],
"to": {
"success": [
{
"element": "package_dash_cmaf"
}
]
}
}
]
},
"subscription_key": "char"
}

Scenario: Wrong DRM provider URL

  • Have a transcode profile that uses DRM, but the DRM provider has the wrong URL.
  • Choose a media.
  • Run transcode using the transcode profile.
  • A popup will appear with the following error.

undefined variable: drm, line:1, col:1, line:13, col:24 A valid DRM URL must be provided.

Media

Example Media

Media

NTP Profile

Example NTP Profile
{
"definitions": {
"profile_name": "AN_11218_profile_",
"source_path": "{# file1.locator.dirname #}",
"source_name": "{# file1.locator.basename #}{# file1.locator.extname #}",
"output_basename": "{# entity.name #}",
"destination_path": "s3://tf-s3-ownzones-arrakis/testarrakis/deliverables/Native_outputs/{# 'now' | date: '%Y-%m-%d_%H-%M-%S' #}",
"transcode_preset": "medium",
"fmp4_segment_duration_sec": 6,
"credentials_key": "oz-dev-credentials",
"title_name": "{# drm.name #}"
},
"name": "{{profile_name}}: {{source_name}} - {{output_basename}}",
"priority": 100,
"payload": {
"elements": [
{
"uid": "source_file",
"kind": "source",
"payload": {
"kind": "asset_url",
"payload": {
"storage_provider": "s3",
"url": "{{source_path}}/{{source_name}}"
}
}
},
{
"uid": "transcode",
"kind": "transcode",
"payload": {
"location": {
"storage_provider": "s3",
"path": "{{destination_path}}/mp4s"
},
"targets": [
{
"file_pattern": "{{output_basename}}_video_6500{default_extension}",
"existing_files": "rename_new",
"container": {
"kind": "fmp4",
"segment_duration": "{{fmp4_segment_duration_sec}}"
},
"video": {
"codec": "h264",
"bitrate_mode": "cbr",
"use_scene_detection": false,
"bitrate_kb": 6500,
"vbv_buffer_size_kb": 6500,
"height": 1080,
"par": "1:1",
"level": "3.1",
"profile": "high"
}
},
{
"file_pattern": "{{output_basename}}_video_5000{default_extension}",
"existing_files": "rename_new",
"container": {
"kind": "fmp4",
"segment_duration": "{{fmp4_segment_duration_sec}}"
},
"video": {
"codec": "h264",
"bitrate_mode": "cbr",
"use_scene_detection": false,
"bitrate_kb": 5000,
"vbv_buffer_size_kb": 5000,
"height": 720,
"par": "1:1",
"level": "3.1",
"profile": "high"
}
},
{
"file_pattern": "{{output_basename}}_audio_128{default_extension}",
"existing_files": "replace",
"container": {
"kind": "fmp4",
"segment_duration": "{{fmp4_segment_duration_sec}}"
},
"audio": [
{
"channels": 2,
"codec": "aac_lc",
"sample_rate": 48000,
"bitrate_kb": 128,
"layer_id": "audio_high"
}
]
}
]
}
},
{
"uid": "package_dash_cmaf",
"kind": "package",
"payload": {
"uid": "hls_and_dash_manifest",
"kind": [
"dash",
"hls"
],
"location": {
"storage_provider": "s3",
"path": "{{destination_path}}/cmaf/dash"
},
"attributes": [
{
"name": "ContentType",
"value": "application/x-mpegURL"
}
],
"dash": {
"file_pattern": "master_manifest.mpd"
},
"hls": {
"file_pattern": "master_manifest.m3u8"
},
"segmentation_mode": "fmp4",
"force_original_media": false,
"encryption": {
"enabled": true,
"schema": "mpeg-cbcs",
"drm": [
"playready",
"widevine"
],
"key_id": "{# drm.contentKeys['HD'].kid | replace: '-', '' #}",
"key": "{# drm.contentKeys['HD'].key | hex #}",
"playready_url": "https://pr-keyos.licensekeyserver.com/core/rightsmanager.asmx",
"playready_pssh": "{# drm.contentKeys['HD'].playreadyPssh | base64 #}",
"widevine_provider": "buydrmkeyos",
"clearkey_pssh_version": 1
}
}
}
],
"connections": [
{
"from": [
{
"element": "source_file"
}
],
"to": {
"success": [
{
"element": "transcode"
}
]
}
},
{
"from": [
{
"element": "transcode"
}
],
"to": {
"success": [
{
"element": "package_dash_cmaf"
}
]
}
}
]
},
"subscription_key": "char"
}

Scenario: Missing Watermark provider

  • Have a transcode profile that has watermark syntax in the template but the Watermark provider is disabled.
  • Choose a media.
  • Run transcode using the transcode profile.
  • A popup will appear with the following error.

undefined variable: watermark, line:1, col:1, line:93, col:41 The watermark provider has not been defined

Media

Example Media

Media

NTP Profile

Example NTP Profile
{
"definitions": {
"profile_name": "MOV File with Nagra watermarking",
"credentials_key": "oz-dev-credentials",
"source_path": "{# videoFile.locator.dirname #}/",
"source_name": "{# videoFile.locator.basename #}{# videoFile.locator.extname #}",
"destination_path":"s3://tf-s3-ownzones-arrakis/testarrakis/deliverables/nagra_tests/{# entity.id #}/{# videoFile.id #}",
"container_kind": "mov",
"output_basename": "nagra-test-clip-clipmark"
},
"name": "{{profile_name}}: {{source_name}}",
"payload": {
"elements": [
{
"uid": "sources",
"kind": "source",
"payload": {
"kind": "asset_complex",
"payload": {
"asset_versions": [
{
"version_uid": "audio_video",
"asset_components": [
{
"component_uid": "audio_video",
"kind": "name",
"name": "{{source_name}}",
"location": {
"storage_provider": "s3",
"path": "{{source_path}}",
"access": {
"credentials_key": "{{credentials_key}}"
}
},
"contents": [
{
"kind": "video"
},
{
"kind": "audio",
"map": [
{
"input": {
"track": 0,
"channel": 0
},
"output": {
"track": 0,
"channel": 0
}
},
{
"input": {
"track": 0,
"channel": 1
},
"output": {
"track": 0,
"channel": 1
}
}
]
}
]
}
]
}
]
}
}
},
{
"uid": "transcode_video",
"kind": "transcode",
"task": {
"tags": [],
"flags": {
"split_task": "smart"
}
},
"payload": {
"location": {
"storage_provider": "s3",
"path": "{{destination_path}}"
},
"watermarking": [
{
"kind": "nexguard_video",
"payload": {
"nexguard_integration_type": "plugin",
"watermark_preset": "{# watermark.preset #}",
"watermark_payload": {# watermark.payload #},
"watermark_strength": "medium",
"license_text": "{# watermark.license #}"
}
}
],
"targets": [
{
"file_pattern": "nagra 1 out_{{output_basename}}{default_extension}",
"existing_files": "replace",
"container": {
"kind": "{{container_kind}}"
},
"video": {
"enabled": true,
"codec": "h264",
"height": 480,
"bitrate_mode": "cbr",
"use_scene_detection": false,
"bitrate_kb": 1200,
"profile": "main",
"closed_captions": {
"suppress": true
},
"filters": [
{
"kind": "deinterlace",
"payload": {
"interlace_mode": "auto"
}
}
]
},
"audio": [
{
"codec": "aac_lc",
"channels": 2,
"bitrate_kb": 64,
"bitrate_mode": "cbr",
"sample_rate": 48000,
"sample_size": 16,
"language": "english",
"source": [
{
"track": 0
}
]
}
]
}
]
}
}
],
"connections": [
{
"from": [
{
"element": "sources"
}
],
"to": {
"success": [
{
"element": "transcode_video"
}
]
}
}
]
}
}

Discover Transcode

Before this development, if an error that was thrown when starting transcode, it was caught and shown in a popup. This is unchanged.

UI behavior:

  • the errors is always on one line.

Examples of various scenarios


Scenario: Missing Watermark provider

  • Have a transcode profile that has watermark syntax in the template but the Watermark provider is disabled.
  • Have a title with file attribute that has a transcode map pointing to the above transcode profile.
  • Run transcode from Discover for the media of the file attribute.
  • A popup will appear with the following error.

Zypline api request failed: undefined variable: watermark, line:1, col:1, line:93, col:41 The watermark provider has not been defined.

Title

Example Title

Title

NTP Profile

Example NTP Profile
{
"definitions": {
"profile_name": "MOV File with Nagra watermarking",
"credentials_key": "oz-dev-credentials",
"source_path": "{# videoFile.locator.dirname #}/",
"source_name": "{# videoFile.locator.basename #}{# videoFile.locator.extname #}",
"destination_path":"s3://tf-s3-ownzones-arrakis/testarrakis/deliverables/nagra_tests/{# entity.id #}/{# videoFile.id #}",
"container_kind": "mov",
"output_basename": "nagra-test-clip-clipmark"
},
"name": "{{profile_name}}: {{source_name}}",
"payload": {
"elements": [
{
"uid": "sources",
"kind": "source",
"payload": {
"kind": "asset_complex",
"payload": {
"asset_versions": [
{
"version_uid": "audio_video",
"asset_components": [
{
"component_uid": "audio_video",
"kind": "name",
"name": "{{source_name}}",
"location": {
"storage_provider": "s3",
"path": "{{source_path}}",
"access": {
"credentials_key": "{{credentials_key}}"
}
},
"contents": [
{
"kind": "video"
},
{
"kind": "audio",
"map": [
{
"input": {
"track": 0,
"channel": 0
},
"output": {
"track": 0,
"channel": 0
}
},
{
"input": {
"track": 0,
"channel": 1
},
"output": {
"track": 0,
"channel": 1
}
}
]
}
]
}
]
}
]
}
}
},
{
"uid": "transcode_video",
"kind": "transcode",
"task": {
"tags": [],
"flags": {
"split_task": "smart"
}
},
"payload": {
"location": {
"storage_provider": "s3",
"path": "{{destination_path}}"
},
"watermarking": [
{
"kind": "nexguard_video",
"payload": {
"nexguard_integration_type": "plugin",
"watermark_preset": "{# watermark.preset #}",
"watermark_payload": {# watermark.payload #},
"watermark_strength": "medium",
"license_text": "{# watermark.license #}"
}
}
],
"targets": [
{
"file_pattern": "nagra 1 out_{{output_basename}}{default_extension}",
"existing_files": "replace",
"container": {
"kind": "{{container_kind}}"
},
"video": {
"enabled": true,
"codec": "h264",
"height": 480,
"bitrate_mode": "cbr",
"use_scene_detection": false,
"bitrate_kb": 1200,
"profile": "main",
"closed_captions": {
"suppress": true
},
"filters": [
{
"kind": "deinterlace",
"payload": {
"interlace_mode": "auto"
}
}
]
},
"audio": [
{
"codec": "aac_lc",
"channels": 2,
"bitrate_kb": 64,
"bitrate_mode": "cbr",
"sample_rate": 48000,
"sample_size": 16,
"language": "english",
"source": [
{
"track": 0
}
]
}
]
}
]
}
}
],
"connections": [
{
"from": [
{
"element": "sources"
}
],
"to": {
"success": [
{
"element": "transcode_video"
}
]
}
}
]
}
}

Before / After scenarios

Below are different scenarios with error messages. How they were before & how they are now. Both at job validation and job runtime.

Description of scenarios can be found in previous chapters.


Scenario: Missing title attribute

Before - Job Validation

Before - Job Validation

After - Job Validation

After - Job Validation

Before - Job Runtime

Before - Job Runtime

After - Job Runtime

After - Job Runtime


Scenario: Wrong KID title attribute

Before - Job Validation

Before - Job Validation

After - Job Validation

After - Job Validation

Before - Job Runtime

Before - Job Runtime

After - Job Runtime

After - Job Validation


Scenario: Wrong DRM provider URL

Before - Job Validation

Before - Job Validation

After - Job Validation

After - Job Validation

Before - Job Runtime

Before - Job Runtime

After - Job Runtime

After - Job Runtime


Scenario: Missing Watermark provider

Before - Quick Transcode

Before - Quick Transcode

After - Quick Transcode

After - Quick Transcode


Scenario: Missing Track tag

Before - Job Validation

Before - Job Validation

After - Job Validation

After - Job Validation

Before - Job Runtime

Before - Job Validation

After - Job Runtime

After - Job Validation


Template TroubleshootingTemplates Syntax Cheat Sheet