The IMAX Stream On-Demand Platform is the enterprise software platform that delivers IMAX’s award-winning technology for file-based (VOD) workflows, seamlessly enabling access to two powerful products, StreamAware and StreamSmart.

IMAX StreamAware™ On-Demand

StreamAware On-Demand is designed for video service providers to streamline quality assurance and control. The software ensures high-quality video by providing complete visibility across file-based workflows, while also enabling automated quality checks to verify videos meet delivery requirements. It uses the Emmy® award-winning IMAX VisionScience™ technology to provide a single, objective metric to monitor quality across the entire media supply chain and provide clear insights into its performance. The software also includes industry standard and customizable quality checks to automate video, audio, and metadata compliance. This results is complete visibility of quality across file-based workflows, and enables automation of tasks that typically require the human eye, improving efficiency and reducing the margin for human error to ensure video consistently meets the highest standards.

For more details, please visit: https://www.imax.com/sct/product/streamaware-on-demand

IMAX StreamSmart™ On-Demand

StreamSmart On-Demand overlays on your existing encoding workflow and uses the most accurate measure of video quality, IMAX VisionScience™, to retain the same visual quality of experience, while delivering a significant reduction in delivery costs. IMAX’s unique VisionScience perceptual quality measurement technology "sees” video the way humans do and takes advantage of opportunities to decrease bits in a way that humans will not notice, and which other methods cannot match. StreamSmart uses this quality-measurement approach to analyze every frame of a video and optimize it for best picture quality and compression efficiency, guaranteeing an optimal viewer experience while maximizing bitrate savings, typically 15% above what top-of-the-line content-aware encoders are already delivering.

For more details, please visit: https://www.imax.com/sct/product/streamsmart-on-demand

The diagram below shows the system architecture of the IMAX Stream On-Demand Platform, including the components of the system, the interactions between them, and where and how a user or an automated system interacts with the solution.

• The IMAX Stream On-Demand Platform is the customer-deployed portion of the system and is designed as a set of cooperating software (Docker) containers that are orchestrated by Kubernetes.

  • Stream On-Demand API & Services is a set of always-running containers that provide the following:

    • A REST API known as the Stream On-Demand Platform REST API that can be used to:

      • submit new StreamAware On-Demand analysis (QC) requests,
      • submit new StreamSmart On-Demand encoding optimization requests,
      • visually inspect the results by extracting frames and a number of VisionScience maps (i.e. quality maps, banding maps, color volume difference maps) and
      • query and configure the system.
        
        
      Important

      The Stream On-Demand Platform REST API provides an OpenAPI Specification (OAS) that can be used to generate language-specific clients for those interested in jump-starting an integration into their production workflow. Both Swagger Codegen and OpenAPITools openapi-generator are examples of tools that can be used for this purpose. Additionally, any laguage that has support for sending and receiving HTTP requests/responses in JSON format can be also be used with minimal effort.

      Please contact your IMAX representative if you require any guidance.

    • Services for interacting with Kubernetes to take advantage of its various deployment, scaling, scheduling and management features

    • Services for interacting with various cloud API services (e.g. AWS S3)
      
      

    For fixed scale deployments of the IMAX Stream On-Demand Platform, there is only ever one running instance of each container as the scalability needs here are expected to be relatively low. Elastic scale deployments of the cluster, by contrast, will elastically scale these containers to meet demand.
    
    

  • The Video Processors are Docker containers that perform the video processing (analysis in the case of StreamAware, encoding (or interfacing with encoders) and optimization in the case of StreamSmart), generating the metadata and metrics that are securely streamed to Insights. Video Processors are configured, deployed and managed by the Stream On-Demand Platform REST API in response to POST requests that invoke StreamAware or StreamSmart.

    • Each Video Processor container is configured to request the compute resources (CPU/RAM) it needs in order to succesfully perform its analysis, given the properties of the assets being processed and the job that has been requested.
    • Video Processor containers never stream any part of the video to Insights and, as such, no content (i.e. video, frames or images) ever leaves the customer’s environment/control.
      Note

      Upon request, IMAX will provide examples of the metrics and metadata streamed to Insights.

    • For fixed scale deployments of the IMAX Stream On-Demand Platform, the number of concurrently running Video Processors will scale but only to a fixed number, as the cluster itself is constrained by the underlying properties of the virtual machine (i.e. OVA/QEMU) or EC2 instance (i.e. AMI). Elastic scale deployments of the cluster, by contrast, can elastically scale without an upper bound, if so desired, in order to meet even the most demanding workloads.
    • For StreamSmart jobs, the Video Processor performs the encoding when FFmpeg is used, and controls external encoding services (e.g. AWS Elemental MediaConvert) when they are used.
  
  

• Insights is IMAX’s multi-tenant cloud-based data platform that leverages the power of our own VisionScience algorithms, along with leading business intelligence services, to generate, store, and present, meaningful representations of the analysis, encoding and optimization results. Some of the key features of Insights include:

  • Performant, secure, and highly-available data storage that scales automatically
  • Insights Stream On-Demand Web UI which can be used to:
    • submit StreamAware and StreamEnhance jobs into the Stream On-Demand platform
    • view and interact with a summary/status of all pending, running and completed analyses
    • review results
    • perform frame-level inspections of your content, including image and map comparisons
  • Insights REST API which can be used to automate job status and progress tracking, and retrieve results, including:
    • metrics and measurements across a variety of time granularities including: per-frame, per-second (1s, 2s, 5s, etc.), per-asset
    • StreamAware QC overall result and specific failures
    • StreamSmart perceptual quality scores and bitrate savings
  • Insights dashboards and reports which are customizable content displays of analysis results that can be used interactively in the Web UI and exported in various archival formats such as PDF, PNG/JPG, Excel, JSON, etc.
    
    

• The Automation computer/laptop icon above illustrates that the full StreamSmart system is designed to be used in a headless manner (i.e. through API/programmatic access) and that all key features are exposed through their respective REST APIs.
  
  

• The user/human icon and its connections above illusrate that:

  • to support those integrating StreamAware and StreamSmart into their production workflows, IMAX has provided Postman collections with numerous REST API examples that can be imported into the Postman UI in order to become familiarized with the API request/response structures and
  • the Insights Stream On-Demand Web UI can equally be used to interact graphically with the system by submitting the underlying API requests on the user’s behalf.

The IMAX Stream On-Demand Platform can be deployed in the following ways:

Fixed scale:

• As an EC2 instance on AWS using the IMAX Stream On-Demand Platform AMI
• As a virtual machine instance using an OVA or QEMU image

Elastic/dynamic scale:

• On AWS EKS as a hosted SaaS instance (deployed in an IMAX account by IMAX)
• On AWS EKS as a managed service (deployed in the customer account by IMAX)

For details, refer to the Deployment Guides section.

The “On-Demand” web application for the IMAX Stream On-Demand Platform can be found at https://insights.sct.imax.com/extensions/ondemand::app, or within Insights, by selecting On-Demand under the Applications menu item.

If you don’t already have an Insights account, contact your IMAX Customer Success representative.

Navigation within the On-Demand UI looks and functions as follows:

• Home - Loads the default page of the UI (Status), which is highlighted in the image.
• Analyze - Configure and submit a StreamAware On-Demand job.
• Optimize - Configure and submit a StreamSmart On-Demand job.
• Status - View the status, progress, and a summary of the results (once completed) of both StreamAware and StreamSmart jobs.
• Results - View the detailed results of StreamSmart and StreamSmart jobs. Typically navigated to by drilling into a job from the Status page, but can also be navigated to directly, in which case job selection will be required.
• Host selection - This drop-down menu is used to select the Stream On-Demand Platform instance to connect to and control. The colored indicator displays the connection status: green if the app is connected to and can communicate with the host, red if it cannot.
• Configuration - Configuration for Analysis and Optimization Templates (more details below) and host management (i.e. adding and deleting Stream On-Demand Platform instances).

The Insights menu can be accessed from the top left (using the “hamburger”), and user account settings can be accessed at the top right.

The majority of the funcionality of the UI requires a connection to the selected host. A dialog will appear to assist you in establishing the connection when required. Things to keep in mind:

• The system on which the browser/web UI is running requires network connectivity to TCP port 443 of the selected host. Depending on how and where the the instance is deployed, this may require a connection to a VPN or access from a specific allow-listed source IP address.
• If the Stream On-Demand Platform instance is deployed with a self-signed TLS certificate, you will need to accept the self-signed certificate before a connection can be established.

When using the UI for the first time, you will be prompted to add your host(s) if one has not already been configured for you.

To analyze one or more video files with StreamAware, navigate to the Analyze page:

The Analyze page allows you to configure and submit a StreamAware analysis, as defined by the Stream On-Demand REST API, allowing you to unlock the benefits of the features described in Terminology and Features. The layout of the page is as follows:

• Title - A title must be specified for each analysis. You can use the name of the asset that is being analyzed (e.g. “Big Buck Bunny”) or a description of the analysis that is being performed (e.g. “Perceptual quality analysis of Big Buck Bunny” or “QC of Big Buck Bunny”).
• Template - An analysis can be configured by loading the configuration from a saved template. Refer to Analysis and Optimization Templates for more details.
• Configuration - Expanding this section allows you to configure:
  • Frames To Process - The number of frames of video to process (the default is to process all frames)
  • Viewing Environment - Configures the viewing environment (viewer-type and device) for the device and viewer-type adaptive metrics like the IMAX ViewerScore and IMAX Encoder Performance Score.
  • Features - Specific metrics and features, such as banding detection, can be enabled and disabled in this section.
  • Content Layout Detection - Features related to content layout detection can be enabled and disabled in this section.
  • Quality Checks - QC features like freeze frame detection and color bar detection can be enabled and configured in this section. The checks conifgured here apply to all assets in the analayis. Asset-specific Quality Checks are configured in the Assets section.
• Assets - This is where the content that is to be analyzed is specified. An analysis must have at least one “test” file. Optionally, a reference file and additional test files can be added, using the Add Reference Asset and Add Test Asset buttons. If no reference file is specified, a no-reference analysis will be performed. If a reference file is specified, a full-reference analysis will be performed for each of the test files specified. For each asset, the following options can be specified:
  • Asset Format - The asset format (e.g. video file, image sequence etc.) must be selected. Some additional configuration must be provided for certain asset formats.
  • Start Frame - This setting can be used to skip a specific number of frames at the beginning of the asset.
  • Dynamic Range - StreamAware can automatically detect the dynamic range for most assets, which it will attempt to do when this is set to Auto Detect. To force the analysis into a specific mode, choose SDR or HDR.
  • Asset Location - Specifies where the asset is located (AWS S3 or Kubernetes PVC), the URI/path, and how to authenticate (if applicable). S3 buckets can be browsed using the Choose button.
  • Region of Interest - Limits the analysis to a particular region of interest. Disabled by default (entire frames are analyzed).
  • Audio - Enables/disables audio loudness measurements, and used to enable and configure audio Quality Checks, such as audio silence detection, clipping detection, and audio phase mismatch.
  • Sidecar - Add audio and Dolby Vision Metadata sidecar files to the analysis.
  • Asset Quality Checks - Enable and configure asset-specific Quality Checks.

Once an analysis has been fully configured, use the Submit Analysis button to submit the job to the host that is selected in the host selection dropdown. If the job is accepted by the host, the Analysis Submission Result page will load. It provides a summary of the analysis configuration, and most importantly, shows the Analysis ID and presents a Status button which can be used to navigate to the Status page filtered on the newly submitted analysis. Refer to Monitoring Progress for more details.

To optimize an encode or an encoding ladder with StreamSmart, navigate to the Optimize page:

The Optimize page allows you to configure and submit a StreamSmart optimization, as defined by the Stream On-Demand REST API. The layout of the top of the page is as follows:

• Title - A title must be specified for each optimization. You can use the name of the asset that is being analyzed (e.g. “Big Buck Bunny”) or a description of the encode that is being performed (e.g. “Experiment with higher CRF values”).
• Template - An optimization can be configured by loading the configuration from a saved template. Refer to Analysis and Optimization Templates for more details.
• Encoder - The underlying encoder must be specified. The encoder that is selected determines how the encoder configuration is provided further down the page (see below).
• Input File - Selects the storage type and URI/path of the input/source file to encode. The storage types that are supported is dependent on the encoder that is selected (e.g. if MediaConvert is selected only S3 can be specified).
• Output Location - Specifies the storage type and path/folder where the output file(s) are placed. The actual names of the output files is specified later.
• Prune redundant renditions - Encoding optimization can sometimes eliminate the need for certain renditions, when optimization places them too close together. Enabling this feature will prune these redundant renditions out of the result, producing not only optimized encodes, but an optimized ladder.

How the encoder, ladder, and output file configuration is specified depends on the encoder that is selected.

FFmpeg

FFmpeg optimizations are configured by adding renditions, and specifying the FFmpeg command and output file suffix and extension for each one.

In this example, two renditions have been configured, optimization has been enabled for both of them, and the FFmpeg commands for each have been specified.

When using the Web UI, output files are named by appending the suffix that is configured in the Output File Name field to the input file name. An extension is also required (.mp4 in the example) and will be used to determine the type of output file that is produced.

To add renditions, click on the Add Rendition button. Once the configuration has been completed, click on the Submit Optimization button to submit the job.

AWS Elemental MediaConvert

MediaConvert optimizations are configured by loading your MediaConvert job configuration in JSON format into the Encoder Configuration field. The list of renditions is derived from this configuration, and optimization can be enabled/disabled for each one in the list that appears below.

In this example, the MediaConvert job configuration defines a single rendition, for which optimization is enabled.

Once the configuration has been completed, click on the Submit Optimization button to submit the job. If the job is accepted by the host, the Submission Result page will load. It provides a summary of the optimization configuration, and most importantly, shows the Analysis ID and presents a Status button which can be used to navigate to the Status page filtered on the newly submitted analysis. Refer to Monitoring Progress for more details.

To view the status of a job and monitor progress, click on the Status button on the job summary after submitting a new job, or navigate to the Status page:

The first element on the page is the job filter. If you navigated to the page from the job submission summary, the filter will be configured with your analysis ID. If you navigated directly to the page, the filter will be empty and all jobs will be displayed, sorted in descending order by job submission time (most recently submitted first). Filter elements can be deleted and added, including filtering by submission date, job title, analysis type (StreamAware and StreamSmart), and analysis ID.

Below the filter are some tiles that summarize the results. Clicking on the tiles will further filter the job list (e.g. clicking on the IN PROGRESS tile will filter the results down to just the currently in-progress jobs). Clicking again removes the additional filter. The following controls can be used to manage the filter and what is displayed on the page:

1. Shows/hides the filter section of the page
2. Shows/hides the summary tiles
3. Clears all elements from the filter field
4. Additional configuration options:
   • Save filter - Filters can be saved and loaded at a later time
   • Manage filters - Load a filter, specify your default filter, delete a filter
   • Share URL - copies a URL to your clipboard that can be shared and will load the Status page with the current filter
   • The slider at the bottom of the menu adjusts how often the results auto-refresh
5. Click here to add an additional filter element
6. Run the query or manually refresh the results

The jobs that match the filters are displayed at the bottom of the page. This example shows the results of filtering on a specific analysis ID:

The overall job status and progress is displayed in the STATUS column. Each row in the job list can be expanded to see the status for each file in the job:

Once a job has completed, the Status page will display a summary of the results for a job when the entry for the job is expanded. Here is an example of the summary for a StreamAware job:

1. Clicking almost anywhere in the empty space of the main section of a job will will load the detailed results for the job in the Results page (see below for details). Clicking on the analysis ID will copy it to the clipboard.
2. Dashboards are accessible via this drop-down menu. Some dashboards are provided by default, but custom dashboards specified to your use-case and needs can easily be created and linked to your account.
3. Expand/collapse the file view.
4. Launch the frame and map viewer for this file. This can be used to see and compare frames of video and quality maps at specific points in time (see below for details).
5. Click here to look at a detailed log for the analysis of the file.
6. There are options under this drop-down menu for cancelling (if the job is still in progress), deleting, re-submitting, and editing and re-summitting a job.

The scores displayed for the files are asset scores, refer to VisionScience Asset Scores for details.

The Test IDs for StreamAware jobs follow the following convention:

• No-Reference Analysis - the files have single-digit Test IDs, starting at 1.
• Full-Reference Analysis - the reference file has Test ID = 1, and the test files (the files being compared to the reference) have two-digit (separated by a dash) Test IDs, where hte first digit is the reference ID, and the second digit represents the test (e.g. 1-1, 1-2). In the example above, it is a reference-based analysis, 1 is the reference, and 1-1, 1-2, 1-3, and 1-4 are the files being tested.

Here is an example of the summary for a StreamSmart job:

These summarized StreamSmart results are similar to the results for StreamAware jobs except for:

• The analysis type is StreamSmart instead of StreamAware
• The device column has been removed.
• A savings column has been added, which displays the overall bitrate savings of the optimized encode when compared to the anchor (original unoptimized encode).
• The Test IDs are different:
  • Input - The input/source file.
  • Anchor <N> - The original unoptimized encode for the Nth rendition. This is the encode you would get if you ran the FFmpeg command or MediaConvert config directly without StreamSmart. In the example above, Anchor 1 is the unoptimized encode for the first rendition, and Anchor 2 is the unoptimized encode for the second rendition.
  • Optimized <N> - The optimized encode for the Nth rendition. In the example above, Optimized 1 is the optimized encode for the first rendition, and Optimized 2 is the optimized encode for the second rendition.

As noted above, clicking in the whitespace of the overall section of the job loads the Results page, which shows a much more detailed view of the results, including time-series views of all of the metrics (including IMAX ViewerScore, Encoder Performance Score, Banding Score, and bitrate), and QC results if Quality Checks were enabled.

1. The overall result - if any Quality Checks were enabled and one or more failed, the overall result will be Fail. If Quality Checks were not enabled, or if they all passed, the overall result will be Pass.
2. If the job was configured with multiple viewer types and/or devices, the pair that determines which results are displayed can be selected here.
3. The filter icon will pop up a dialog that can be used to search for and load the results of a different job.
4. The gear icon allows you to configure some settings related to how the time-series data is displayed.
5. The summary section, similar to the Status page, with additional details found by expanding the individual files:
   • Quality Checks - A list of Quality Check failures for the file
   • Video Metrics - Additional video metrics, such as Color Gamut and Color Volume Difference (if enabled).
   • Video Metadata - Information about the video file, such as the frame rate, pixel format, and scan type.
   • Audio - The list of audio streams, including metadata, and audio loudness measurements (if enabled).
6. This section is displayed if there were any Quality Check failures. It shows the failures, and can be toggled between two modes:
   • Summary - This view summarizes the Quality Check results by file. Each file can be expanded to show a summary of the checks that failed for that file, including the total number of failures, affected frames and duration. When a file is selected, the time-series view shows all of the segments in the file that have failures. When a specific type of failure is selected, the graph shows the sections that are affected by that specific type. Clicking on the highlighted section of the graph loads the Viewer for the first frame of the selected range.
   • Detailed - This view shows the full list of all Quality Check failures, including the start and end time for each one. When a failure is selected, the graph highlights the section of video affected by the failure, and plots the associated metric (if applicable). Clicking on the highlighted section of the graph loads the Viewer for the first frame of the selected range.
7. This section displays time-series views of the metrics that were collected by the job.
   • Each file is represented by a different color, and the data for files can be hidden by unselecting the checkbox on the left side of the summary section (5).
   • Different metrics can be selected by choosing one of the tabs at the top of this section, and tabs can be hidden by clicking on the gear icon.
   • You can switch between a tabbed view and a stacked view by clicking on the expand/collapse icon.
   • Zoom into a section of a graph by clicking and dragging across the area of interest.
   • For StreamAware jobs, you can launch into the Viewer by clicking on a point on a graph. The Viewer allows you to view and compare frames and quality maps (see below for details).

As noted above, in most cases, clicking on a point on time-series graph will load the Viewer, which can be used to display and compare frames and quality maps.

In the example above, clicking on this point on the IMAX Encoder Performance Score graph would load frames for that point in time (which looks like a problem area that needs to be investigated further) into the Viewer:

1. Selects the file that is displayed on the left of the hairline. If the same file is selected for the left side and the right side (2), the hairline will not be displayed.
2. Selects the file that is displayed on the right of the hairline. If the same file is selected for the left side (1) and the right side, the hairline will not be displayed.
3. Downloads the frame or map that is currently displayed for the file selected on the left (1).
4. Downloads the frame or map that is currently displayed for the file selected on the right (2).
5. Selects the image that is displayed for the file selected on the left: frame, quality map, banding map, color volume difference map.
6. Selects the image that is displayed for the file selected on the right: frame, quality map, banding map, color volume difference map.
7. When different files or images are displayed for the left and right sides, the hairline can be used and moved to compare them.
8. Displays the current frame number and can be used to move backwards and forwards through the frames.
9. The page can be exported as a self contained HTML page in a zip file, that can be shared with people that do not have access to Insights. The exported page will display what is currently visible with a functional hairline.
10. Toggles the display of the metric graphs, which can be used to see where the currently visible frame/map sits within the asset. Clicking on a new point on the graph will move to that point in time.
11. Time mode settings.

Analysis and Optimization Templates can be used to streamline the submission of StreamAware and StreamSmart jobs through the web UI. With templates, you can save commonly used configurations, and then load them on the Ananlyze and Optimize pages. The most commonly used configurations can be saved as defaults and will load automatically each time you start a new configuration.

New templates can be created in one of two ways:

1. Templates can be managed by clicking on Analysis Templates or Optimization Templates after clicking on the gear icon on the right side of the top menu:

2. New templates can be created by clicking on the New Template button at the top right of the Analyze and Optimize pages:

In the following Optimization Template example:

• FFmpeg is selected as the encoder.
• The Input File is left blank, which means it will not be filled in when the template is loaded.
• The Output Location is specified.
• Two renditions are configured and the output file suffixes are specified.

When this template is loaded, the only thing you would need to configure before submitting the job would be input file information.

A template can be set as the default template, which will load it by default when a new job is configured. To make a template the default, navigate to Analysis Templates or Optimization Templates under the gear menu, click on the three dots next to the template you want to make the default, and choose Set as default:

To load a template, select it from the Template dropdown on the Analyze or Optimize page:

Let’s walk through an example of using the Stream On-Demand Platform API to sumbit a new analysis to the system’s StreamAware component, along with the related JSON requests and responses.

In this example, we are going to use the StreamAware /analyses endpoint to perform a no-reference analysis of a single video asset.

{
  "content": {
    "title": "NR Analysis With XVS Score Check"
  },
  "subjectAssets": [
    {
      "name": "dog_running.mp4",
      "path": "royalty_free/dog_running/source",
      "storageLocation": {
        "type": "S3",
        "name": "video-files",
        "credentials": {
          "useAssumedIAMRole": true
        }
      },
      "qualityCheckConfig": {
        "scoreChecks": [
          {
            "metric": "SVS",
            "threshold": 85,
            "durationSeconds": 5,
            "viewingEnvironmentIndex": 0
          }
        ]
      }
    }
  ],
  "analyzerConfig": {
    "viewingEnvironments": [
      {
        "device": {
          "name": "oled65c9pua"
        },
        "viewerType": "EXPERT"
      }
    ]
  }
}

Notice in the JSON request body above that:

• our asset lives in an S3 bucket and we are relying on IAM roles to provide read-only access to the bucket,
• we are configuring the On-Demand Analyzer to target the LG OLED65C9PUA device from an expert viewer’s perspective and
• we are configuring a score check on our asset that will consider the video to have failed if there is any period of 5 or more seconds where the IMAX ViewerScore (XVS) for our selected device falls below 85.
  
  

A successful response from the Stream On-Demand Platform API would look something like the following:

{
  "submittedAnalyses": [
    {
      "analyzerConfig": {
        "viewingEnvironments": [
          {
            "device": {
              "name": "oled65c9pua"
            },
            "viewerType": "EXPERT"
          }
        ]
      },
      "id": "13afcc15-6808-4819-9cd3-fde950dda0f2",
      "subjectAsset": {
        "content": {
          "title": "NR Analysis With XVS Score Check"
        },
        "hdr": false,
        "name": "dog_running.mp4",
        "path": "royalty_free/dog_running/source",
        "qualityCheckConfig": {
          "scoreChecks": [
            {
              "durationSeconds": 5,
              "metric": "SVS",
              "skipEnd": 0,
              "skipStart": 0,
              "threshold": 85,
              "viewingEnvironmentIndex": 0
            }
          ]
        },
        "storageLocation": {
          "name": "video-files",
          "type": "S3"
        }
      },
      "submissionTimestamp": "2023-08-08T17:53:40.969Z",
      "testId": "1"
    }
  ]
}

Notice in the response above the line "id": "13afcc15-6808-4819-9cd3-fde950dda0f2" that reports back the analysis UUID which is a required key for fetching the associated results from Insights.

Referring back to the system architecture, at this point the Stream On-Demand API & Services have:

• validated the POST Analysis Request,
• ensured that the asset is present and accessible,
• inspected the properties of the underlying asset to determine the compute resources needed to analyze the asset,
• scheduled a new Analyzer container to process the video file and
• prepared and returned a response to the caller.
  
  

Once the Analyzer container starts:

• the Analyzer analyzes the video asset and streams the IMAX VisionScience measurements and metrics to Inisghts (Note: no part of the video itself is streamed to Insights),
• Insights applies additional viewer intelligence algorithms to the incoming streams before storing the data and,
• upon completion of the analysis, acknowledgements are recognized by both sides and the Analyzer container is destroyed (i.e. elastic scaling) in order to release the compute resources being used by the IMAX Stream On-Demand Platform.
  
  

Let’s walk through an example of using the Stream On-Demand Platform API to sumbit a new optimization to the system’s StreamSmart component, along with the related JSON requests and responses.

In this example, we are going to use the StreamSmart /optimizations endpoint to perform an optimization of a video asset using the FFmpeg encoder.

{
  "content": {
    "title": "Example Title"
  },
  "input": {
    "assetUri": "s3://videos/examples/example_source_file.mp4"
  },
  "encoderConfig": {
    "type": "FFmpegConfig",
    "encodes": [
      {
        "command": [
          "ffmpeg -i {INPUT_LOCATION} -c:v libx264 -x264-params \"ref=3:bframes=3:b_adapt=2:keyint=48:keyint_min=48:scenecut=0\" -profile:v high -level:v 4.1 -preset slow -b:v 4500k -maxrate 4500k -bufsize 6000k -an {OUTPUT_LOCATION}"
        ],
        "outputLocation": {
          "assetUri": "s3://videos/examples/output/encoded_video.mp4"
        }
      }
    ]
  }
}

A successful response from the Stream On-Demand Platform API would look something like the following:

{
    "uuid": "5fb38092-f787-4fc9-9789-6b0834cba832",
    "organization": "IMAX",
    "site": "StreamSmart",
    "status": "Created",
    "config": {
        "content": {
            "title": "Example Title"
        },
        "encoderConfig": {
            "encodes": [
                {
                    "command": [
                        "ffmpeg -i {INPUT_LOCATION} -c:v libx264 -x264-params ref=3:bframes=3:b_adapt=2:keyint=48:keyint_min=48:scenecut=0 -profile:v high -level:v 4.1 -preset slow -b:v 4500k -maxrate 4500k -bufsize 6000k -an {OUTPUT_LOCATION}"
                    ],
                    "outputLocation": {
                        "name": "examples/output/encoded_video.mp4",
                        "storageLocation": {
                            "name": "videos",
                            "type": "S3"
                        }
                    }
                }
            ],
            "type": "FFmpegConfig"
        },
        "input": {
            "name": "examples/example_source_file.mp4",
            "storageLocation": {
                "name": "videos",
                "type": "S3"
            }
        }
    }
}

Notice in the response above the line "id": "5fb38092-f787-4fc9-9789-6b0834cba832" that reports back the optimization UUID which is a required key for fetching the associated results from Insights.

Alternatively, you can submit an optimization that uses AWS Elemental MediaConvert (EMC) using an example like the following:

{
  "content": {
    "title": "Example Title"
  },
  "encoderConfig": {
    "type": "EMCConfig",
    "config": {
      "JobTemplate": "",
      "Queue": "arn:aws:mediaconvert:us-east-1:315835334412:queues/Default",
      "UserMetadata": {},
      "Role": "arn:aws:iam::315835334412:role/mediaconvert-optimizer",
      "Settings": {
        "OutputGroups": [
          {
            "CustomName": "top-profile-encode",
            "Name": "CMAF",
            "Outputs": [
              {
                "ContainerSettings": {
                  "Container": "CMFC"
                },
                "VideoDescription": {
                  "Width": 1920,
                  "ScalingBehavior": "STRETCH_TO_OUTPUT",
                  "Height": 1080,
                  "TimecodeInsertion": "DISABLED",
                  "AntiAlias": "ENABLED",
                  "Sharpness": 50,
                  "CodecSettings": {
                    "Codec": "H_264",
                    "H264Settings": {
                      "InterlaceMode": "PROGRESSIVE",
                      "NumberReferenceFrames": 3,
                      "Syntax": "DEFAULT",
                      "Softness": 0,
                      "GopClosedCadence": 1,
                      "GopSize": 2,
                      "Slices": 1,
                      "GopBReference": "ENABLED",
                      "HrdBufferSize": 16000000,
                      "MaxBitrate": 8000000,
                      "EntropyEncoding": "CABAC",
                      "RateControlMode": "QVBR",
                      "QvbrSettings": {
                        "QvbrQualityLevel": 9
                      },
                      "CodecProfile": "HIGH",
                      "MinIInterval": 0,
                      "AdaptiveQuantization": "AUTO",
                      "CodecLevel": "AUTO",
                      "SceneChangeDetect": "ENABLED",
                      "QualityTuningLevel": "SINGLE_PASS",
                      "UnregisteredSeiTimecode": "DISABLED",
                      "GopSizeUnits": "SECONDS",
                      "ParControl": "INITIALIZE_FROM_SOURCE",
                      "NumberBFramesBetweenReferenceFrames": 3,
                      "RepeatPps": "DISABLED",
                      "DynamicSubGop": "ADAPTIVE"
                    }
                  }
                },
                "NameModifier": "_8Mbps"
              }
            ],
            "OutputGroupSettings": {
              "Type": "CMAF_GROUP_SETTINGS",
              "CmafGroupSettings": {
                "TargetDurationCompatibilityMode": "SPEC_COMPLIANT",
                "WriteHlsManifest": "ENABLED",
                "WriteDashManifest": "ENABLED",
                "SegmentLength": 4,
                "Destination": "s3://s3-bucket/destination/path/",
                "FragmentLength": 2,
                "SegmentControl": "SEGMENTED_FILES",
                "WriteSegmentTimelineInRepresentation": "ENABLED",
                "ManifestDurationFormat": "FLOATING_POINT",
                "StreamInfResolution": "INCLUDE"
              }
            }
          }
        ],
        "Inputs": [
          {
            "AudioSelectors": {
              "Audio Selector 1": {
                "DefaultSelection": "DEFAULT"
              }
            },
            "VideoSelector": {
              "ColorSpace": "FOLLOW",
              "Rotate": "DEGREE_0",
              "AlphaBehavior": "DISCARD"
            },
            "FilterEnable": "AUTO",
            "PsiControl": "USE_PSI",
            "FilterStrength": 0,
            "DeblockFilter": "DISABLED",
            "DenoiseFilter": "DISABLED",
            "TimecodeSource": "ZEROBASED",
            "FileInput": "s3://s3-bucket/sources/source.mov"
          }
        ]
      },
      "AccelerationSettings": {
        "Mode": "DISABLED"
      },
      "StatusUpdateInterval": "SECONDS_15",
      "Priority": 0,
      "HopDestinations": []
    },
    "region": "us-east-1",
    "endpointURL": "https://vasjpylpa.mediaconvert.us-east-1.amazonaws.com"
  }
}

Referring back to the system architecture, at this point the Stream On-Demand API & Services have:

• validated the POST Optimization Request,
• ensured that the asset is present and accessible,
• inspected the properties of the underlying asset to determine the compute resources needed to optimize the asset,
• scheduled a new Optimizer container to process the video file and
• prepared and returned a response to the caller.
  
  

Once the Optimizer container starts:

• the Optimizer uses the chosen encoder, along with the specified configuration, to encode, optimize and analyze the video, streaming the IMAX VisionScience measurements and metrics to Inisghts (Note: no part of the video itself is streamed to Insights),
• Insights applies additional viewer intelligence algorithms to the incoming streams before storing the data and,
• upon completion of the optimization process, acknowledgements are recognized by both sides and the Optimizer container is destroyed (i.e. elastic scaling) in order to release the compute resources being used by the IMAX Stream On-Demand Platform.
  
  

In order to use the Insights REST API, you will need an Insights account along with an API clientId and clientSecret. If you have not done so already, please use the IMAX Help Center to create a ticket specifying your organization and the email address you wish to use as your account login. Alternatively, you can send these values to your IMAX representative using Slack/email. In short order, you should receive an invitation email with further instructions on how to finish setting up your Insights account. Your support ticket will also include your clientId and clientSecret.

The Insights REST API is secured using OAuth 2.0 tokens. As such, you are required to submit a POST request to the login endpoint in order to receive an OAuth2 token, as shown below:

curl -X POST \
  https://insights.sct.imax.com/api/4.0/login \
  -header 'Accept: application/application/json' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'client_id=<clientID>&client_secret=<clientSecret>'

where <clientID> and <clientSecret> are the values you received when setting up your Insights account.

The alphanumeric OAuth2 token will be included in the payload of the response and labelled as access_token:

{
    "access_token": "4RtxmBmskRFfcnvJXjVdXHZbWDbwCdrHHJH8qJNp",
    "token_type": "Bearer",
    "expires_in": 3600
}

The OAuth2 token must be included as a Bearer token in the Authorization header for all requests sent to an Insights REST API endpoint, as shown below:

curl -X POST \
  https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false \
  -H 'Authorization: Bearer 4RtxmBmskRFfcnvJXjVdXHZbWDbwCdrHHJH8qJNp'
  .
  .
  .

Each OAuth2 token expires after 1 hour, regardless of activity. To obtain a new token, you will need to resubmit the POST request to the login endpoint.

The Insights REST API exposes a single endpoint for all requests: /api/4.0/queries/run/json?cache=false and each request must be sent using HTTP POST semantics.

Let’s examine the body of a typical Insights POST request shown below:

{
    "model": "on_demand",
    "view": "vod_analysis_results",
    "fields": [
        "vod_analysis_results.test_id",
        "vod_analysis_results.test_video_time_1s",
        "vod_analysis_results.xvs"
        .
        .
    ],
    "filters": {
        "vod_analysis_results.analysis_uuid": "59508525-070d-428b-8857-d5482bf7c690",
        .
        .
    },
    "sorts": [
        "vod_analysis_results.test_video_time_1s",
        "vod_analysis_results.test_id",
        .
        .
    ]
}

• Every Insights request must specify the model attribute. For both the StreamAware and StreamSmart On-Demand products, the value should always be on_demand.
• Every Insights request must specify a view attribute (see Supported Views below).
• Every Insights request should specify a list of view fields representing the data that will be included in the response. A field is specified using the <view>.<field_name> notiation.
• An Insights request may specify a list of filters which are view fields that are constrained to various values and/or ranges in order to limit/control the data retrieved. The most common filter to use here is <view>.analysis_uuid, which constrains the results to a single analysis or optimization UUID (see example above). Filters become invaluable when your Insights responses return large datasets, as is the case when fetching frame results.
• An Insights request may specify a list of sorts which are view fields that control the sort order applied to the data in the response. Sorting is particularly useful when retrieving results in a time series.

Supported views

Insights supports the following views:

The views above, along with the fields they support, are presented in more detail, along with some examples, in the sections below.

Common view fields

Most Insights views for the StreamAware and StreamSmart products expose several common fields which can be used to add detail and context to your Insights responses. Use the following list of common fields:

Use the vod_status_analysis view and an analysis/optimization UUID filter to see a summary or roll-up status of a given analysis/optimization, as shown below:

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer 7FYtb3FB39ngTgDXF4WsCXb2NMvR3PRvhJGgypVx' \
--data '{
  "model": "on_demand",
  "view": "vod_status_analysis",
  "fields": [
    "vod_status_analysis.analysis_uuid",
    "vod_status_analysis.time",
    "vod_status_analysis.min_time",
    "vod_status_analysis.test_id",
    "vod_status_analysis.test_video",
    "vod_status_analysis.reference_video",
    "vod_status_analysis.stage",
    "vod_status_analysis.percent_done",
    "vod_status_analysis.test_count",
    "vod_quality_checks.count"
  ],
  "filters": {
    "vod_status_analysis.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b"
  }
}'

Consider the following JSON sample response:

[
  {
    "vod_status_analysis.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_analysis.time": "2023-08-30 16:41:00",
    "vod_status_analysis.min_time": "2023-08-30 16:40:14",
    "vod_status_analysis.stage": "Results Ready",
    "vod_status_analysis.percent_done": 100,
    "vod_status_analysis.test_count": 3,
    "vod_quality_checks.count": 4
  }
]

Note in the example response above:

• the test_count value indicates that there are 3 separate NR and/or FR analyses being performed and
• the count value indicates that there were 4 quality check failures found.

To fetch the status for a given analysis/optimization and to see the details of all no-reference and/or full-reference analyses being performed, use the vod_status_test view and filter on the analysis UUID, as shown below:

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer mFs5sKXwcVnCS4FC6kKbdFYqzcQYqVmHCKnKsNk5' \
--data '{
  "model": "on_demand",
  "view": "vod_status_test",
  "fields": [
    "vod_status_test.analysis_uuid",
    "vod_status_test.time",
    "vod_status_test.test_id",
    "vod_status_test.test_video",
    "vod_status_test.reference_video",
    "vod_status_test.stage",
    "vod_status_test.percent_done"
  ],
  "filters": {
    "vod_status_test.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b"
  },
  "sorts": [
    "vod_status_test.test_id"
  ]
}'

Assuming that 2e4db929-d00b-493a-98fb-77e321f3219b represents a full-reference analysis, you may see a response like the following if you query for the status immediately after submitting via the Stream On-Demand Platform REST API:

[
  {
    "vod_status_test.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_test.time": "2023-08-30 16:40:36",
    "vod_status_test.test_id": "1",
    "vod_status_test.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_status_test.reference_video": null,
    "vod_status_test.stage": "Analyzing",
    "vod_status_test.percent_done": 33.33
  },
  {
    "vod_status_test.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_test.time": "2023-08-30 16:40:37",
    "vod_status_test.test_id": "1-1",
    "vod_status_test.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_31.mp4",
    "vod_status_test.reference_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_status_test.stage": "Analyzing",
    "vod_status_test.percent_done": 20.12
  },
  {
    "vod_status_test.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_test.time": "2023-08-30 16:40:37",
    "vod_status_test.test_id": "1-2",
    "vod_status_test.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_41.mp4",
    "vod_status_test.reference_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_status_test.stage": "Aligning",
    "vod_status_test.percent_done": 88.75
  }
]

Notice the following in the response above:

• The NR analysis of 1 (dog_running.mp4) 33% complete.
• The FR analysis of 1-1 (dog_running_1080_h264_qp_31.mp4) has been temporarily aligned with the reference (i.e. Aligning state is finished) and is 20% of the way through its analysis.
• The 1-2 asset (dog_running_1080_h264_qp_41.mp4) is still being temporarily aligned with the reference and, as such, has not yet started its FR analysis.

Repeating the request above some minutes later results in the following response:

[
  {
    "vod_status_test.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_test.time": "2023-08-30 16:40:55",
    "vod_status_test.test_id": "1",
    "vod_status_test.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_status_test.reference_video": null,
    "vod_status_test.stage": "Results Ready",
    "vod_status_test.percent_done": 100
  },
  {
    "vod_status_test.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_test.time": "2023-08-30 16:41:00",
    "vod_status_test.test_id": "1-1",
    "vod_status_test.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_31.mp4",
    "vod_status_test.reference_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_status_test.stage": "Results Ready",
    "vod_status_test.percent_done": 100
  },
  {
    "vod_status_test.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_test.time": "2023-08-30 16:40:59",
    "vod_status_test.test_id": "1-2",
    "vod_status_test.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_41.mp4",
    "vod_status_test.reference_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_status_test.stage": "Results Ready",
    "vod_status_test.percent_done": 100
  }
]

In the result above, all analyses are complete and their results are ready to be used.

If you an encounter an error or unexpected behavior with a given analysis, you can choose to fetch a log of all the status messages for the analysis using the vod_status_updates view and filters on the vod_status_updates.analysis_uuid and vod_status_updates.test_id, as shown below:

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer rjV3dmFFrWfNWKPMmtcRtQdrPtjhYczCBFHPbn5z' \
--data '{
  "model": "on_demand",
  "view": "vod_status_updates",
  "fields": [
    "vod_status_updates.analysis_uuid",
    "vod_status_updates.time",
    "vod_status_updates.test_id",
    "vod_status_updates.test_video",
    "vod_status_updates.reference_video",
    "vod_status_updates.stage",
    "vod_status_updates.percent_done",
    "vod_status_updates.details",
    "vod_status_updates.error_details"
  ],
  "filters": {
    "vod_status_updates.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_updates.test_id": "1-1"
  },
  "sorts": [
    "vod_status_updates.test_id",
    "vod_status_updates.time desc"
  ]
}'

An abbreviated sample JSON response would be as follows:

[
  {
    "vod_status_updates.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_updates.time": "2023-08-30 16:41:01.176000",
    "vod_status_updates.reference_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_status_updates.test_id": "1-1",
    "vod_status_updates.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_31.mp4",
    "vod_status_updates.stage": "Analyzing",
    "vod_status_updates.details": "{\"result\":\"pass\",\"total\":492}",
    "vod_status_updates.error_details": null,
    "vod_status_updates.percent_done": 1
  },
  {
    "vod_status_updates.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_updates.time": "2023-08-30 16:41:01.122000",
    "vod_status_updates.reference_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_status_updates.test_id": "1-1",
    "vod_status_updates.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_31.mp4",
    "vod_status_updates.stage": "Analyzing",
    "vod_status_updates.details": "{\"progress\":{\"current\":492,\"total\":492}}",
    "vod_status_updates.error_details": null,
    "vod_status_updates.percent_done": 1
  },
  {
    "vod_status_updates.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_updates.time": "2023-08-30 16:41:00.937000",
    "vod_status_updates.reference_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_status_updates.test_id": "1-1",
    "vod_status_updates.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_31.mp4",
    "vod_status_updates.stage": "Analyzing",
    "vod_status_updates.details": null,
    "vod_status_updates.error_details": null,
    "vod_status_updates.percent_done": 1
  },
  {
    "vod_status_updates.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_updates.time": "2023-08-30 16:41:00.937000",
    "vod_status_updates.reference_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_status_updates.test_id": "1-1",
    "vod_status_updates.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_31.mp4",
    "vod_status_updates.stage": "Results Ready",
    "vod_status_updates.details": null,
    "vod_status_updates.error_details": null,
    "vod_status_updates.percent_done": 1
  },
  {
    "vod_status_updates.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_updates.time": "2023-08-30 16:41:00.615000",
    "vod_status_updates.reference_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_status_updates.test_id": "1-1",
    "vod_status_updates.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_31.mp4",
    "vod_status_updates.stage": "Analyzing",
    "vod_status_updates.details": "{\"progress\":{\"current\":485,\"total\":492}}",
    "vod_status_updates.error_details": null,
    "vod_status_updates.percent_done": 0.9858
  },
  {
    "vod_status_updates.analysis_uuid": "2e4db929-d00b-493a-98fb-77e321f3219b",
    "vod_status_updates.time": "2023-08-30 16:40:59.613000",
    "vod_status_updates.reference_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_status_updates.test_id": "1-1",
    "vod_status_updates.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_31.mp4",
    "vod_status_updates.stage": "Analyzing",
    "vod_status_updates.details": "{\"progress\":{\"current\":466,\"total\":492}}",
    "vod_status_updates.error_details": null,
    "vod_status_updates.percent_done": 0.9472
  },
  .
  .
  .
]

If the system encounters an error while processing an asset, it will report the content of the error in the vod_status_updates.error_details field for inspection. Many errors will contain sufficient information such that a remedy can be readily applied and the analysis rerun with minimal effort (i.e. fixing an invalid asset path or filename). Some errors are more serious and may require input from a video analyst and/or manipulation of the assets themselves. A failure to achieve automatic alignment between the reference and subject asset, for example, will result in an alignment error and manual actions must be taken to apply fixes before resubmitting the analysis. If an error does occur for which the cause or solution is unknown, please post a trouble ticket using the IMAX Help Center.

The following table lists the various fields of interest on the vod_status_analysis, vod_status_test and vod_status_updates views for extracting data on the status of an analysis. You can use these fields in the body of any request sent to the Insights REST API.

An Asset Score is a summary metric that provides a single value for an entire asset that is more accurate than peforming the mathematical average of the underlying frame scores. StreamAware and StreamSmart provide Asset Scores for the following metrics:

• IMAX ViewerScore (XVS),
• IMAX Encoder Performance Score (XEPS),
• IMAX Banding Score (XBS) and
• IMAX Content Complexity Score (XCCS).

Using the Stream On-Demand Platform REST API, we can submit a full-reference analysis that configures the On-Demand Analyzer to capture XBS and CCS in addition to XVS and XEPS, that are captured by default, as shown below:

curl --location 'https://example.sct.imax.lan/api/vod/v1/analyses' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--data '{
  "content": {
    "title": "Dog Running FR - Asset Scores"
  },
  "referenceAssets": [
    {
      "name": "dog_running.mp4",
      "path": "royalty_free/dog_running/source",
      "storageLocation": {
        "type": "PVC",
        "name": "video-files-pvc"
      }
    }
  ],
  "subjectAssets": [
    {
      "name": "dog_running_1080_h264_qp_31.mp4",
      "path": "royalty_free/dog_running/outputs",
      "storageLocation": {
        "type": "PVC",
        "name": "video-files-pvc"
      }
    },
    {
      "name": "dog_running_1080_h264_qp_41.mp4",
      "path": "royalty_free/dog_running/outputs",
      "storageLocation": {
        "type": "PVC",
        "name": "video-files-pvc"
      }
    }
  ],
  "analyzerConfig": {
    "enableBandingDetection": true,
    "enableComplexityAnalysis": true,
    "viewingEnvironments": [
      {
        "device": {
          "name": "oled65c9pua"
        },
        "viewerType": "TYPICAL"
      }
    ]
  }
}'

Asset XVS, XEPS, XBS and CCS are exposed as fields on the vod_video_asset_score view, which is a subview of the vod_analysis_results view and can be fetched from Insights as shown below:

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer NPp7SWngHvMXYJsyPRNQXc6nKGgZ4Dx7FFWsb9S5' \
--data '{
  "model": "on_demand",
  "view": "vod_analysis_results",
  "fields": [
    "vod_analysis_results.test_id",
    "device.device_name",
    "vod_analysis_results.reference_video",
    "vod_analysis_results.test_video",
    "vod_video_asset_score.asset_xvs",
    "vod_video_asset_score.asset_xeps",
    "vod_video_asset_score.asset_xbs",
    "vod_video_asset_score.asset_content_complexity"
  ],
  "filters": {
    "vod_analysis_results.analysis_uuid": "de66d3ab-7469-476b-9d15-965e173811ce"
  },
  "sorts": [
    "vod_analysis_results.test_id"
  ]
}'

The JSON response payload is as follows:

[
  {
    "vod_analysis_results.analysis_uuid": "de66d3ab-7469-476b-9d15-965e173811ce",
    "vod_analysis_results.test_id": "1",
    "device.device_name": "OLED65C9PUA",
    "vod_analysis_results.reference_video": null,
    "vod_analysis_results.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_video_asset_score.asset_xvs": 95.375,
    "vod_video_asset_score.asset_xeps": null,
    "vod_video_asset_score.asset_xbs": 0,
    "vod_analysis_results.asset_content_complexity": 37.4
  },
  {
    "vod_analysis_results.analysis_uuid": "de66d3ab-7469-476b-9d15-965e173811ce",
    "vod_analysis_results.test_id": "1-1",
    "device.device_name": "OLED65C9PUA",
    "vod_analysis_results.reference_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_analysis_results.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_31.mp4",
    "vod_video_asset_score.asset_xvs": 83.416,
    "vod_video_asset_score.asset_xeps": 88.02,
    "vod_video_asset_score.asset_xbs": 18.2,
    "vod_analysis_results.asset_content_complexity": null
  },
  {
    "vod_analysis_results.analysis_uuid": "de66d3ab-7469-476b-9d15-965e173811ce",
    "vod_analysis_results.test_id": "1-2",
    "device.device_name": "OLED65C9PUA",
    "vod_analysis_results.reference_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/source/dog_running.mp4",
    "vod_analysis_results.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_41.mp4",
    "vod_video_asset_score.asset_xvs": 60.048,
    "vod_video_asset_score.asset_xeps": 60.057,
    "vod_video_asset_score.asset_xbs": 42.6,
    "vod_analysis_results.asset_content_complexity": null
  }
]

We can make the following conclusions regarding the values above:

• The reference asset (dog_running.mp4) is a reasonably high quality asset as its Asset XVS score of 95 and Asset XBS of 0 places it in the excellent category where impairments, including color banding, are imperceptable for the typical viewer.
• The 1-1 subject/test asset (dog_running_1080_h264_qp_31.mp4) qualifies as an excellent encode as its Asset XVS is above 80 and, although it has some banding, its Asset XBS of 18.2 falls into the imperceptible category. The Asset XEPS of 88 indicates that the encoder did an excellent job of maintaining the reference quality.
• The 1-2 subject/test asset (dog_running_1080_h264_qp_41.mp4) qualifies as a fair encode as its Asset XVS is only 60 and its Asset XBS of 42 means that the color banding is creeping into the slight annoying category. The Asset EPS score of 60 indicates that the encoder did not do a very good job of maintaining the reference quality, especially considering that the content complexity score of the reference (Asset CCS) is reasonably low at only 37.
• The example above is purposefully terse in order to illustrate the fetching of Asset Scores. The vod_analysis_results has many additional fields that may of interest to include in your response. Please refer to Supported view fields for details.

When you’re interested in a more granular treatment of the VisionScience scores for a given asset, you’ll want to use the measurement period fields provided on the vod_analysis_results view in order to build a time series view of your asset’s scores.

Using the Stream On-Demand Platform REST API, let’s submit a no-reference analysis for an HDR asset that configures the On-Demand Analyzer to capture color information and statistics, in addition to banding, as shown below:

curl --location 'https://example.sct.imax.lan/api/vod/v1/analyses' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--data '{
  "content": {
    "title": "HDR NR Test"
  },
  "subjectAssets": [
    {
      "name": "samsung-demo-reel-version-a.mp4",
      "path": "content-similarity-assets/UHD-HDR-MP4-samsung-demo-reel",
      "storageLocation": {
        "type": "PVC",
        "name": "on-201"
      },
      "hdr": true
    }
  ],
  "analyzerConfig": {
    "enableBandingDetection": true,
    "enableColorInformation": true,
    "enableColorStatsCollection": true,
    "viewingEnvironments": [
      {
        "device": {
          "name": "oled65c9pua"
        },
        "viewerType": "TYPICAL"
      }
    ]
  }
}'

Next, let’s fetch our analysis results from Insights, including color gamut and luminance data, in a time series view using a measurement period of 2s:

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer 9tVkH455J5Z8X5jGVDBpRbSwBJz5f7vcHTftRv7K' \
--data '{
  "model": "on_demand",
  "view": "vod_analysis_results",
  "fields": [
    "vod_analysis_results.analysis_uuid",
    "vod_analysis_results.test_video_time_2s",
    "vod_analysis_results.test_video",
    "vod_analysis_results.xvs",
    "vod_analysis_results.max_xvs",
    "vod_analysis_results.min_xvs",
    "vod_analysis_results.avg_xbs",
    "vod_analysis_results.max_xbs",
    "vod_analysis_results.min_xbs",
    "vod_analysis_results.color_primaries",
    "vod_analysis_results.color_space",
    "vod_analysis_results.color_transfer_characteristics",
    "vod_analysis_results.max_content_light_level",
    "vod_analysis_results.max_frame_average_light_level",
    "vod_analysis_results.color_gamut",
    "vod_analysis_results.color_gamut_rec709",
    "vod_analysis_results.color_gamut_p3d65",
    "vod_analysis_results.color_gamut_rec2020",
    "vod_analysis_results.color_gamut_unknown",
    "vod_analysis_results.max_pixel_luminance",
    "vod_analysis_results.min_pixel_luminance",
    "vod_analysis_results.avg_frame_luminance",
    "vod_analysis_results.max_frame_luminance",
    "vod_analysis_results.min_frame_luminance",
    "vod_analysis_results.frame_count"
  ],
  "filters": {
    "vod_analysis_results.analysis_uuid": "b9b437a7-f966-4e2b-8bb4-02d9cd1aa2a8"
  },
  "sorts": [
    "vod_analysis_results.test_video_time_2s"
  ]
}'

The abbreviated JSON response payload is as follows:

[
  {
    "vod_analysis_results.analysis_uuid": "b9b437a7-f966-4e2b-8bb4-02d9cd1aa2a8",
    "vod_analysis_results.test_video_time_2s": "00:00:02",
    "vod_analysis_results.test_video": "file:///mnt/on-201/content-similarity-assets/UHD-HDR-MP4-samsung-demo-reel/samsung-demo-reel-version-a.mp4",
    "vod_analysis_results.color_primaries": "bt2020",
    "vod_analysis_results.color_space": "bt2020nc",
    "vod_analysis_results.color_transfer_characteristics": "smpte2084",
    "vod_analysis_results.max_content_light_level": 1000,
    "vod_analysis_results.max_frame_average_light_level": 1000,
    "vod_analysis_results.xvs": 97.9375,
    "vod_analysis_results.max_xvs": 98,
    "vod_analysis_results.min_xvs": 97.875,
    "vod_analysis_results.avg_xbs": 53.6875,
    "vod_analysis_results.max_xbs": 56,
    "vod_analysis_results.min_xbs": 50,
    "vod_analysis_results.color_gamut": "P3D65",
    "vod_analysis_results.color_gamut_rec709": 0,
    "vod_analysis_results.color_gamut_p3d65": 48,
    "vod_analysis_results.color_gamut_rec2020": 0,
    "vod_analysis_results.color_gamut_unknown": 0,
    "vod_analysis_results.max_pixel_luminance": 10000,
    "vod_analysis_results.min_pixel_luminance": 0,
    "vod_analysis_results.avg_frame_luminance": 281.4792,
    "vod_analysis_results.max_frame_luminance": 283,
    "vod_analysis_results.min_frame_luminance": 280,
    "vod_analysis_results.frame_count": 48
  },
  {
    "vod_analysis_results.analysis_uuid": "b9b437a7-f966-4e2b-8bb4-02d9cd1aa2a8",
    "vod_analysis_results.test_video_time_2s": "00:00:04",
    "vod_analysis_results.test_video": "file:///mnt/on-201/content-similarity-assets/UHD-HDR-MP4-samsung-demo-reel/samsung-demo-reel-version-a.mp4",
    "vod_analysis_results.color_primaries": "bt2020",
    "vod_analysis_results.color_space": "bt2020nc",
    "vod_analysis_results.color_transfer_characteristics": "smpte2084",
    "vod_analysis_results.max_content_light_level": 1000,
    "vod_analysis_results.max_frame_average_light_level": 1000,
    "vod_analysis_results.xvs": 97.9165,
    "vod_analysis_results.max_xvs": 98,
    "vod_analysis_results.min_xvs": 97.833,
    "vod_analysis_results.avg_xbs": 55.7083,
    "vod_analysis_results.max_xbs": 61,
    "vod_analysis_results.min_xbs": 52,
    "vod_analysis_results.color_gamut": "P3D65",
    "vod_analysis_results.color_gamut_rec709": 0,
    "vod_analysis_results.color_gamut_p3d65": 48,
    "vod_analysis_results.color_gamut_rec2020": 0,
    "vod_analysis_results.color_gamut_unknown": 0,
    "vod_analysis_results.max_pixel_luminance": 9043,
    "vod_analysis_results.min_pixel_luminance": 0,
    "vod_analysis_results.avg_frame_luminance": 280.0417,
    "vod_analysis_results.max_frame_luminance": 281,
    "vod_analysis_results.min_frame_luminance": 279,
    "vod_analysis_results.frame_count": 48
  },
  {
    "vod_analysis_results.analysis_uuid": "b9b437a7-f966-4e2b-8bb4-02d9cd1aa2a8",
    "vod_analysis_results.test_video_time_2s": "00:00:06",
    "vod_analysis_results.test_video": "file:///mnt/on-201/content-similarity-assets/UHD-HDR-MP4-samsung-demo-reel/samsung-demo-reel-version-a.mp4",
    "vod_analysis_results.color_primaries": "bt2020",
    "vod_analysis_results.color_space": "bt2020nc",
    "vod_analysis_results.color_transfer_characteristics": "smpte2084",
    "vod_analysis_results.max_content_light_level": 1000,
    "vod_analysis_results.max_frame_average_light_level": 1000,
    "vod_analysis_results.xvs": 96.5925,
    "vod_analysis_results.max_xvs": 98,
    "vod_analysis_results.min_xvs": 95.185,
    "vod_analysis_results.avg_xbs": 45.3542,
    "vod_analysis_results.max_xbs": 60,
    "vod_analysis_results.min_xbs": 28,
    "vod_analysis_results.color_gamut": "P3D65",
    "vod_analysis_results.color_gamut_rec709": 0,
    "vod_analysis_results.color_gamut_p3d65": 48,
    "vod_analysis_results.color_gamut_rec2020": 0,
    "vod_analysis_results.color_gamut_unknown": 0,
    "vod_analysis_results.max_pixel_luminance": 8835,
    "vod_analysis_results.min_pixel_luminance": 0,
    "vod_analysis_results.avg_frame_luminance": 236.8125,
    "vod_analysis_results.max_frame_luminance": 282,
    "vod_analysis_results.min_frame_luminance": 184,
    "vod_analysis_results.frame_count": 48
  },
  .
  .
  .
]

Notice in the results above:

• We have one result in the response for each 2s measurement period (i.e. 2s, 4s, 6s etc).
• Insights provides the mathematical average, minimum and maximum for the various metrics and measurements requested. These values are calculated using the values of all the underlying frames that make up the measurement period (i.e. a frame count of 48 frames in the example above).
• We choose to sort the results based off the chosen measurement period (i.e. vod_analysis_results.test_video_time_2s)

The vod_analysis_results view exposes fields and subviews (e.g. vod_video_asset_score) to capture the various VisionScience measurements and metrics for an analysis or optimization over an entire asset (i.e. Asset Scores) and/or various measurement periods (time series). The following subsections lists the various fields of interest on the vod_analysis_results view (and its subviews) that can be used in the body of any request sent to the Insights REST API.

Metrics and measurements

The Insights vod_analysis_results view exposes the following list of fields:

And the following filter-only fields:

Measurement periods

Insights supports the following measurement period fields:

For some use cases, it may prove desirable to fetch the requested VisionScience measurements and metrics for each frame in your analyzed assets. Frame level data is provided using the vod_analysis_frame_results view.

Using the Stream On-Demand Platform REST API, we can submit a full-reference analysis for HDR assets that configures the On-Demand Analyzer to capture color information and statistics, in addition to XBS (color banding), XEPS (encoder performance) and CVD (color volume differncing), as shown below:

curl --location 'https://example.sct.imax.lan/api/vod/v1/analyses' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--data '{
  "content": {
    "title": "HDR FR Demo"
  },
  "referenceAssets": [
    {
      "hdr": true,
      "name": "demo1_hdr10_ref.mp4",
      "storageLocation": {
        "type": "PVC",
        "name": "on-201"
      },
      "path": "demo/encode_validation/HDR_demo"
    }
  ],
  "subjectAssets": [
    {
      "hdr": true,
      "name": "demo1_hdr10_test_cbr_1920x1080_5400kbps.mp4",
      "storageLocation": {
        "type": "PVC",
        "name": "on-201"
      },
      "path": "demo/encode_validation/HDR_demo"
    }
  ],
  "analyzerConfig": {
    "enableBandingDetection": true,
    "enableColorInformation": true,
    "enableColorStatsCollection": true,
    "enableColorVolumeDifference": true,
    "viewingEnvironments": [
      {
        "device": {
          "name": "oled65c9pua"
        },
        "viewerType": "TYPICAL"
      },
      {
        "device": {
          "name": "ssimpluscore"
        },
        "viewerType": "TYPICAL"
      },
      {
        "device": {
          "name": "xl2420t"
        },
        "viewerType": "TYPICAL"
      }
    ]
  }
}'

The following Insights request fetches the XVS, XEPS, XBS and CVD scores along with the color gamut and pixel and frame luminance values for every frame:

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer 29RJCp5vn6WVt68rj4XMqh3dmyHcFbBxzD2WsM9J' \
--data '{
  "model": "on_demand",
  "view": "vod_analysis_frame_results",
  "fields": [
    "vod_analysis_frame_results.analysis_uuid",
    "vod_analysis_frame_results.reference_video",
    "vod_analysis_frame_results.test_video",
    "vod_analysis_frame_results.test_id",
    "vod_analysis_frame_results.reference_video_time_1frame",
    "vod_analysis_frame_results.test_video_time_1frame",
    "vod_analysis_frame_results.avg_xeps",
    "vod_analysis_frame_results.avg_xvs",
    "vod_analysis_frame_results.avg_xbs",
    "vod_analysis_frame_results.color_primaries",
    "vod_analysis_frame_results.color_space",
    "vod_analysis_frame_results.color_transfer_characteristics",
    "vod_analysis_frame_results.color_gamut",
    "vod_analysis_frame_results.avg_color_volume_difference",
    "vod_analysis_frame_results.avg_pixel_luminance",
    "vod_analysis_frame_results.min_pixel_luminance",
    "vod_analysis_frame_results.min_pixel_luminance_x",
    "vod_analysis_frame_results.min_pixel_luminance_y",
    "vod_analysis_frame_results.max_pixel_luminance",
    "vod_analysis_frame_results.max_pixel_luminance_x",
    "vod_analysis_frame_results.max_pixel_luminance_y",
    "vod_analysis_frame_results.avg_frame_luminance",
    "device.device_name"
  ],
  "filters": {
    "vod_analysis_frame_results.analysis_uuid": "de66d3ab-7469-476b-9d15-965e173811ce"
  },
  "sorts": [
    "vod_analysis_frame_results.reference_video_time_1frame",
    "vod_analysis_frame_results.test_id"
  ]
}'

The snippet below represents a small sampling of the response output:

[
  .
  .
  {
    "vod_analysis_frame_results.analysis_uuid": "0e37bc9b-34ca-45ea-aaf5-da713fada956",
    "vod_analysis_frame_results.reference_video": null,
    "vod_analysis_frame_results.test_video": "file:///mnt/on-201/demo/encode_validation/HDR_demo/demo1_hdr10_ref.mp4",
    "vod_analysis_frame_results.test_id": "1",
    "vod_analysis_frame_results.reference_video_time_1frame": "00:00:04:23",
    "vod_analysis_frame_results.test_video_time_1frame": "00:00:04:23",
    "vod_analysis_frame_results.color_primaries": "bt2020",
    "vod_analysis_frame_results.color_space": "bt2020nc",
    "vod_analysis_frame_results.color_transfer_characteristics": "smpte2084",
    "device.device_name": "OLED65C9PUA",
    "vod_analysis_frame_results.avg_xeps": null,
    "vod_analysis_frame_results.avg_xvs": 100,
    "vod_analysis_frame_results.avg_xbs": 50,
    "vod_analysis_frame_results.color_gamut": "P3D65",
    "vod_analysis_frame_results.avg_color_volume_difference": null,
    "vod_analysis_frame_results.min_pixel_luminance": 0,
    "vod_analysis_frame_results.min_pixel_luminance_x": 1170,
    "vod_analysis_frame_results.min_pixel_luminance_y": 0,
    "vod_analysis_frame_results.max_pixel_luminance": 749,
    "vod_analysis_frame_results.max_pixel_luminance_x": 2490,
    "vod_analysis_frame_results.max_pixel_luminance_y": 1028,
    "vod_analysis_frame_results.avg_frame_luminance": 37
  },
  {
    "vod_analysis_frame_results.analysis_uuid": "0e37bc9b-34ca-45ea-aaf5-da713fada956",
    "vod_analysis_frame_results.reference_video": "file:///mnt/on-201/demo/encode_validation/HDR_demo/demo1_hdr10_ref.mp4",
    "vod_analysis_frame_results.test_video": "file:///mnt/on-201/demo/encode_validation/HDR_demo/demo1_hdr10_test_cbr_1920x1080_5400kbps.mp4",
    "vod_analysis_frame_results.test_id": "1-1",
    "vod_analysis_frame_results.reference_video_time_1frame": "00:00:04:23",
    "vod_analysis_frame_results.test_video_time_1frame": "00:00:04:23",
    "vod_analysis_frame_results.color_primaries": "bt2020",
    "vod_analysis_frame_results.color_space": "bt2020nc",
    "vod_analysis_frame_results.color_transfer_characteristics": "smpte2084",
    "device.device_name": "OLED65C9PUA",
    "vod_analysis_frame_results.avg_xeps": 92,
    "vod_analysis_frame_results.avg_xvs": 89,
    "vod_analysis_frame_results.avg_xbs": 23,
    "vod_analysis_frame_results.color_gamut": "P3D65",
    "vod_analysis_frame_results.avg_color_volume_difference": 8.52735,
    "vod_analysis_frame_results.min_pixel_luminance": 0,
    "vod_analysis_frame_results.min_pixel_luminance_x": 1198,
    "vod_analysis_frame_results.min_pixel_luminance_y": 0,
    "vod_analysis_frame_results.max_pixel_luminance": 1237,
    "vod_analysis_frame_results.max_pixel_luminance_x": 300,
    "vod_analysis_frame_results.max_pixel_luminance_y": 674,
    "vod_analysis_frame_results.avg_frame_luminance": 37
  },
  .
  .
]

Please note the following about the Insights request and response above:

• While the vod_analysis_frame_results view exposes fields for the average, minimum and maximum values for most of the VisionScience measurements and metrics, one need only request the average (e.g. avg_xvs) since the granularity of this view is already at the lowest (i.e. frame) level. Requesting the minimum and maximum fields will result in identical values to that of the average.
• Where pixel values are provided, however, average, minimum and maximum will have proper meaning and different results (i.e. see avg/min/max_pixel_luminance above).
• The vod_analysis_frame_results view exposes two fields, reference_video_time_1frame and test_video_time_1frame, to capture the respective time within the reference and subject/test videos. These time fields are of the format HH:MM:SS:FF, where FF is the frame number within the given second and the possible values depend on the framerate of the underlying asset.

As with all other queries submitted to the Insights REST API, you are encouraged to use filters to help reduce the size of your results and focus on specific parts of your data set. Using the query above, we can add a subject/test asset filter (i.e. vod_analysis_frame_results.test_id) and device filter (i.e. device.device_name) to return all the frame scores just for the subject/test asset identified by 1-1 and the OLED65C9PUA device:

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer 29RJCp5vn6WVt68rj4XMqh3dmyHcFbBxzD2WsM9J' \
--data '{
  "model": "on_demand",
  "view": "vod_analysis_frame_results",
  "fields": [
    "vod_analysis_frame_results.analysis_uuid",
    "vod_analysis_frame_results.reference_video",
    "vod_analysis_frame_results.test_video",
    "vod_analysis_frame_results.test_id",
    "vod_analysis_frame_results.reference_video_time_1frame",
    "vod_analysis_frame_results.test_video_time_1frame",
    "vod_analysis_frame_results.avg_xeps",
    "vod_analysis_frame_results.avg_xvs",
    "vod_analysis_frame_results.avg_xbs",
    "vod_analysis_frame_results.color_primaries",
    "vod_analysis_frame_results.color_space",
    "vod_analysis_frame_results.color_transfer_characteristics",
    "vod_analysis_frame_results.color_gamut",
    "vod_analysis_frame_results.avg_color_volume_difference",
    "vod_analysis_frame_results.avg_pixel_luminance",
    "vod_analysis_frame_results.min_pixel_luminance",
    "vod_analysis_frame_results.min_pixel_luminance_x",
    "vod_analysis_frame_results.min_pixel_luminance_y",
    "vod_analysis_frame_results.max_pixel_luminance",
    "vod_analysis_frame_results.max_pixel_luminance_x",
    "vod_analysis_frame_results.max_pixel_luminance_y",
    "vod_analysis_frame_results.avg_frame_luminance",
    "device.device_name"
  ],
  "filters": {
    "vod_analysis_frame_results.analysis_uuid": "de66d3ab-7469-476b-9d15-965e173811ce",
    "vod_analysis_frame_results.test_id": "1-1",
    "device.device_name": "OLED65C9PUA"
  },
  "sorts": [
    "vod_analysis_frame_results.reference_video_time_1frame",
    "vod_analysis_frame_results.test_id"
  ]
}'

Supported view fields

The vod_analysis_frame_results view exposes fields to capture the various VisionScience measurements and metrics (e.g. XVS, XEPS, XBS, CVD, CCM, physical and visual noise, HDR color gamut and luminance) for each frame of the assets included in an analysis. The following section lists the various fields of interest on the vod_analysis_frame_results view that can be used in the body of any request sent to the Insights REST API.

And the following filter-only fields:

When audio measurement is enabled, each physical audio track within a video asset will be measured based using the measurement standard configured within the audio object of the analysis submission. Using the Stream On-Demand Platform REST API, we can submit a new analysis choosing to include an audio configuration object with any of the associated assets, an example of which is shown below:

{
  "audio": {
    "groups": [
      {
        "qualityCheckConfig": {
          "loudnessChecks": [
            {
              "checkType": "MIN_LOUDNESS_RANGE",
              "enabled": true,
              "threshold": 8
            },
            {
              "checkType": "MAX_LOUDNESS_RANGE",
              "enabled": true,
              "threshold": 30
            },
            {
              "checkType": "MAX_INTEGRATED_LOUDNESS",
              "enabled": true,
              "threshold": -2
            },
            {
              "checkType": "MAX_MOMENTARY_LOUDNESS",
              "enabled": true,
              "duration": 2,
              "skipStart": 1.25,
              "skipEnd": 1.25,
              "threshold": -2
            }
          ]
        },
        "loudnessMeasurements": {
          "algorithm": "ITU_BS_1770_3",
          "enabled": true
        }
      }
    ]
  }
}

In the current release only a single group is permitted. All configuration and quality checks are applied to all discovered physical audio tracks within the video asset. Future releases will add the following:

• performing measurements and quality checks on selectable tracks and
• mixing of uncompressed audio channels from physical audio tracks to create a logical audio track for measurement and quality checks.

The audio loudness results can be fetched from Insights using the vod_audio_loudness view as shown below

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer 9tVkH455J5Z8X5jGVDBpRbSwBJz5f7vcHTftRv7K' \
--data '{
  "model": "on_demand",
  "view": "vod_audio_loudness",
  "fields": [
    "vod_audio_loudness.analysis_uuid",
    "vod_audio_loudness.test_video_time",
    "vod_audio_loudness.integrated_loudness",
    "vod_audio_loudness.loudness_range",
    "vod_audio_loudness.high_lra",
    "vod_audio_loudness.low_lra",
    "vod_audio_loudness.max_momentary_loudness",
    "vod_audio_loudness.min_momentary_loudness",
    "vod_audio_loudness.max_short_term_loudness",
    "vod_audio_loudness.min_short_term_loudness",
    "vod_audio_loudness.max_true_peak_channel_ch1",
    "vod_audio_loudness.min_true_peak_channel_ch1",
    "vod_audio_loudness.max_true_peak_channel_ch2",
    "vod_audio_loudness.min_true_peak_channel_ch2",
    "vod_audio_loudness.max_true_peak",
    "vod_audio_loudness.min_true_peak",
    "vod_audio_loudness.stream_index"
  ],
  "filters": {
    "vod_audio_loudness.analysis_uuid": "b8b634d3-11f5-4598-b842-9cde5d58c713"
  },
  "sorts": [
    "vod_audio_loudness.test_video_time desc"
  ]
}'

The abbreviated JSON response payload is as follows:

[
  {
    "vod_audio_loudness.analysis_uuid": "b8b634d3-11f5-4598-b842-9cde5d58c713",
    "vod_audio_loudness.test_video_time": "00:03:00",
    "vod_audio_loudness.stream_index": 1,
    "vod_audio_loudness.integrated_loudness": -23.53,
    "vod_audio_loudness.loudness_range": 4.14,
    "vod_audio_loudness.high_lra": -21.86,
    "vod_audio_loudness.low_lra": -26,
    "vod_audio_loudness.max_momentary_loudness": -22.52,
    "vod_audio_loudness.min_momentary_loudness": -25.05,
    "vod_audio_loudness.max_short_term_loudness": -22.87,
    "vod_audio_loudness.min_short_term_loudness": -23.27,
    "vod_audio_loudness.max_true_peak_channel_ch1": -19.58,
    "vod_audio_loudness.min_true_peak_channel_ch1": -31.7,
    "vod_audio_loudness.max_true_peak_channel_ch2": -19.66,
    "vod_audio_loudness.min_true_peak_channel_ch2": -31.7,
    "vod_audio_loudness.max_true_peak": -10.52,
    "vod_audio_loudness.min_true_peak": -120
  },
  {
    "vod_audio_loudness.analysis_uuid": "b8b634d3-11f5-4598-b842-9cde5d58c713",
    "vod_audio_loudness.test_video_time": "00:02:59",
    "vod_audio_loudness.stream_index": 1,
    "vod_audio_loudness.integrated_loudness": null,
    "vod_audio_loudness.loudness_range": null,
    "vod_audio_loudness.high_lra": null,
    "vod_audio_loudness.low_lra": null,
    "vod_audio_loudness.max_momentary_loudness": -20.88,
    "vod_audio_loudness.min_momentary_loudness": -24.08,
    "vod_audio_loudness.max_short_term_loudness": -23.24,
    "vod_audio_loudness.min_short_term_loudness": -23.55,
    "vod_audio_loudness.max_true_peak_channel_ch1": -18.56,
    "vod_audio_loudness.min_true_peak_channel_ch1": -24.58,
    "vod_audio_loudness.max_true_peak_channel_ch2": -18.56,
    "vod_audio_loudness.min_true_peak_channel_ch2": -24.44,
    "vod_audio_loudness.max_true_peak": -9.4,
    "vod_audio_loudness.min_true_peak": -120
  },
  {
    "vod_audio_loudness.analysis_uuid": "b8b634d3-11f5-4598-b842-9cde5d58c713",
    "vod_audio_loudness.test_video_time": "00:02:58",
    "vod_audio_loudness.stream_index": 1,
    "vod_audio_loudness.integrated_loudness": null,
    "vod_audio_loudness.loudness_range": null,
    "vod_audio_loudness.high_lra": null,
    "vod_audio_loudness.low_lra": null,
    "vod_audio_loudness.max_momentary_loudness": -22.14,
    "vod_audio_loudness.min_momentary_loudness": -24.72,
    "vod_audio_loudness.max_short_term_loudness": -23.49,
    "vod_audio_loudness.min_short_term_loudness": -23.88,
    "vod_audio_loudness.max_true_peak_channel_ch1": -19.58,
    "vod_audio_loudness.min_true_peak_channel_ch1": -27.74,
    "vod_audio_loudness.max_true_peak_channel_ch2": -19.49,
    "vod_audio_loudness.min_true_peak_channel_ch2": -27.54,
    "vod_audio_loudness.max_true_peak": -10.46,
    "vod_audio_loudness.min_true_peak": -120
  },
  .
  .
  .
]

We can make the following conclusions based on the above results:

• The asset is 3 minutes in length
• The stream index in the response indicates the first physical audio track that is scored
  • Audio track enumeration should be related to the vod_audio_metadata view
• Each result object contains 1s worth of measurements
  • This means that for min/max True Peak, Short-Term Loudness and Momentary loudness is only for that reporting period and do not represent measurement for the entire asset
• Integrated Loudness and Loudness Range are returned only on the last data point
  • "vod_audio_loudness.test_video_time": "00:03:00" in the example above as these measures are based on the entire audio track.
• There are at least 2 audio channels monitored and their individual True Peak measurements are returned
  • The mapping of audio channel index (ch1, ch2) map are discussed in the following section

Details regarding units for these measures can be found in the Audio section.

Please note that audio loudness measurements are reported in terms of when they occur during the presentation of video.

Audio frames that exist in an asset or sidecar will not be processed if the presentation time for those frames is prior to the presentation time of the first frame of video in the analysis. Processing of audio frames will begin at the time of the first video frame, such that audio loudness and quality checks always be reported in terms of a video time greater than or equal to 0 seconds.

In the event that audio presentation continues after the last frame of video, audio frames will continue to be processed. Audio loudness measurements and quality checks will continue to be reported in terms of video time, as if the presentation of video had continued until the end of playback for all associated audio tracks.

True Peak and Channel Ordering

The vod_audio_loudness view enumerates the individual channels within the track when reporting True Peak levels from channels 1 to 32 based on the channel layout. This is based off the information in Audio section titled Number of Channels to Channel Layout Mapping and Number of Channels to Channel Layout Mapping. Note that this is not an exhaustive list and other channel layouts may differ.

Supported view fields

The vod_audio_loudness view exposes fields to capture various audio loudness measurements and metrics (e.g. integrated loudness, loudness range, momentary loudness, short-term loudness, true peak level) for the assets included in an analysis. The following section lists the various fields of interest on the vod_audio_loudness view that can be used in the body of any request sent to the Insights REST API.

StreamAware can be used to detect and report the cadence patterns for the videos in a given analysis. To enable cadence pattern detection, you would use the Stream On-Demand Platform REST API, to submit an analyzerConfig object with the enableCadencePatternDetection set to true as shown below:

"analyzerConfig": {
  "enableCadencePatternDetection": true,
  "enableBandingDetection": true,
  "enableColorVolumeDifference": true,
  .
  .
  .
}

The cadence pattern results can be fetched from Insights using the vod_cadence_pattern view as shown below:

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer 9tVkH455J5Z8X5jGVDBpRbSwBJz5f7vcHTftRv7K' \
--data '{
  "model": "on_demand",
  "view": "vod_cadence_pattern",
  "fields": [
    "vod_cadence_pattern.cadence_pattern",
    "vod_cadence_pattern.details",
    "vod_cadence_pattern.pattern_start_frame",
    "vod_cadence_pattern.pattern_end_frame",
    "vod_cadence_pattern.cadence_pattern_offset",
    "vod_cadence_pattern.pattern_start_time_timecode",
    "vod_cadence_pattern.pattern_end_time_timecode",
    "vod_cadence_pattern.test_id"
  ],
  "filters": {
    "vod_cadence_pattern.analysis_uuid": "28fe7faa-e396-4b7a-8fc0-9b9f34915ccf"
  },
  "sorts": [
    "vod_cadence_pattern.pattern_start_frame"
  ]
}'

A few things to note:

• The vod_cadence_pattern.details field contains additional information such as a confidence threshold that can be used to filter out results
  • The JSON object is in this form
    "{\"confidence\":66}"
• The vod_cadence_pattern.cadence_pattern will be one of the cadences described in the Cadence Pattern Detection section or Unknown (Low Motion) or 1:1 (No Pattern)

The above request will give the following JSON result:

[
  {
    "vod_cadence_pattern.pattern_start_frame": 1,
    "vod_cadence_pattern.pattern_end_frame": 90,
    "vod_cadence_pattern.test_id": "1",
    "vod_cadence_pattern.cadence_pattern": "Unknown (Low Motion)",
    "vod_cadence_pattern.details": "{\"confidence\":66}",
    "vod_cadence_pattern.cadence_pattern_offset": 0,
    "vod_cadence_pattern.pattern_start_time_timecode": "00:00:00:01",
    "vod_cadence_pattern.pattern_end_time_timecode": "00:00:03:00"
  },
  {
    "vod_cadence_pattern.pattern_start_frame": 2,
    "vod_cadence_pattern.pattern_end_frame": 101,
    "vod_cadence_pattern.test_id": "1-1",
    "vod_cadence_pattern.cadence_pattern": "Unknown (Low Motion)",
    "vod_cadence_pattern.details": "{\"confidence\":73}",
    "vod_cadence_pattern.cadence_pattern_offset": 0,
    "vod_cadence_pattern.pattern_start_time_timecode": "00:00:00:02",
    "vod_cadence_pattern.pattern_end_time_timecode": "00:00:03:11"
  },
  {
    "vod_cadence_pattern.pattern_start_frame": 91,
    "vod_cadence_pattern.pattern_end_frame": 1800,
    "vod_cadence_pattern.test_id": "1",
    "vod_cadence_pattern.cadence_pattern": "7:8",
    "vod_cadence_pattern.details": "{\"confidence\":100}",
    "vod_cadence_pattern.cadence_pattern_offset": 8,
    "vod_cadence_pattern.pattern_start_time_timecode": "00:00:03:01",
    "vod_cadence_pattern.pattern_end_time_timecode": "00:01:00:00"
  },
  {
    "vod_cadence_pattern.pattern_start_frame": 102,
    "vod_cadence_pattern.pattern_end_frame": 1800,
    "vod_cadence_pattern.test_id": "1-1",
    "vod_cadence_pattern.cadence_pattern": "2:2:3:3",
    "vod_cadence_pattern.details": "{\"confidence\":100}",
    "vod_cadence_pattern.cadence_pattern_offset": 8,
    "vod_cadence_pattern.pattern_start_time_timecode": "00:00:03:12",
    "vod_cadence_pattern.pattern_end_time_timecode": "00:01:00:00"
  }
]

We can make the following conclusions based on the results above:

• There are two assets in the analysis, one with test ID 1 and one with test ID 1-1
• For asset with test ID 1
  • There were 2 observed cadences
  • One cadence at the beginning of the sequence had low motion and thus a cadence could not be detected
    • Reported as "Unknown (Low Motion)" and was observed between frame 1 to frame 90
  • There was another observed cadence from frame 91 to the end of the asset - frame 1800
    • This cadence was reported as "7:8" with 100% confidence
• For asset with test ID 1-1
  • There were 2 observed cadences
  • One cadence at the beginning of the sequence had low motion and thus a cadence could not be detected
    • Reported as "Unknown (Low Motion)" and was observed between frame 2 to frame 101
  • There was another observed cadence from frame 102 to the end of the asset - frame 1800
    • This cadence was reported as "2:2:3:3" with 100% confidence

Supported view fields

The vod_cadence_pattern view exposes fields to capture metadata about the various video cadence pattern detected (e.g. start/end time, duration, pattern, pattern offset etc.) for the assets included in an analysis. The following section lists the various fields of interest on the vod_cadence_pattern view that can be used in the body of any request sent to the Insights REST API.

And the following filter-only fields:

StreamAware can be used to perform a number of video quality checks. A video quality check focuses on conditions outside of perceptual quality that would result in unaccepable quality, such as freeze frames, black frames, solid color frames, FPS mismatches, unwanted cadence changes and incorrect and/or mismatched metadata.

Using the Stream On-Demand Platform REST API, you can submit a no-reference analysis to find all known video quality problems, as shown below:

curl --location 'https://example.sct.imax.lan/api/vod/v1/analyses' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--data '{
  "content": {
    "title": "Video Quality Check Failures"
  },
  "subjectAssets": [
    {
      "name": "meridian_short_2020.mp4",
      "path": "demo",
      "storageLocation": {
        "type": "PVC",
        "name": "video-files-pvc"
      }
    }
  ],
  "analyzerConfig": {
    "qualityCheckConfig": {
      "enabled": true
    },
    "viewingEnvironments": [
      {
        "device": {
          "name": "oled65c9pua"
        },
        "viewerType": "TYPICAL"
      }
    ]
  }
}'

When you enable the qualityCheckConfig as shown above, the On-Demand Analyzer will perform all the video quality checks allowed by your feature license that are configurable at the per-analysis level, using the default configuration values described here. However, in our example, we want to override the defaults and find video quality check failures at more granular durations while also relaxing the sensitivity of the freeze frame detection, as shown below:

curl --location 'https://example.sct.imax.lan/api/vod/v1/analyses' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--data '{
  "content": {
    "title": "Video Quality Check Failures"
  },
  "subjectAssets": [
    {
      "name": "meridian_short_2020.mp4",
      "path": "demo",
      "storageLocation": {
        "type": "PVC",
        "name": "video-files-pvc"
      }
    }
  ],
  "analyzerConfig": {
    "qualityCheckConfig": {
      "enabled": true,
      "duration": 1,
      "freezeFrame": {
        "sensitivity": 25
      }
    },
    "viewingEnvironments": [
      {
        "device": {
          "name": "oled65c9pua"
        },
        "viewerType": "TYPICAL"
      }
    ]
  }
}'

Let’s check for video quality check failures by querying Insights using the fields on the vod_quality_checks view, as shown below:

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer mF8hbXKbJZxfCTxbPstKjchG8b72g22rMszRQDk6' \
--data '{
  "model": "on_demand",
  "view": "vod_quality_checks",
  "fields": [
    "vod_quality_checks.analysis_uuid",
    "vod_quality_checks.time",
    "vod_quality_checks.title",
    "vod_quality_checks.test_id",
    "vod_quality_checks.test_video",
    "vod_quality_checks.category",
    "vod_quality_checks.type",
    "vod_quality_checks.interval_start_time",
    "vod_quality_checks.interval_start_time_seconds",
    "vod_quality_checks.interval_end_time",
    "vod_quality_checks.interval_end_time_seconds",
    "vod_quality_checks.check_duration_seconds",
    "vod_quality_checks.check_duration_percentage",
    "vod_quality_checks.test_video_duration",
    "vod_quality_checks.test_video_duration_seconds",
    "vod_quality_checks.trimmed_test_video_duration",
    "vod_quality_checks.trimmed_test_video_duration_seconds",
    "vod_quality_checks.details"
  ],
  "filters": {
    "vod_quality_checks.analysis_uuid": "53702b37-b30c-4c79-9c31-409a907a11ac"
  },
  "sorts": [
    "vod_quality_checks.interval_start_time"
  ]
}'

The abbreviated JSON response payload is as follows:

[
  {
    "vod_quality_checks.analysis_uuid": "af2889e7-26f6-46b0-99e8-41848f0e2b00",
    "vod_quality_checks.time": "2023-04-14 11:39:48.244000",
    "vod_quality_checks.title": "UHD HDR Source Validation",
    "vod_quality_checks.test_id": "1",
    "vod_quality_checks.test_video": "file:///mnt/on-201/demo/meridian_short_2020.mp4",
    "vod_quality_checks.category": "Video",
    "vod_quality_checks.type": "Freeze Frame",
    "vod_quality_checks.interval_start_time": "00:01:05.549",
    "vod_quality_checks.interval_start_time_seconds": 65.549,
    "vod_quality_checks.interval_end_time": "00:01:09.102",
    "vod_quality_checks.interval_end_time_seconds": 69.102,
    "vod_quality_checks.test_video_duration": "00:02:52",
    "vod_quality_checks.test_video_duration_seconds": 172.088,
    "vod_quality_checks.trimmed_test_video_duration": "00:02:52",
    "vod_quality_checks.trimmed_test_video_duration_seconds": 172.088,
    "vod_quality_checks.details": null,
    "vod_quality_checks.check_duration_seconds": 3.553,
    "vod_quality_checks.check_duration_percentage": 0.020646413463
  },
  {
    "vod_quality_checks.analysis_uuid": "af2889e7-26f6-46b0-99e8-41848f0e2b00",
    "vod_quality_checks.time": "2023-04-14 11:41:30.477000",
    "vod_quality_checks.title": "UHD HDR Source Validation",
    "vod_quality_checks.test_id": "1",
    "vod_quality_checks.test_video": "file:///mnt/on-201/demo/meridian_short_2020.mp4",
    "vod_quality_checks.category": "Video",
    "vod_quality_checks.type": "Black Frame",
    "vod_quality_checks.interval_start_time": "00:01:18.495",
    "vod_quality_checks.interval_start_time_seconds": 78.495,
    "vod_quality_checks.interval_end_time": "00:01:23.483",
    "vod_quality_checks.interval_end_time_seconds": 83.483,
    "vod_quality_checks.test_video_duration": "00:02:52",
    "vod_quality_checks.test_video_duration_seconds": 172.088,
    "vod_quality_checks.trimmed_test_video_duration": "00:02:52",
    "vod_quality_checks.trimmed_test_video_duration_seconds": 172.088,
    "vod_quality_checks.details": null,
    "vod_quality_checks.check_duration_seconds": 4.988,
    "vod_quality_checks.check_duration_percentage": 0.028985170378
  },
  .
  .
  .
]

Notice in the results above:

• A freeze frame condition was detected at approximately 1:05 and lasted 3.5s.
• A black frame condition was detected at approximately 1:18 and lasted almost 5s.

Cadence Pattern Quality Checks

There are three types of cadence pattern quality checks that can be made:

• Multiple Cadences - a quality check is raised if multiple cadence patterns are observed in the test asset
• Broken Cadence - a quality check is raised if a broken cadence pattern is detected where broken cadence pattern means a dropped field/frame within the cadence pattern
• Allowed Cadences - a quality check is raised if a cadence pattern is observed but not within the list

Using the Stream On-Demand Platform REST API, we can configure the On-Demand Analyzer as shown below:

"analyzerConfig": {
  "enableCadencePatternDetection": true,
  "qualityCheckConfig": {
    "multipleCadences": true,
    "brokenCadence": true,
    "allowedCadences": ["2:2"]
  }
  .
  .
  .
}

In the example above, we do the following:

• enable measurement of Cadence Pattern Detection,
• raise a quality check if multiple cadence patterns are detected,
• raise a quality check if a broken cadence pattern is detected and
• raise a quality check if any cadence pattern that is detected is not “2:2”.

To check if this is the case, the following query to Insights can be made:

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer mF8hbXKbJZxfCTxbPstKjchG8b72g22rMszRQDk6' \
--data '{
  "model": "on_demand",
  "view": "vod_quality_checks",
  "fields": [
    "vod_quality_checks.test_id",
    "vod_quality_checks.interval_start_time",
    "vod_quality_checks.interval_end_time",
    "vod_quality_checks.type",
    "vod_quality_checks.details"
  ],
  "filters": {
    "vod_quality_checks.analysis_uuid": "810e1040-7825-49b2-96a1-0716cc7e1e66"
  },
  "sorts": [
    "vod_quality_checks.interval_start_time_full_label",
    "vod_quality_checks.type"
  ]
}'

When one of the conditions for the quality check is triggered, the response will look similar to the following

[
  {
    "vod_quality_checks.test_id": "1",
    "vod_quality_checks.interval_start_time": "00:00:00.000",
    "vod_quality_checks.interval_end_time": "00:00:30.000",
    "vod_quality_checks.type": "Cadence Not Allowed",
    "vod_quality_checks.details": "Detected cadence 1:1:1:2"
  }
]

We can make the following conclusions based on the above results:

• there is a quality check failure with asset test ID 1,
• the duration of where the quality check is raised is between 0 and 30 seconds
• the quality check type is "Cadence Not Allowed" and
• the detected cadence pattern is "1:1:1:2" which was not in the quality check config allowed cadence pattern list "2:2".

StreamAware can be used to perform a number of audio quality checks.

Audio Loudness Quality Checks

An audio loudness quality check performs level-based and range-based assessments of the audio tracks to ensure that standards of acceptabilty are being met.

There are two types of audio loudness quality checks supported:

• Level Based Quality Checks
• Range Based Quality Checks

Level Based Quality Checks

Level Based Quality Checks only have a single threshold that will trigger a quality check failure. This includes

• MAX_MOMENTARY_LOUDNESS
• MAX_SHORT_TERM_LOUDNESS

Once the threshold is exceeded for the duration amount, a quality check failure is raised.

An example "audio" section for an asset within an analysis submission for a MAX_SHORT_TERM_LOUDNESS quality check can be found below.

{
.
.
.
  "audio": {
    "groups": [
      {
        "loudnessMeasurements": {
          "enabled": true,
          "algorithm": "ITU_BS_1770_3"
        },
        "qualityCheckConfig": {
          "loudnessChecks": [
            {
              "checkType": "MAX_SHORT_TERM_LOUDNESS",
              "enabled": true,
              "threshold": -130.7,
              "duration": 8
            }
          ]
        }
      }
    ],
    "enabled": true
  }
  .
  .
  .
}

The above quality check will fail if the max Short-Term Loudness exceeds -130.7 LUFS for at least 8s.

To check if this is the case, the following query to Insights can be made.

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer mF8hbXKbJZxfCTxbPstKjchG8b72g22rMszRQDk6' \
--data '{
  "model": "on_demand",
  "view": "vod_quality_checks",
  "fields": [
    "vod_quality_checks.test_id",
    "vod_quality_checks.interval_start_time_full_label",
    "vod_quality_checks.interval_end_time_full_label",
    "vod_quality_checks.type",
    "vod_quality_checks.check_duration_seconds",
    "vod_quality_checks.details",
    "vod_quality_checks.stream_index"
  ],
  "filters": {
    "vod_quality_checks.analysis_uuid": "a650491d-5365-41e9-9c86-8d63d9859472"
  },
  "sorts": [
    "vod_quality_checks.interval_start_time_full_label",
    "vod_quality_checks.type"
  ]
}'

When the quality check is triggered, the JSON response will look like this:

[
  {
    "vod_quality_checks.test_id": "1",
    "vod_quality_checks.interval_start_time_full_label": "00:00:00.000",
    "vod_quality_checks.interval_end_time_full_label": "00:02:59.800",
    "vod_quality_checks.type": "High Short-Term Loudness",
    "vod_quality_checks.details": "Short-Term Loudness was above the configured threshold of -130.7 LUFS for longer than 8 seconds",
    "vod_quality_checks.stream_index": 1,
    "vod_quality_checks.check_duration_seconds": 179.8
  }
]

We can make the following conclusions based on the above results:

• There was an exception raised on test_id 1
• A stream index of 1 means that this quality check failure occured on the first observed audio track
  • See the vod_audio_metadata for the ability to link more metadata to this result
• The analyzer measured a Short-Term Loudness above -130.7 LUFS for greater than 8 seconds
• The period of Short-Term Loudness occurred during video playback between 00:00:00.000 and 00:02:59.800

Range Based Quality Checks

Often a single threshold is insufficient and instead a value should remain within a specific range. This type of quality check is supported for the following check types

• MAX_TRUE_PEAK_LEVEL
• MIN_TRUE_PEAK_LEVEL
• MAX_LOUDNESS_RANGE
• MIN_LOUDNESS_RANGE
• MAX_INTEGRATED_LOUDNESS
• MIN_INTEGRATED_LOUDNESS

An example "audio" section for an asset within an analysis submission can be found below.

{
.
.
.
  "audio": {
    "groups": [
      {
        "loudnessMeasurements": {
          "enabled": true,
          "algorithm": "ITU_BS_1770_3"
        },
        "qualityCheckConfig": {
          "loudnessChecks": [
            {
              "checkType": "MIN_LOUDNESS_RANGE",
              "enabled": true,
              "threshold": 10.1
            },
            {
              "checkType": "MAX_LOUDNESS_RANGE",
              "enabled": true,
              "threshold": 15.1
            }
          ]
        }
      }
    ],
    "enabled": true
  }
  .
  .
  .
}

The above quality check will fail if the Loudness Range is outside the range of 10.1 LU to 15.1 LU.

To check if this is the case, the following query to Insights can be made.

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer mF8hbXKbJZxfCTxbPstKjchG8b72g22rMszRQDk6' \
--data '{
  "model": "on_demand",
  "view": "vod_quality_checks",
  "fields": [
    "vod_quality_checks.test_id",
    "vod_quality_checks.interval_start_time_full_label",
    "vod_quality_checks.interval_end_time_full_label",
    "vod_quality_checks.type",
    "vod_quality_checks.details",
    "vod_quality_checks.stream_index"
  ],
  "filters": {
    "vod_quality_checks.analysis_uuid": "a650491d-5365-41e9-9c86-8d63d9859472"
  },
  "sorts": [
    "vod_quality_checks.interval_start_time_full_label",
    "vod_quality_checks.type"
  ]
}'

When the quality check is triggered, the response will look like this:

[
  {
    "vod_quality_checks.test_id": "1",
    "vod_quality_checks.interval_start_time_full_label": null,
    "vod_quality_checks.interval_end_time_full_label": null,
    "vod_quality_checks.type": "Low Loudness Range",
    "vod_quality_checks.details": "Loudness Range of 4.14 LU was below the configured threshold of 10.1 LU",
    "vod_quality_checks.stream_index": 1
  }
]

We can make the following conclusions based on the above results:

• There was an exception raised on test_id 1
• A stream index of 1 means that this quality check failure occured on the first observed audio track
  • See the vod_audio_metadata for the ability to link more metadata to this result
• The analyzer measured a Loudness Range of 4.14 LU upon completion of the analysis
• Loudness Range applies to the entire asset and therefore interval_start_time_full_label and interval_end_time_full_label are null

Audio Artifact Quality Checks

An audio artifact quality check performs level-based and signal-based assessments of the audio tracks to ensure that unwanted noise (eg. clicks, pops and clipping distortion) does not occur during playback.

Clicks and Pops Quality Checks

The On-Demand Analyzer can be configured to determine if clicks and pops are audible in audio tracks belonging to an asset.

An example "audio" section with this quality check enabled for an asset within an analysis submission can be found below.

{
.
.
.
  "audio": {
    "groups": [
      {
        "loudnessMeasurements": {
          "enabled": true,
          "algorithm": "ITU_BS_1770_3"
        },
        "qualityCheckConfig": {
          "clicksAndPopsCheck": [
            {
              "sensitivity": 50,
              "enabled": true,
              "skipStart": 1.25,
              "skipEnd": 1.25
            }
          ]
        }
      }
    ],
    "enabled": true
  }
  .
  .
  .
}

An optional sensitivity parameter is provided to tune the detection algorithm, where a higher value leads to more detections but may result in false positives.

The above quality check event will be triggered if any clicks or pops are detected within an audio track.

To check if this is the case, the following query to Insights can be made.

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer mF8hbXKbJZxfCTxbPstKjchG8b72g22rMszRQDk6' \
--data '{
  "model": "on_demand",
  "view": "vod_quality_checks",
  "fields": [
    "vod_quality_checks.test_id",
    "vod_quality_checks.interval_start_time_full_label",
    "vod_quality_checks.interval_end_time_full_label",
    "vod_quality_checks.type",
    "vod_quality_checks.interval_start_time_seconds",
    "vod_quality_checks.interval_end_time_seconds",
    "vod_quality_checks.check_duration_seconds",
    "vod_quality_checks.details",
    "vod_quality_checks.stream_index"
  ],
  "filters": {
    "vod_quality_checks.analysis_uuid": "a650491d-5365-41e9-9c86-8d63d9859472"
  },
  "sorts": [
    "vod_quality_checks.interval_start_time_full_label",
    "vod_quality_checks.type"
  ]
}'

When the quality check is triggered, the response will look like this:

[
  {
    "vod_quality_checks.test_id": "1",
    "vod_quality_checks.interval_start_time_full_label": null,
    "vod_quality_checks.interval_end_time_full_label": null,
    "vod_quality_checks.type": "Clicks/Pops On Front Center Channel",
    "vod_quality_checks.interval_start_time_seconds": "00:00:04:500",
    "vod_quality_checks.interval_end_time_seconds": "00:00:04:550",
    "vod_quality_checks.check_duration_seconds": 0.05,
    "vod_quality_checks.details": "Click/Pops were detected for the front center channel",
    "vod_quality_checks.stream_index": 1
  }
]

We can make the following conclusions based on the above results:

• There was an exception raised on test_id 1
• A stream index of 1 means that this quality check failure occured on the first observed audio track
  • See the vod_audio_metadata for the ability to link more metadata to this result
• Clicks and/or pops were detected on the front center channel for 0.05 seconds, between the 4.50 and 4.55 second mark of video playback

Clipping Quality Check

The On-Demand Analyzer can be configured to determine if clipping is present in audio tracks belonging to an asset.

An example "audio" section with this quality check enabled for an asset within an analysis submission can be found below.

{
.
.
.
  "audio": {
    "groups": [
      {
        "loudnessMeasurements": {
          "enabled": true,
          "algorithm": "ITU_BS_1770_3"
        },
        "qualityCheckConfig": {
          "clippingCheck": [
            {
              "sensitivity": 50,
              "enabled": true,
              "duration": 1.5,
              "skipStart": 1.25,
              "skipEnd": 1.25
            }
          ]
        }
      }
    ],
    "enabled": true
  }
  .
  .
  .
}

An optional sensitivity parameter is provided to tune the detection algorithm, where a higher value leads to more detections but may result in false positives.

The above quality check event will be triggered if any periods of clipping of at least 1.5 seconds are detected within an audio track.

To check if this is the case, the following query to Insights can be made.

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer mF8hbXKbJZxfCTxbPstKjchG8b72g22rMszRQDk6' \
--data '{
  "model": "on_demand",
  "view": "vod_quality_checks",
  "fields": [
    "vod_quality_checks.test_id",
    "vod_quality_checks.interval_start_time_full_label",
    "vod_quality_checks.interval_end_time_full_label",
    "vod_quality_checks.type",
    "vod_quality_checks.interval_start_time_seconds",
    "vod_quality_checks.interval_end_time_seconds",
    "vod_quality_checks.check_duration_seconds",
    "vod_quality_checks.details",
    "vod_quality_checks.stream_index"
  ],
  "filters": {
    "vod_quality_checks.analysis_uuid": "a650491d-5365-41e9-9c86-8d63d9859472"
  },
  "sorts": [
    "vod_quality_checks.interval_start_time_full_label",
    "vod_quality_checks.type"
  ]
}'

When the quality check is triggered, the response will look like this:

[
  {
    "vod_quality_checks.test_id": "1",
    "vod_quality_checks.interval_start_time_full_label": null,
    "vod_quality_checks.interval_end_time_full_label": null,
    "vod_quality_checks.type": "Clipping On Front Left Channel",
    "vod_quality_checks.interval_start_time_seconds": "00:00:10:500",
    "vod_quality_checks.interval_end_time_seconds": "00:00:12:500",
    "vod_quality_checks.check_duration_seconds": 2.000,
    "vod_quality_checks.details": "Clipping was detected for the front left channel for longer than 1.5 seconds",
    "vod_quality_checks.stream_index": 1
  }
]

We can make the following conclusions based on the above results:

• There was an exception raised on test_id 1
• A stream index of 1 means that this quality check failure occured on the first observed audio track
  • See the vod_audio_metadata for the ability to link more metadata to this result
• Clipping was detected on the front left channel for 2 seconds, between the 10.5 and 12.5 second mark of video playback

StreamAware can be used to perform a number of score checks. A score check allows one to apply numeric thresholds to various VisionScience measurements and metrics and reject assets that don’t meet the threshold.

Using the Stream On-Demand Platform REST API, let’s submit a full-reference analysis for a couple of encoded assets with suspected quality issues, as shown below:

curl --location 'https://example.sct.imax.lan/api/vod/v1/analyses' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--data '{
  "content": {
    "title": "Dog Running FR"
  },
  "referenceAssets": [
    {
      "name": "dog_running.mp4",
      "path": "royalty_free/dog_running/source",
      "storageLocation": {
        "type": "PVC",
        "name": "video-files-pvc"
      }
    }
  ],
  "subjectAssets": [
    {
      "name": "dog_running_1080_h264_qp_31.mp4",
      "path": "royalty_free/dog_running/outputs",
      "storageLocation": {
        "type": "PVC",
        "name": "video-files-pvc"
      },
      "qualityCheckConfig": {
        "scoreChecks": [
          {
            "metric": "SVS",
            "threshold": 85,
            "durationSeconds": 1,
            "viewingEnvironmentIndex": 0
          },
          {
            "metric": "SBS",
            "threshold": 40,
            "durationSeconds": 1,
            "viewingEnvironmentIndex": 0
          }
        ]
      }
    },
    {
      "name": "dog_running_1080_h264_qp_41.mp4",
      "path": "royalty_free/dog_running/outputs",
      "storageLocation": {
        "type": "PVC",
        "name": "video-files-pvc"
      },
      "qualityCheckConfig": {
        "scoreChecks": [
          {
            "metric": "SVS",
            "threshold": 85,
            "durationSeconds": 1,
            "viewingEnvironmentIndex": 0
          },
          {
            "metric": "SBS",
            "threshold": 40,
            "durationSeconds": 1,
            "viewingEnvironmentIndex": 0
          }
        ]
      }
    }
  ],
  "analyzerConfig": {
    "enableBandingDetection": true,
    "viewingEnvironments": [
      {
        "device": {
          "name": "oled65c9pua"
        },
        "viewerType": "TYPICAL"
      }
    ]
  }
}'

Notice in the request above:

• We have configured an XVS (formerly SVS and still referenced this way in the API request) score check on both subject/test assets where we consider it be unacceptable to have any period of more than 1s where the IMAX ViewerScore is less than 85.
• We have configured an XBS (formerly SBS and still referenced this way in the API request) score check on both subject/test assets where we consider it be unacceptable to have any period of more than 1s where the IMAX BandingScore is greater than 40.

As with the other quality checks, let’s use the fields on the vod_quality_checks view to find any score check failures, as shown below:

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer 22srnScNTRYKNTpPpBtMBfTYjHDJqTPkfhQX6TQK' \
--data '{
  "model": "on_demand",
  "view": "vod_quality_checks",
  "fields": [
    "vod_quality_checks.analysis_uuid",
    "vod_quality_checks.time",
    "vod_quality_checks.title",
    "vod_quality_checks.test_id",
    "vod_quality_checks.test_video",
    "vod_quality_checks.category",
    "vod_quality_checks.type",
    "vod_quality_checks.interval_start_time",
    "vod_quality_checks.interval_start_time_seconds",
    "vod_quality_checks.interval_end_time",
    "vod_quality_checks.interval_end_time_seconds",
    "vod_quality_checks.check_duration_seconds",
    "vod_quality_checks.check_duration_percentage",
    "vod_quality_checks.test_video_duration",
    "vod_quality_checks.test_video_duration_seconds",
    "vod_quality_checks.trimmed_test_video_duration",
    "vod_quality_checks.configuration_threshold",
    "vod_quality_checks.trimmed_test_video_duration_seconds"
  ],
  "filters": {
    "vod_quality_checks.analysis_uuid": "5473a508-4dc8-47e2-9c89-792a4fafb061"
  },
  "sorts": [
    "vod_quality_checks.interval_start_time"
  ]
}'

The abbreviated JSON response payload is as follows:

[
  {
    "vod_quality_checks.analysis_uuid": "5473a508-4dc8-47e2-9c89-792a4fafb061",
    "vod_quality_checks.time": "2023-08-29 23:22:40.488000",
    "vod_quality_checks.title": "Dog Running FR",
    "vod_quality_checks.test_id": "1-2",
    "vod_quality_checks.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_41.mp4",
    "vod_quality_checks.category": "Video",
    "vod_quality_checks.type": "SSIMPLUS Viewer Score",
    "vod_quality_checks.interval_start_time": "00:00:00.000",
    "vod_quality_checks.interval_start_time_seconds": 0,
    "vod_quality_checks.interval_end_time": "00:00:19.160",
    "vod_quality_checks.interval_end_time_seconds": 19.16,
    "vod_quality_checks.test_video_duration": "00:00:20",
    "vod_quality_checks.test_video_duration_seconds": 19.68,
    "vod_quality_checks.trimmed_test_video_duration": "00:00:20",
    "vod_quality_checks.configuration_threshold": "85",
    "vod_quality_checks.trimmed_test_video_duration_seconds": 19.68,
    "vod_quality_checks.check_duration_seconds": 19.16,
    "vod_quality_checks.check_duration_percentage": 0.973577235772
  },
  {
    "vod_quality_checks.analysis_uuid": "5473a508-4dc8-47e2-9c89-792a4fafb061",
    "vod_quality_checks.time": "2023-08-29 23:22:24.839000",
    "vod_quality_checks.title": "Dog Running FR",
    "vod_quality_checks.test_id": "1-1",
    "vod_quality_checks.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_31.mp4",
    "vod_quality_checks.category": "Video",
    "vod_quality_checks.type": "SSIMPLUS Viewer Score",
    "vod_quality_checks.interval_start_time": "00:00:00.000",
    "vod_quality_checks.interval_start_time_seconds": 0,
    "vod_quality_checks.interval_end_time": "00:00:05.280",
    "vod_quality_checks.interval_end_time_seconds": 5.28,
    "vod_quality_checks.test_video_duration": "00:00:20",
    "vod_quality_checks.test_video_duration_seconds": 19.68,
    "vod_quality_checks.trimmed_test_video_duration": "00:00:20",
    "vod_quality_checks.configuration_threshold": "85",
    "vod_quality_checks.trimmed_test_video_duration_seconds": 19.68,
    "vod_quality_checks.check_duration_seconds": 5.28,
    "vod_quality_checks.check_duration_percentage": 0.268292682927
  },
  {
    "vod_quality_checks.analysis_uuid": "5473a508-4dc8-47e2-9c89-792a4fafb061",
    "vod_quality_checks.time": "2023-08-29 23:22:30.009000",
    "vod_quality_checks.title": "Dog Running FR",
    "vod_quality_checks.test_id": "1-2",
    "vod_quality_checks.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_41.mp4",
    "vod_quality_checks.category": "Video",
    "vod_quality_checks.type": "SSIMPLUS Banding Score",
    "vod_quality_checks.interval_start_time": "00:00:09.560",
    "vod_quality_checks.interval_start_time_seconds": 9.56,
    "vod_quality_checks.interval_end_time": "00:00:11.240",
    "vod_quality_checks.interval_end_time_seconds": 11.24,
    "vod_quality_checks.test_video_duration": "00:00:20",
    "vod_quality_checks.test_video_duration_seconds": 19.68,
    "vod_quality_checks.trimmed_test_video_duration": "00:00:20",
    "vod_quality_checks.configuration_threshold": "40",
    "vod_quality_checks.trimmed_test_video_duration_seconds": 19.68,
    "vod_quality_checks.check_duration_seconds": 1.68,
    "vod_quality_checks.check_duration_percentage": 0.085365853659
  },
  {
    "vod_quality_checks.analysis_uuid": "5473a508-4dc8-47e2-9c89-792a4fafb061",
    "vod_quality_checks.time": "2023-08-29 23:22:41.502000",
    "vod_quality_checks.title": "Dog Running FR",
    "vod_quality_checks.test_id": "1-1",
    "vod_quality_checks.test_video": "file:///mnt/video-files-pvc/royalty_free/dog_running/outputs/dog_running_1080_h264_qp_31.mp4",
    "vod_quality_checks.category": "Video",
    "vod_quality_checks.type": "SSIMPLUS Viewer Score",
    "vod_quality_checks.interval_start_time": "00:00:16.480",
    "vod_quality_checks.interval_start_time_seconds": 16.48,
    "vod_quality_checks.interval_end_time": "00:00:19.160",
    "vod_quality_checks.interval_end_time_seconds": 19.16,
    "vod_quality_checks.test_video_duration": "00:00:20",
    "vod_quality_checks.test_video_duration_seconds": 19.68,
    "vod_quality_checks.trimmed_test_video_duration": "00:00:20",
    "vod_quality_checks.configuration_threshold": "85",
    "vod_quality_checks.trimmed_test_video_duration_seconds": 19.68,
    "vod_quality_checks.check_duration_seconds": 2.68,
    "vod_quality_checks.check_duration_percentage": 0.136178861789
  }
]

Notice in the results above:

• The dog_running_1080_h264_qp_31.mp4 asset failed the XVS (SVS) score check at two different times for a total of approximately 6s. However, this asset passed the XBS score check which indicates that the level of color banding is acceptable. To know exactly what the XBS (SBS) values were, you would need to use consult the analysis_results view (see here).
• The dog_running_1080_h264_qp_41.mp4 asset failed both the XVS and XBS score checks but its low XVS scores is clearly the main problem as 97% of the asset’s duration was of unacceptable quality, while only 1.68s or 8% of the asset had unacceptable levels of color banding.

Duration filtering and On-Demand Analyzer Configuration

Most quality checks have a duration associated with them, either specified in seconds or frames, after which the triggered condition will cause a warning or failure event to be raised. Refer to the Stream On-Demand Platform REST API for details on configuring quality checks. Once an anlysis has been submitted, changes to any associated quality check would require a new analysis. However, as you’ve already learned, the Insights REST API allows you to filter on any view field, including vod_quality_checks.check_duration_seconds. The implication here is that you can effectively alter the duration of any quality check post-analysis, thereby changing the results of the check, by applying modifiers to a vod_quality_checks.check_duration_seconds filter, as shown in the example below:

.
.
.
"filters": {
  "vod_quality_checks.analysis_uuid": "{{last_analysis_uuid}}"
  "vod_quality_checks.check_duration_seconds": "<5"
}
.
.
.

The caveat here is that you cannot chose a smaller value for the vod_quality_checks.check_duration_seconds field than what was used to configure the On-Demand Analyzer via the REST API.

"analyzerConfig": {
  "qualityCheckConfig": {
    "enabled": true,
    "duration": 1
  }
}

In the example above, we configured the On-Demand Analyzer to apply a quality check with a duration of 1s which is less than the 5s we used in the Insights filter above.

This raises the question of why we don’t always configure all quality checks with the lowest possible duration. While this would lead to the most flexibility in terms of post-analysis filtering using Insights, it may also put a significant amount of additional computational effort on the On-Demand Analyzer itself, thereby slowing down your analyses. The Stream On-Demand Platform REST API has made efforts to pick reasonable defaults for all applicable quality checks but there may be times where you want to lower these in order to give you additional flexibility with your results.

Filtering results based on percentages

In some production scenarios, it may prove difficult to determine, in advance of submitting the analysis, how much of the start and end of the asset, in seconds, you may wish to exempt from quality check analysis. It may be easier to specify amounts of the asset to skip at either end as percentages of the overall length of the asset. Consider, for example, feature length films where it is understood that the first and last 5% of the asset are typically studio logos and credits. Similarly, one may find it useful to define a quality check failure in terms of its duration with respect to the (trimmed) asset length, instead of in absolute seconds. A freeze frame, for example, that lasts 5% of a 30s commercial may not be considered a quality check failure worthy of asset rejection yet a 5% freeze frame in a 120m feature length film surely would. For these types of situations, you are encouraged to filter your quality check results using the following percentage-based fields:

• vod_quality_checks.skip_start_percentage
• vod_quality_checks.skip_end_percentage
• vod_quality_checks.check_duration_percentage

The following is an example query where we’ve skipped the first and last 5% of the asset and told the system to only find video quality check failures that exceed 4.5% of the trimmed/shortened asset length:

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer mF8hbXKbJZxfCTxbPstKjchG8b72g22rMszRQDk6' \
--data '{
  "model": "on_demand",
  "view": "vod_quality_checks",
  "fields": [
    "vod_quality_checks.analysis_uuid",
    "vod_quality_checks.time",
    "vod_quality_checks.comparison_analysis_uuid",
    "vod_quality_checks.title",
    "vod_quality_checks.organization",
    "vod_quality_checks.site",
    "vod_quality_checks.test_id",
    "vod_quality_checks.test_video",
    "vod_quality_checks.stream_index",
    "vod_quality_checks.category",
    "vod_quality_checks.type",
    "vod_quality_checks.interval_start_time",
    "vod_quality_checks.interval_start_time_seconds",
    "vod_quality_checks.interval_end_time",
    "vod_quality_checks.interval_end_time_seconds",
    "vod_quality_checks.check_duration_seconds",
    "vod_quality_checks.check_duration_percentage",
    "vod_quality_checks.test_video_duration",
    "vod_quality_checks.test_video_duration_seconds",
    "vod_quality_checks.trimmed_test_video_duration",
    "vod_quality_checks.trimmed_test_video_duration_seconds",
    "vod_quality_checks.details"
  ],
  "filters": {
    "vod_quality_checks.analysis_uuid": "9bac5d71-83ad-43f5-b3c5-f5a076dd7be7",
    "vod_quality_checks.check_duration_percentage": ">=4.5",
    "vod_quality_checks.skip_start_percentage": "5",
    "vod_quality_checks.skip_end_percentage": "5",
    "vod_quality_checks.category": "video",
  },
  "sorts": [
    "vod_quality_checks.interval_start_time"
  ]
}'

The following is an example response for the request above:

[
  {
    "vod_quality_checks.analysis_uuid": "9bac5d71-83ad-43f5-b3c5-f5a076dd7be7",
    "vod_quality_checks.time": "2021-02-24 17:34:13.998000",
    "vod_quality_checks.comparison_analysis_uuid": null,
    "vod_quality_checks.title": "Curated Big Buck Bunny with Quality Check Failures",
    "vod_quality_checks.organization": "IMAX",
    "vod_quality_checks.site": "VOD Testing",
    "vod_quality_checks.test_id": "1",
    "vod_quality_checks.test_video": "/mnt/videos/test/big_buck_bunny_problematic.mp4",
    "vod_quality_checks.stream_index": 1,
    "vod_quality_checks.category": "video",
    "vod_quality_checks.type": "Solid Color Frame",
    "vod_quality_checks.interval_start_time": "00:00:20.062",
    "vod_quality_checks.interval_start_time_seconds": 20.062,
    "vod_quality_checks.interval_end_time": "00:00:25.025",
    "vod_quality_checks.interval_end_time_seconds": 25.025,
    "vod_quality_checks.check_duration_seconds": 4.963,
    "vod_quality_checks.check_duration_percentage": 4.58917498,
    "vod_quality_checks.test_video_duration": "00:02:00",
    "vod_quality_checks.test_video_duration_seconds": 120.162,
    "vod_quality_checks.trimmed_test_video_duration": "00:01:48",
    "vod_quality_checks.trimmed_test_video_duration_seconds": 108.1458,
    "vod_quality_checks.details": null
  },
  {
    "vod_quality_checks.analysis_uuid": "9bac5d71-83ad-43f5-b3c5-f5a076dd7be7",
    "vod_quality_checks.time": "2021-02-24 17:35:24.607000",
    "vod_quality_checks.comparison_analysis_uuid": null,
    "vod_quality_checks.title": "Curated Big Buck Bunny with Quality Check Failures",
    "vod_quality_checks.organization": "IMAX",
    "vod_quality_checks.site": "VOD Testing",
    "vod_quality_checks.test_id": "1",
    "vod_quality_checks.test_video": "/mnt/videos/test/big_buck_bunny_problematic.mp4",
    "vod_quality_checks.stream_index": 1,
    "vod_quality_checks.category": "video",
    "vod_quality_checks.type": "Freeze Frame",
    "vod_quality_checks.interval_start_time": "00:01:20.080",
    "vod_quality_checks.interval_start_time_seconds": 80.08,
    "vod_quality_checks.interval_end_time": "00:01:25.085",
    "vod_quality_checks.interval_end_time_seconds": 85.085,
    "vod_quality_checks.check_duration_seconds": 5.005,
    "vod_quality_checks.check_duration_percentage": 4.62801144,
    "vod_quality_checks.test_video_duration": "00:02:00",
    "vod_quality_checks.test_video_duration_seconds": 120.162,
    "vod_quality_checks.trimmed_test_video_duration": "00:01:48",
    "vod_quality_checks.trimmed_test_video_duration_seconds": 108.1458,
    "vod_quality_checks.details": null
  },
  {
    "vod_quality_checks.analysis_uuid": "9bac5d71-83ad-43f5-b3c5-f5a076dd7be7",
    "vod_quality_checks.time": "2021-02-24 17:35:34.671000",
    "vod_quality_checks.comparison_analysis_uuid": null,
    "vod_quality_checks.title": "Curated Big Buck Bunny with Quality Check Failures",
    "vod_quality_checks.organization": "IMAX",
    "vod_quality_checks.site": "VOD Testing",
    "vod_quality_checks.test_id": "1",
    "vod_quality_checks.test_video": "/mnt/videos/test/big_buck_bunny_problematic.mp4",
    "vod_quality_checks.stream_index": 1,
    "vod_quality_checks.category": "video",
    "vod_quality_checks.type": "Black Frame",
    "vod_quality_checks.interval_start_time": "00:01:30.132",
    "vod_quality_checks.interval_start_time_seconds": 90.132,
    "vod_quality_checks.interval_end_time": "00:01:35.095",
    "vod_quality_checks.interval_end_time_seconds": 95.095,
    "vod_quality_checks.check_duration_seconds": 4.963,
    "vod_quality_checks.check_duration_percentage": 4.58917498,
    "vod_quality_checks.test_video_duration": "00:02:00",
    "vod_quality_checks.test_video_duration_seconds": 120.162,
    "vod_quality_checks.trimmed_test_video_duration": "00:01:48",
    "vod_quality_checks.trimmed_test_video_duration_seconds": 108.1458,
    "vod_quality_checks.details": null
  }
]

Notice that the vod_quality_checks.check_duration_percentage value reported for each video quality check failure exceeds 4.5% of the length of the asset, after skipping the first and last 5% (i.e. the trimmed asset length).

The vod_quality_checks view exposes fields to capture the various video quality checks, asset audio quality checks and asset score checks supported by StreamAware. The following subsections lists the various fields of interest on the vod_quality_checks view that can be used in the body of any request sent to the Insights REST API.

To fetch the video metadata associated with an analysis or optimization, we use vod_video_metadata view, as shown below:

curl --location 'https://insights.sct.imax.com/api/4.0/queries/run/json?cache=false' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer btPfpMGhxZTZYmQ7gH5tfGCFMRM93FFqC7kZsPj8' \
--data '{
  "model": "on_demand",
  "view": "vod_video_metadata",
  "fields": [
    "vod_video_metadata.analysis_uuid",
    "vod_video_metadata.title",
    "vod_video_metadata.organization",
    "vod_video_metadata.site",
    "vod_video_metadata.test_id",
    "vod_video_metadata.test_video",
    "vod_video_metadata.stream_index",
    "vod_video_metadata.resolution",
    "vod_video_metadata.resolution_name",
    "vod_video_metadata.vertical_resolution_pixels",
    "vod_video_metadata.horizontal_resolution_pixels",
    "vod_video_metadata.dynamic_range",
    "vod_video_metadata.dynamic_range_format",
    "vod_video_metadata.encoder_decoder",
    "vod_video_metadata.encoder_decoder_profile",
    "vod_video_metadata.frame_rate",
    "vod_video_metadata.bit_depth",
    "vod_video_metadata.time_base",
    "vod_video_metadata.time_base_numerator",
    "vod_video_metadata.time_base_denominator",
    "vod_video_metadata.storage_aspect_ratio",
    "vod_video_metadata.storage_aspect_ratio_numerator",
    "vod_video_metadata.storage_aspect_ratio_denominator",
    "vod_video_metadata.frame_count",
    "vod_video_metadata.color_primaries",
    "vod_video_metadata.color_space",
    "vod_video_metadata.color_transfer_characteristics",
    "vod_video_metadata.max_content_light_level",
    "vod_video_metadata.max_frame_average_light_level",
    "vod_video_metadata.start_presentation_timestamp"
  ],
  "filters": {
    "vod_video_metadata.analysis_uuid": "1486832c-a022-43a2-a4d7-4f75a1a3cc52"
  }
}'

The following is an example JSON response for the request above:

[
  {
    "vod_video_metadata.analysis_uuid": "1486832c-a022-43a2-a4d7-4f75a1a3cc52",
    "vod_video_metadata.title": "HDR NR Test",
    "vod_video_metadata.organization": "IMAX",
    "vod_video_metadata.site": "VOD Demo",
    "vod_video_metadata.test_id": "1",
    "vod_video_metadata.test_video": "file:///mnt/on-201/UHD-HDR-MP4-samsung-demo-reel/samsung-demo-reel-version-a.mp4",
    "vod_video_metadata.stream_index": 0,
    "vod_video_metadata.resolution": "3840 x 2160",
    "vod_video_metadata.resolution_name": "4K UHD",
    "vod_video_metadata.vertical_resolution_pixels": 2160,
    "vod_video_metadata.horizontal_resolution_pixels": 3840,
    "vod_video_metadata.dynamic_range": "HDR",
    "vod_video_metadata.dynamic_range_format": "HDR10",
    "vod_video_metadata.encoder_decoder": "HEVC",
    "vod_video_metadata.encoder_decoder_profile": "Main 10",
    "vod_video_metadata.frame_rate": 24,
    "vod_video_metadata.bit_depth": 10,
    "vod_video_metadata.time_base": "1 / 12288",
    "vod_video_metadata.time_base_numerator": 1,
    "vod_video_metadata.time_base_denominator": 12288,
    "vod_video_metadata.storage_aspect_ratio": "1:1",
    "vod_video_metadata.storage_aspect_ratio_numerator": 1,
    "vod_video_metadata.storage_aspect_ratio_denominator": 1,
    "vod_video_metadata.frame_count": 241,
    "vod_video_metadata.color_primaries": "bt2020",
    "vod_video_metadata.color_space": "bt2020nc",
    "vod_video_metadata.color_transfer_characteristics": "smpte2084",
    "vod_video_metadata.max_content_light_level": 1000,
    "vod_video_metadata.max_frame_average_light_level": 1000,
    "vod_video_metadata.start_presentation_timestamp": 6144
  }
]

Supported view fields

The vod_video_metadata view exposes fields to capture the video metadata (e.g. title, resolution, dynamic range, encoder, frame rate, time base, aspect ratio, frame count, color primaries and space, color transfer characteristics and content/frame light levels) for the assets included in an analysis. The following section lists the various fields of interest on the vod_video_metadata view that can be used in the body of any request sent to the Insights REST API.