How to Test GStreamer Pipelines with gst-validate Scenarios

The gst-validate utility allows for the detection of known issues in GStreamer pipelines, and it’s part of the developer tools the GStreamer community offers through the gst-devtols modules. In this guide, I will demonstrate how to create playback scenarios to test a pipeline’s reaction to a new set of controlling actions.

There are a couple of common and not-so-common scenarios that are already included with the GStreamer validate suite. This set allows for the identification of known error conditions on a running pipeline subjected to the actions the provided scenarios express. Now, what if you want to inspect the reaction of your pipeline to a new set of actions? Then, you need a new scenario that describes it.

What’s in a Scenario?

Scenarios are built from serialized actions on a .scenario file. These actions and their parameters are expressed using a GstStructure-based text format, and includes the following core actions: seek, pause, switch-track, wait, dot-pipeline, set-feature-rank, set-state, set-property, set-debug-threshold, emit-signal, disable-plugin. It also includes a few others implemented in gst validate plugins, like validatefaultinjection, corrupt-socket-recv, validategtk, gtk-put-event, validate-launcher, set-subtitle. This introductory article will deal with only a few of them to provide an idea of how everything fits together.

Let’s say we want to test the resilience of a pipeline to a flush seek to 10s after the pipeline is paused at playback-time 1s for 5s. Lets also say that we want to stop playback at 20s. A preliminary version of our scenario file will look like the following:

description, seek=true, need-clock-sync=true
 pause, name=Our-pause, playback-time=1.0, duration=5.0
 seek, name=Our-forward-seek, playback-time=6.0, start=10.0, flags=accurate+flush
 stop, playback-time=20.0

Whenever you run into a new action name (first member on each line), use the following command to get a full description of what the action does and what parameters you can use to control it:

gst-validate-1.0 -t <action name>

For example, the description line describes two important aspects of the scenario:

  • seek=true – This scenario performs seek operations
  • need-clock-sync=true – The scenario execution needs to be synchronized to the pipeline’s clock

Pretty straightforward isn’t it? Now, it’s necessary to add some boilerplate for this scenario to be fully operational. For example, if the media has a duration that doesn’t allow a proper forward seek from 5s to 10s, it’s likely this scenario won’t provide the desired results. For the sake of this example, let’s inform the scenario runner that the test media should have a minimum duration of 30 seconds. This can be done with the following description line:

description, seek=true, need-clock-sync=true, min-media-duration=30s

The scenario parsing system assumes each line of the scenario file represents an action, but it’s also possible to use backslashes to enter multi-line actions. Here’s how the scenario file looks with a multi-line description (sic) action and a long summary:

description, seek=true, need-clock-sync=true, min-media-duration=30s, \
 summary="This example scenario seeks forward to 10s after pausing playback for 5. \
 Then, it stops the execution at 20s"
 pause, name=Our-pause, playback-time=1.0, duration=5.0
 seek, name=Our-forward-seek, playback-time=6.0, start=10.0, flags=accurate+flush
 stop, playback-time=20.0

By the way, I’ve been carrying this one for the entire article so far, but description is not really an action. Sorry to break it to you, but please consider this an implementation detail you’ll have to live with. It’s referred to as an action in the documentation, so I’m following suit.

Test a GStreamer Pipeline

Now that we have our scenario script, it needs to be saved to a file named after the scenario. In this particular example, we’ll save it as test-seek.scenario and pass this file name (without extension) directly to gst-validate for testing a pipeline:

gst-validate-1.0 --set-scenario test-seek filesrc location=test20sVP9.mkv ! matroskademux ! vp9dec ! autovideosink

Beyond passing the full PATH to the scenario file with –set-scenario, it’s also possible to put this file somewhere else and set the GST_VALIDATE_SCENARIOS_PATH environment variable to point to that location. These are not the only options, but they should be enough to get going with the example.

You can try this procedure out on your own pipeline, but if you want to follow this guide to the letter you will need to download the test20sVP9.mkv sample file. Additionally, if the syntax used to express the pipeline is not something you are familiar with, you can find our best attempt at an explanation on the gst-launch documentation page.

And that’s it! Gst-validate will subject the pipeline to the actions that were scripted via the scenario file, and will provide a report on whether any of the built-on gst-validate tests failed. Beyond a simple statement of problem, sometimes gst-validate will even provide hints towards a full diagnostic.

In this article, we briefly explored the scenario-writing aspect of gst-validate. While the suite of tools goes far beyond, a good understanding of the steps required to express action scenarios provides a great foray into this GStreamer testing system. Moving forward, If you want additional examples, take a look at the real-life examples included with gst-validate, and consider contributing your own by submitting a patch to the gst-devtools component on the project’s Bugzilla.

Good luck!

Author: Reynaldo Verdejo

Husband, Dad, and Multimedia FOSS developer by trade in between. His work is primarily found in GStreamer and FFmpeg, but he has also contributed to an assorted list of free and open source software during his more than a decade-long software development career.