Exynos SoCs have an IP block known as the Multi-Format Codec (MFC) that allows them to do hardware accelerated video encoding/decoding, and the mainline kernel has a s5p-mfc Video for Linux2 (V4L2) driver that supports the MFC. The s5p-mfc driver is a Memory-to-Memory (M2M) V4L2 driver, it’s called M2M because the kernel moves video buffers from an output queue to a capture queue. The user-space enqueues buffers into the output queue, then the kernel passes these buffers to the MFC where they are converted and put it in the capture queue so the user-space can dequeue them.
The GStreamer (gst) multimedia framework supports V4L2 M2M devices, but only for decoders the v4l2videodec element supports. Randy Li is working to also support M2M encoders in GStreamer (v4l2videoenc), but this hasn’t landed in upstream GStreamer yet.
This post will explain how to use GStreamer and the Linux mainline kernel to do hardware video decoding on an Exynos based machine.
Install a Mainline Kernel on an Exynos Machine
The first step is to install a mainline kernel in the Exynos board. We have been working lately to make the s5p-mfc driver more stable and to have everything enabled in by default, so the Linux kernel version used should be at least v4.8-rc1 and the defconfig used should be exynos_defconfig. I’ve made previous blog posts that explain how to cross compile and install a mainline kernel on different Exynos-based machines, including Exynos5 Odroid boards and Exynos5 Chromebooks, so refer to these to learn how to do this.
GStreamer Uninstalled Setup
The GStreamer package in most Linux distributions doesn’t come with support for the v4l2videodec gst element enabled, so GStreamer should be built from source to do hardware accelerated video decoding. Fortunately, GStreamer has the gst-uninstalled script to setup a development environment that allows the use of gst elements that were built from source and not installed in the system. There are many guides on how to use gst-uninstalled, including this great tutorial from Arun Raghavan.
When building the gst-plugins-good module, the –enable-v4l2-probe configure option must be enabled to allow GStreamer to probe v4l2 devices. This is needed because v4l2 device drivers can be probed in different orders, meaning the video device nodes can change. This option needs to be enabled explicitly before building gst-plugins-good:
$ cd gst-plugins-good $ ./configure --enable-v4l2-probe $ make
Find the Video Device Node for MFC Decoding
At this point, the s5p-mfc driver can be used to do hardware accelerated video decoding. You first need to know which video device nodes are for the s5p-mfc video decoder; gst v4l2videodec expects to use a gst element named v4l2videoNdec, where N is the video device node number.
First, get the video device nodes the s5p-mfc driver has registered; the v4l2-ctl command from the v4l-utils can be use for this:
$ v4l2-ctl --list-devices s5p-mfc-dec (platform:11000000.codec): /dev/video5 /dev/video6
There are two device nodes the driver has registered, v4l2-ctl can also be used to know more about these nodes:
$ v4l2-ctl --info -d /dev/video5 Driver Info (not using libv4l2): Driver name : s5p-mfc Card type : s5p-mfc-dec Bus info : platform:11000000.codec Driver version: 4.8.0 Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format
$ v4l2-ctl --info -d /dev/video6 Driver Info (not using libv4l2): Driver name : s5p-mfc Card type : s5p-mfc-enc Bus info : platform:11000000.codec Driver version: 4.8.0 Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format
The video device node for the decoder is /dev/video5 in this case, there should be a v4l2video5dec gst element that should be used.
$ gst-inspect-1.0 v4l2video5dec Factory Details: Rank primary + 1 (257) Long-name V4L2 Video Decoder Klass Codec/Decoder/Video Description Decode video streams via V4L2 API Author Nicolas Dufresne Plugin Details: Name video4linux2 Description elements for Video 4 Linux ...
MFC supports a variety of video formats for decoding and produces decoded NV12/NV21 YUV frames. The v4l2-ctl tool can be used with the option –list-formats-out to list all supported formats.
$ v4l2-ctl -d /dev/video5 --list-formats-out ioctl: VIDIOC_ENUM_FMT Index : 0 Type : Video Output Multiplanar Pixel Format: 'H264' (compressed) Name : H.264 Index : 1 Type : Video Output Multiplanar Pixel Format: 'M264' (compressed) Name : H.264 MVC Index : 2 Type : Video Output Multiplanar Pixel Format: 'H263' (compressed) Name : H.263 ...
Now that the video device node for the decoder and its corresponding v4l2videodec gst element are known, a supported video (e.g. H.264) can be decoded using GStreamer such as with this example pipeline:
$ gst-launch-1.0 filesrc location=test.mov ! qtdemux ! h264parse ! v4l2video5dec ! videoconvert ! kmssink