Introducing Cairo GL ES v3

A while back, I landed a change to update Cairo’s GL backend to support OpenGL ES version 3.0. In this blog post I’ll describe what this is, what makes it important, and what future steps can be taken.

What is Cairo?

Cairo is a drawing library that provides high quality 2D rendering of vector graphics. It’s claim to fame is a focus on making what you see on the screen identical to what gets printed on the printer. It’s raison d’être is to create SVG and PDF files with fancy graphics and nicely anti-aliased text. Cairo provides a stable, well-tested, well-documented API for applications to build against; historically, the list of applications and products using Cairo is impressively long.

Rendering performance is also important to Cairo, which it addresses by making provisions for ‘rendering backends.’ The backends tap into platform-specific underlying 2D graphics systems as output targets; the win32 backend ties into Window’s GDI services, xlib interfaces with the X window system’s windows and pixmaps, and so on.

However, computer hardware advancements have focused on optimization of 3D functionality and performance far more than 2D, to such a degree that it is faster to delegate 2D rendering to the 3D GPU. If nothing else, on most systems the GPU is quite powerful – essentially a whole secondary computer inside your computer – so offloading any computation to it can prove to be a win. Particularly if it can utilize the graphics optimization features, this is what the Cairo OpenGL backend aims to achieve.

What is OpenGL?

OpenGL, or the Open Graphics Library, is a high performance 3D vector graphics API for doing hardware-accelerated rendering. It’s been under constant development since 1991 and is an industry-wide standard. OpenGL ES is a variant of the OpenGL API, originally designed to be more suitable for embedded devices. OpenGL ES has evolved over time to approach feature parity with “regular” OpenGL and expanded to support a vast array of hardware, including the desktop platforms that OpenGL targets.

Cairo’s OpenGL backend (supporting GLX, WGL, and EGL) promises use of the underlying 3D tech for hardware-accelerating 2D drawings, although the efficiency of translating Cairo’s canvas model to the GPL has not always panned out as well as hoped. There is also an OpenVG backend, though it hasn’t received quite as much development attention.  Cairo’s OpenGL implementation also includes support for OpenGL ES v2.

Say Hello to Cairo GL ES v3

The OpenGL ES 3.0 spec was released five years after ES 2.0 and packs quite a bit of new functionality. Numerous enhancements were made to the rendering pipeline that, if leveraged properly in Cairo, could provide much faster rendering performance. Texturing functionality was greatly enhanced, and while Cairo’s basic graphics operations aren’t heavy users of texturing, the enhancements could provide a significant functional and performance boon for more advanced drawing.

These new functions don’t come automatically though. The first step, which has landed in Cairo’s trunk, simply detects if GL ES 3.0 is available; if so, it creates a surface that’s able to talk with the GPU with this version of the protocol. Additional coding to leverage the new functionality will be introduced later as follow-on work.

This new GLESv3 support can be turned on at compile time via the –enable-glesv3 configure flag. A configure flag can also be used to select a particular GL variant. For instance, to use EGL with GL ES 3:

./configure --enable-egl --enable-glesv3

Samsung initially implemented the GLESv3 support for the Tizen platform, and it had been proposed for Cairo upstream in 2013, but this work was interleaved with a number of unrelated features and optimizations, so wasn’t in a directly-implementable state. I extracted the baseline GLESv3 support from this work and did the polishing needed to land it. The additional features and optimizations from that branch will leverage GL ES 3.0’s capabilities to provide improved performance over GL ES 2.0, but extracting them and rebasing them remains for future work.

Looking Ahead

Since the original (experimental) introduction of cairo-gl as a means to leverage the power of the GPU, graphics technology has increased its capabilities. The promise of 3D-hardware rendering to 2D drawing is even bigger than before; yet, to tap it Cairo must advance forward to adopt newer APIs, algorithms, and optimizations. Adopting GLESv3 support, while perhaps not exactly aggressive, is still necessary to keep the leading edge of technology within sight.

Much can be done to further leverage, optimize, and polish Cairo’s OpenGL ES 3.0 support, and we can also look to adopting even newer versions: OpenGL ES 3.1 was published in March 2014 to add compute shaders and independent vertex and fragment shaders; OpenGL ES 3.2 became available in August 2015, bringing functionality to support complex scenes and various optimizations for advanced usages.

Beyond OpenGL, there is also the new Vulkan 3D API to consider. This is not advertised as having any particular advantage for 2D rendering over what OpenGL can do, although it does include hardware management and parallelization advancements that theoretically could bring some benefit. However, since it’s a newer technology, support on older hardware may be an issue, at least for a while. Further, it remains to be seen if Cairo’s backend architecture would leverage or limit Vulkan’s capabilities.

If you use Cairo, take a look at what it can do with OpenGL ES v3 and let us know what you think!

Adding Community-Driven Wayland Support to Servo

It’s been some time since the last Servo article on the OSG blog, but this has no relation to the speed at which the browser engine’s development has been progressing.

In the last post, the Offscreen Rendering (OSR) integration status was explored, culminating in both some code snippets as well as videos of an embedded browser application. That post can be considered the foundation for the recently-tweeted screenshot of Servo running with Wayland support.

The Technical Hurdle

Before delving into the technical details of Wayland integration, it’s important to know the background of Servo’s rendering stack. In order to provide support for a broad range of platforms, Servo uses the rust-layers library to create hardware-accelerated drawing abstractions for Android, Linux, MacOS, and Windows; this enables the browser engine to use a unified API for all its internal compositor painting.

Under Linux, rust-layers uses GLX, the OpenGL extension for X11, to perform hardware accelerated drawing in the commonly-used Xorg display server. This is the go-to method for using the GPU when running Xorg on desktop Linux.

GLX is problematic in some cases, however. First, GLX is X-specific, meaning that it cannot run under other display servers such as Wayland, or directly in the framebuffer, such as on Raspberry Pi devices. Requiring GLX in this case becomes an issue for anyone who wants to use Servo on embedded devices that run non-Android Linux variants. Additionally, continuous integration tests involving GLX have been prone to intermittent failures for some time, so replacing it with something else could yield benefits in this area. Finally, GLX code is also very driver-specific, and special cases are often needed for different platforms, hardware, and even driver versions.

Based on this information, using Servo on Wayland seems to be quite impossible. There have been tickets open for some time on GitHub related to converting Servo’s graphics stack over from GLX to EGL, a newer OpenGL variant which is usable outside of X, but no progress had been made on this transition over a long period of time.

Solving the Problem Through Education

Enter Dr. Gehringer’s Object Oriented Design class at NC State University. For the past few terms (link, link, link), this professor has been requiring students to contribute to open source projects as a practical application of their learning over the course of the term. This term, one of the open source project choices available was to refactor the OpenGL usage in Servo’s graphics stack, effectively adding support for EGL to all Linux-based systems.

The task was quite difficult for the students; “it took some time trying to figure some things out, as the Servo and Rust layer code-bases are really huge, but after some time, most of us could figure out where things were.” wrote Prashant Gupta, a student involved in the project. Considering that Servo consists of over 800k lines of Rust code and nearly 1.2 million lines of C/C++, that’s quite a feat!

While the rust-layers library may seem small by comparison to the main repository at around 3,000 lines of code, it’s important to consider that it’s written in a language that university students are unlikely to be familiar with, and the code exclusively deals with complex graphics APIs. “The redesign ended up being fairly daunting for newbies, because it required a lot of carefully interconnected code to be changed […]”, commented Neal O’Hara, another student from the team.

This isn’t to say that the project was set adrift with no guidance or assistance, however. Prashant was quick to note that “we were helped a lot […] through the IRC channel of the Mozilla group, and they were extremely prompt in helping us out in any way they could”, and Neal confirmed that “for the actual coding process, we had a lot of help […] which allowed us to quickly find the code to change, and work through cascaded errors.” With the final resulting pull request weighing in at 30 commits made over the course of a week; it’s no small feat that they managed this much – less than two months after having chosen to undertake this project – especially considering that none of the students had ever worked with the Rust language before.

The Key to It All: Community-Supported Learning

Mozilla, in particular the Servo project, is no stranger to student contributors, so it’s no surprise that help was readily available; Servo has been receiving student contributions for some time, and from a number of institutions with differing backgrounds. For example, Servo’s WebGL implementation was largely created by a student from the University of Salamanca for a GSoC-like project that’s associated with academic institutions.

On the topic of getting students involved with Servo and Open Source, Josh Matthews was quoted as saying: “I think all parties get a lot out of the projects – we regularly get to interact with newcomers who expose ways we can improve our onboarding experience, and we receive significant contributions that improve Servo. The students get a real-world experience contributing to a huge codebase in a cutting-edge language and have something to put on their resumes. It can certainly be a lot of work, but both the process and the results are valuable every term, so I feel that it’s well worth it. One of the students this term described it as ‘the most challenging and most rewarding project’ that she had ever been part of.”

As for applying the students’ work in order to run Servo within Wayland, this was a fairly trivial task by comparison. After the previously-described work running Servo embedded within an application using the OSR API for rendering, the same application more-or-less worked out of the box for the purpose of taking Wayland screenshots such as this:

Adding Community-Driven Wayland Support to Servo - WaylandScreenshot

This endeavor has been an immense success and we look forward to future involvement from academia.

Footnote: the first question that was asked after the story hit the twittersphere and news sites was whether the application was using Glutin to create the window for drawing in Wayland. To be short: no. After trying to run the Glutin port, the only output was an error about DISPLAY not being set; given that EFL is easier to work with than Glutin – based on available time and expertise – and that the reference embedding application uses EFL, it was an easy choice to not dig further into this issue and instead use something which has been known to work well in Wayland since 2012.It’s worth noting, however, that there is ongoing work to improve Wayland support in Glutin, so Servo and other Rust applications may be able to run using this port in the near future.

Anshita Sayal, another student from the team involved with the EGL refactoring project in Servo, was unable to be reached for comments.

Guest Author: Josh Matthews

Josh Matthews, Senior Platform Engineer at Mozilla, is the coordinator of Servo’s DOM implementation and is constantly on the lookout for ways to engage with students and volunteers more effectively.

Guest Author: Lars Bergstrom

Lars Bergstrom is a researcher at Mozilla, currently working on the servo parallel web browser project.He received his Ph.D. from the University of Chicago‘s Computer Science department, studying under Dr. John Reppy. His Master’s paper was on the implementation of analysis and optimization passes in our parallel compiler, Manticore. His Ph.D. research was on how to add mutation safely and efficiently into a functional parallel programming language. He has also been doing work on a runtime, garbage collector, and most recently some extensions to control-flow analysis. Before that, he was a manager and a developer at Microsoft in the Visual Studio organization, working on next-generation software development tools technology out at the Redmond, WA offices.