This is the first article in a two part series on compositing with Compiz and Enlightenment.
I recently implemented a proof of concept for Compiz effect modules in the Enlightenment compositor. In this article, I will explore the methodology behind the implementation and cover some of the AWESOME hacks that made it possible.
Replacing Default Compositor Behaviors with Compiz
To start, let’s examine how Compiz operates. There is an OpenGL extension that allows an X11 pixmap to be bound to a texture. Compiz operates based on the principle that the compositor can render this texture in any way, at any time, and the underlying X11 input window will be moved quietly to wherever the rendered content ends up. This creates the illusion that the entire window (input and output) is moving in an irregular manner, when in reality, the window’s output is moving around and the window’s input region is moved later. An possible comparison is the shell game: the compositor always knows the state of everything, but the observer is totally fooled.
Compiz uses a callback interception system to provide module support. It works by allowing modules to insert callbacks at specific points in the compositor’s operations, replacing the default operation function with one provided by the module. For example, the wobbly window effect occurs when the corresponding module hijacks callbacks for window movement to determine velocity and find the anchor point. It then takes over the pre-render call for the window to manipulate the vertices of the window’s texture before it is rendered. By performing matrix transforms on the vertices, the window appears to wobble around as though the plane it lies upon is constantly shifting even though it isn’t.
Rendering within the Compiz compositor is done through a combination of per-screen damage tracking and occlusion detection. There is no concept of “object” damages, and everything renders directly to the screen. A buffer swap occurs in the event of a full screen redraw, but in all other cases, the damaged buffer region is copied onto the front buffer. If the window manager component of Compiz determines that the window (or part of the window) is not visible, then the non-visible region is not drawn.
Plugging Compiz into Enlightenment
At the outset of this project, my goal was simple: determine whether or not it was possible to load unmodified Compiz effect plugins and have them work “as expected” in Enlightenment. The first use case was, of course, the wobbly plugin. After considering a number of ways to NIH a fake Compiz layer, I decided to skip that process (for once) and instead put all of Compiz-core into a module for Enlightenment. The last Compiz release (0.8.x) was ideal for this since, amazingly enough, it did not use any external toolkit, only libX11 and OpenGL. After copying all the source files into the module tree, the hacks began in earnest.
The first thing to understand about Enlightenment integration is that all display server events are completely abstracted at the module level. This means that while it’s still possible for a developer to create handlers explicitly for X11 events, Enlightenment translates all display server events into a set of general events. When writing modules that are expected to function in both X11 and Wayland environments this is good, but when trying to interface with Compiz –which only handles X11 events, and is both a compositor and a window manager– this didn’t seem like it would be the easiest to work with.
In order to make this work, I ended up completely sandboxing all of Compiz. It effectively has no knowledge of anything happening on the system, and it’s only allowed to have CPU time when various functions are called. The sandbox works by handling various events and callbacks within Enlightenment and synthesizing fake X11 events to send to Compiz.
For a more concrete example, an XDamageNotifyEvent is created from the “damage” smart callback on a compositor object. All events that are necessary for rendering are translated within the Enlightenment module integration layer, and are then sent to Compiz to be processed before the next frame is drawn. The Compiz event processing loop is manually triggered during the canvas pre-render, meaning that the time spent on Compiz contexts is kept to a minimum.
Expanding This Work
By far, the most complex part ended up being managing the rendering for Compiz, and the next post in this series will explain this and Enlightenment integration in more detail. For a quick summary: when Compiz is loaded, Enlightenment completely stops rendering clients. The module creates and manages a GL FBO for each window and draws to it during the compositor canvas pre-render. Then, this FBO is bound to an image object which gets added into the window’s compositor object to handle clipping and stacking correctly. This image object covers the window, and frame area, and the stacks above it; since the Compiz window manager has received the synthesized events, it draws into the correct area, resulting in everything looking mostly normal. A side effect of this is that the window’s frame will never animate; this is drawn directly onto the compositor canvas, and so it is not part of the window’s texture.
I haven’t created a mechanism for handling configuration of Compiz plugins; instead, the module directly loads the “ccp” plugin from libcompizconfig. The user edits settings as normal using the Compiz Settings Manager UI, and then the corresponding plugin will automatically load all the effect plugins that have been enabled. On final thing that is missing is support for any kind of screen effect plugin. For example, the desktop cube plugin is not able to be used at this time. I think this is definitely doable, and potentially could be fairly easy, but I haven’t found time to work on this.
All in all, this was an interesting proof of concept project. It took about two weeks of relatively casual hacking, though I successfully rendered my first wobbly window after the first week. I don’t plan to actively develop this module. While it was fun to work on something new and extend the Enlightenment compositor API in order to support external compositing, it’s hard to continue working on a project that I personally have no use for–especially after I succeeded in my initial goal. My hope is that this will inspire other developers to try writing cool effects for Enlightenment, whether by continuing work on this Compiz module or by creating something entirely new.