Common EFL Focus Pitfalls

I started patching the focus subsystem of the EFL widget toolkit quite some time ago. During this time, people have started to assign me everything that somehow looks like an issue with focus, it sometimes only takes the existence of the word “focus” somewhere in a backtrace for this to happen. I’ve discovered that most people mix up these different types, so in this blog post I hope to provide some clarity about them.

How EFL Gets Focused

There are 3 different places focus happens in EFL:

  • ecore-evas – the window manager abstraction
  • evas – the canvas library
  • elementary – the widget toolkit.

First of all, I should point out what focus itself is, I think a good example is to consider your typical smartphone interaction. While interacting with your smartphone, your complete attention is given to its screen and all interactions are with the interface of the device; you will also likely cease interactions with the outside environment entirely. In the same way, each abstraction in EFL has its own interaction partners:

  • The focused canvas object gets the attention from the keyboard
  • The focused widget is highlighted visually, so the user can see where his attention should go
  • And the window manager in the end focuses an application, which is probably an EFL Application.

These differences are often the source of people’s confusion when it comes to focus in EFL. For example, loosing the toolkit focus on a window object does not mean that the window lost the input from the user; instead, it means that another widget got the toolkit focus and the window manager still has focus on this window.

For another example, consider a toolkit widget that’s built out of two objects: an image and a text field below the image. In this example, the widget receives the toolkit’s focus, and the focus of the canvas moves to the image. Then, the user presses some key bindings to change the name of the image and the canvas focus moves to the text field. In this case, the canvas focus moves, creating update events on the canvas focus. However, the widget’s focus stayed the same, and the user is meant to have their attention on that widget, meaning there was no change to it.

Some Tips for Understanding EFL Focus

The focus property can only be true on one single object in an entity, in practice this means:

  • One window focused per user window manager session
  • One object focused per user canvas
  • One widget focused per widget tree in the toolkit

Additionally:

  • Canvas focus is only used for injecting keyboard inputs from the input system of the display technology.
  • Widget focus is used for navigation, in the case of focus movement initiated by key bindings, this is the position from where the next upper/down/right/left element is calculated.

If you ever use change events for some kind of a focus property, here’s what you need to know about focus in EFL:

  • It’s window manager focus if the user has their attention on your application.
  • It’s canvas focus if you need to know where the keyboard inputs are coming from.
  • It’s a widget focus if the user has moved their attention to a subset of canvas objects that are bound together as a widget implementation.

If you have any questions about any of this content, head to the comments section!

EFL Multiple Output Support with Wayland

Supporting multiple outputs is something most of us take for granted in the world of X11 because the Xorg team has had years to implement and perfect support for multiple outputs. While we have all enjoyed the fruits of their labor, this has not been the case for Wayland. There are two primary types of multiple output configurations that are relevant: cloned and extended outputs.

Cloned Outputs

Cloned output mode is the one that most people are familiar with. This means that the contents of the primary output will be duplicated on any additional outputs that are enabled.  If you have ever hot plugged an external monitor into your Windows laptop, then you have seen this mode in action.

EFL Multiple Output Support with Wayland - mirrored-display
An example of cloned output

Extended Outputs

Extended output mode is somewhat less common, yet still very important. In this mode, the desktop is extended to span across multiple outputs, while the primary output retains sole ownership of any panels, shelves, or gadgets that reside there. This enables the secondary monitor to act as an extended desktop space where other applications can be run.

EFL Multiple Output Support with Wayland - extended-display
An example of extended outputs

Multiple Output Support in Weston and EFL

Weston, the reference Wayland compositor, has had the ability to support multiple outputs in an extended configuration for quite some time now. This extended mode configuration allows each output to run with completely separate framebuffer and repaint loops. There are patches currently being worked on to enable cloned output support, but these have not been pushed upstream yet.

While EFL has had support for multiple outputs when running under X11 for quite some years now, the support for multiple outputs under Wayland has been missing. A major hurdle to implementing this support has been that Evas, the EFL rendering library, was not accounting for multiple outputs in it’s rendering update loop. With this commit, our very own Cedric Bail has removed that hurdle and enabled work to progress further on implementing this feature.

Complete Multiple Output Support is on the Way

While the current implementation of multiple output support in EFL Wayland only supports outputs in a cloned configuration, support for extended mode is in development and is expected to be available in upstream EFL soon. When these patches get published upstream, this will bring our EFL Wayland implementation much closer in parity to our X11 support. Thankfully there are little to no changes needed to add this support to our Enlightenment Wayland compositor.

EFL: Enabling Wayland Output Rotation

The Enlightenment Foundation Libraries have long supported the ability to do output rotation when running under X11 utilizing the XRandR (resize and rotate) extension. This functionality is exposed to the user by way of the Ecore_X library which provides API function calls that can be used to rotate a given output.

Wayland

While this functionality has been inside EFL for a long time, the ability to rotate an output while running under the Enlightenment Wayland Compositor has not been possible. This is due, in part, to the fact that the Wayland protocol does not provide any form of RandR extension. Normally this would have proved a challenge when implementing output rotation inside the Enlightnement Wayland compositor, however EFL already has the ability to do this.

Software-Based Rotation

EFL’s Ecore_Evas library, which is used as the base of the Enlightenment Compositor canvas, has the ability to perform software-based rotation. This means that when a user asks for the screen to be rotated via Enlightenment’s Screen Setup dialog, the Enlightenment Compositor will draw it’s canvas rotated to the desired degree using the internal Evas engine code. While this works for any given degree of rotation, it is not incredibly efficient considering that modern video cards can do rotation themselves.

Hardware-Based Rotation

Many modern video cards support some form of hardware-based rotation. This allows the hardware to rotate the final scanout buffer before displaying to the screen and is much more efficient than software-based rotation. The main drawback is that many will only support rotating to either 0 degrees or 180 degrees. While this is a much more practical and desired approach to output rotation, the lack of other available degrees of rotation make it a less than ideal method.

Hybrid Rotation

In order to facilitate a more ideal user experience, we have decided to implement something I call hybrid rotation. Hybrid rotation simply means that when asked to rotate the compositor canvas, we will first check if the hardware is able to handle the desired rotation itself. In the event that the hardware is unable to accommodate the necessary degree of rotation, it will then fall back to the software implementation to handle it. With this patch we can now handle any degree of output rotation in the most efficient way available.

The major benefit for application developers is that they do not need to know anything about the capabilities of the underlying hardware. They can continue to use their existing application code to rotate the canvas:

/**
 * Ecore example illustrating the basics of ecore evas rotation usage.
 *
 * You'll need at least one Evas engine built for it (excluding the
 * buffer one). See stdout/stderr for output.
 *
 * @verbatim
 * gcc -o ecore_evas_basics_example ecore_evas_basics_example.c `pkg-config --libs --cflags ecore evas ecore-evas`
 * @endverbatim
 */

#include <Ecore.h>
#include <Ecore_Evas.h>
#include <unistd.h>

static Eina_Bool
_stdin_cb(void *data EINA_UNUSED, Ecore_Fd_Handler *handler EINA_UNUSED)
{
   Eina_List *l;
   Ecore_Evas *ee;
   char c;

   int ret = scanf("%c", &c);
   if (ret < 1) return ECORE_CALLBACK_RENEW;

   if (c == 'h')
     EINA_LIST_FOREACH(ecore_evas_ecore_evas_list_get(), l, ee)
       ecore_evas_hide(ee);
   else if (c == 's')
     EINA_LIST_FOREACH(ecore_evas_ecore_evas_list_get(), l, ee)
       ecore_evas_show(ee);
   else if (c == 'r')
     EINA_LIST_FOREACH(ecore_evas_ecore_evas_list_get(), l, ee)
       ecore_evas_rotation_set(ee, 90);

   return ECORE_CALLBACK_RENEW;
}

static void
_on_delete(Ecore_Evas *ee)
{
   free(ecore_evas_data_get(ee, "key"));
   ecore_main_loop_quit();
}

int
main(void)
{
   Ecore_Evas *ee;
   Evas *canvas;
   Evas_Object *bg;
   Eina_List *engines, *l;
   char *data;

   if (ecore_evas_init() <= 0)
     return 1;

   engines = ecore_evas_engines_get();
   printf("Available engines:\n");
   EINA_LIST_FOREACH(engines, l, data)
     printf("%s\n", data);
   ecore_evas_engines_free(engines);

   ee = ecore_evas_new(NULL, 0, 0, 200, 200, NULL);
   ecore_evas_title_set(ee, "Ecore Evas basics Example");
   ecore_evas_show(ee);

   data = malloc(sizeof(char) * 6);
   sprintf(data, "%s", "hello");
   ecore_evas_data_set(ee, "key", data);
   ecore_evas_callback_delete_request_set(ee, _on_delete);

   printf("Using %s engine!\n", ecore_evas_engine_name_get(ee));

   canvas = ecore_evas_get(ee);
   if (ecore_evas_ecore_evas_get(canvas) == ee)
     printf("Everything is sane!\n");

   bg = evas_object_rectangle_add(canvas);
   evas_object_color_set(bg, 0, 0, 255, 255);
   evas_object_resize(bg, 200, 200);
   evas_object_show(bg);
   ecore_evas_object_associate(ee, bg, ECORE_EVAS_OBJECT_ASSOCIATE_BASE);

   ecore_main_fd_handler_add(STDIN_FILENO, ECORE_FD_READ, _stdin_cb, NULL, NULL, NULL);

   ecore_main_loop_begin();

   ecore_evas_free(ee);
   ecore_evas_shutdown();

   return 0;
}

 

Summary

While not all hardware can support the various degrees of output rotation, EFL provides the ability to rotate an output via software. This gives the Enlightenment Wayland Compositor the ability to handle output rotation in the most efficient method available while being transparent to the user.

Improving EFL Graphics With Wayland Application Redraws

One of the most heinous visual artifacts modern displays are capable of is tearing, on the desktop, there are two major potential sources of this:

  1. A compositor redraw during screen refresh – this leads to global artifacts such as application windows breaking up while being repositioned.
  2. Applications redrawing while the compositor is reading their content – the contents of a single window will be displayed in an indeterminate state.

Under X, application redraws are tricky to do without tearing because content can be updated at any chosen time with no clear feedback as to when the compositor will read it. EFL uses some clever tricks to this end (check out the state of the art X redraw timing for yourself), but it’s difficult to get right in all cases. For a lot of people this just works, or they’re not sensitive to the issue when it doesn’t.

Wayland Does Application Redraws Better

For the rest of us, Wayland has a different approach; prepare the buffer, attach it to a surface, and commit it for display. If you want the compositor to help with timing, you can request a frame callback before committing the surface; the compositor will display the frame on the next vertical retrace and will send a callback event at some point in the future. This callback informs the program that if it submits a buffer now, it has a reasonable chance to hit the upcoming vertical retrace.

Pro-tip: You can request a frame callback without submitting a buffer as well, and the compositor will still send a callback (which may be immediate) when it thinks you should commit a buffer for display.

The client shouldn’t expect the frame callback to be generated when the client is obscured or otherwise invisible. That is, the frame callback only tells you to draw when it’s useful for you to draw. This means the frame callback is not useful as a general purpose timing mechanism, it’s only for display.

Once the buffer is committed to the compositor, the compositor owns it. The Wayland compositor will send the buffer back when it’s done with it via a release event; until this point you’re simply not allowed to draw into it again. The result is undefined, and can cause your program to terminate as punishment for being poorly written.

Because of this commit/release paradigm, it’s very difficult to cause tearing artifacts under Wayland (it requires out of spec behavior that can lead to application termination).

Some people (notably gamers) will complain that synchronizing display with monitor refresh like this introduces input lag, and while I’m not going to wander into that firefight, I’d be remiss if I didn’t mention there’s no obligation to use frame callbacks for timing. If your goal is to render your frames as temporally proximal as possible to the screen retrace, then standard triple buffering is easily accomplished without using frame callbacks.

The client can render and commit frames as quickly as the CPU/GPU allows, and the compositor will release many of them undisplayed as soon as they’re replaced with a newer frame. Only the most recently-submitted frame at the time the compositor redraws the screen will be used. When an application doesn’t need twitchy response times though, using frame callbacks results in smooth animation with no wasted rendering.

Bringing This Improvement to EFL

New for the upcoming EFL 1.21 release (TBA soon, we promise) EFL applications now (finally!) drive animations directly from these frame callbacks. This has been a long time coming as it required major changes to our core timing code. Previously, we used a separate timer source that ran continuously in a thread, sending frame times through a pipe to the main thread. These “ticks” were then gated by a clever hack that prevented a redraw from occurring between buffer submission time and frame callback time. Prior to this, we simply triple buffered and threw away a few frames.

The most obvious immediate benefit of doing this properly has been that when the compositor blanks the screen, many clients stop consuming CPU cycles entirely. This is a departure from our previous timing implementation which burned many CPU cycles sending timestamps nothing cared about. It’s also a radical change from the behavior under X where animations continue despite the black screen, while X, enlightenment, and the client all continue to work needlessly.

So, here we have yet another case where Wayland does less than X.

New Features in Enlightenment 22

The E22 development cycle has been underway for over a year, and it has included over 1,500 patches to address nearly 200 tickets on our issue tracker. With this has come a number of new features and improvements.

Greatly Improved Wayland Support

The majority of development for this cycle has gone towards improving Wayland support. This covers, but is not limited to: adding support for xdg-shell v6, pointer constraints, and relative pointer motion protocols. These additions improve XWayland support and increase stability across all components running under Wayland.

Continued Improvements to New Gadget Infrastructure

As previous posts have indicated, a lot of work is being done in this area. The goal is to create a more robust infrastructure with a simpler and more intuitive EFL-based API, moving away from the legacy “gadcon” interface, which has its own API and currently only functions due to mountains of gadget-specific workarounds that make safely adding any new functionality nearly impossible.
Presently, almost all gadgets and functionality that existed for the legacy gadget interface have been ported or rewritten for the new system, with future improvements already underway.

sudo/ssh Password GUI

A decision was made to implement a GUI for the ASKPASS feature of sudo/ssh which will pop up a dialog any time a password needs to be entered for these services.
More information about this feature can be found by referencing the SUDO_ASKPASS or SSH_ASKPASS environment variables.

Meson Build System

Meson is a relatively new build system that is significantly faster than autotools while providing a similar set of features. All modern distributions should be compatible with this build system, and autotools is planned to be removed after the E22 release.

Tiling Window Policy Improvements

Many issues related to this have been resolved, and some new features – such as a nicer window dragging UI – have been added.

Per-Window Pulseaudio Volume Controls

Pulseaudio exports volume controls for each audio-producing application, and work has been done to bind this into a GUI control.

New Features in Enlightenment 22 - e22-mixer

Stay Tuned for More!

More work is planned to improve the desktop experience in future versions of Enlightenment. Check out our contact page to find out more or get involved.

How to Create an EFL Gadget Sandbox

The new gadget API and infrastructure for Enlightenment continue to undergo heavy development. In addition to improving and extending the base gadget UI, work has recently begun on creating a gadget provider with the new API to provide sandboxing and allow gadgets to be written as regular applications that don’t have or require access to compositor internals.

The primary enabler of the new sandboxing system is the efl-wl compositor widget. This allows the compositor to launch applications in isolation, and also provides the ability to add protocol extensions for only that specific instance of the compositor widget. Using these features, it becomes possible to add gadget-specific protocols and utilities on the compositor side that are passed through transparently to the client gadget application.

Currently, there is one base protocol in use: the e-gadget protocol, which looks like this:



  
    
      
      
      
    
    
      
      
      
      
      
      
    
    
      
      
      
      
      
    
    
      
    
    
      
    
    
      
    
  


The purpose of this is to mimic the gadget API. Applications can choose to listen for events on the window object with the listed event name and receive the corresponding property value. This is made possible because of a shared library that is preloaded into applications by the gadget provider using the LD_PRELOAD environment variable. This preloaded library intercepts calls to create EFL window objects and converts Wayland protocol events into callbacks on the window object. Gadget application developers can then write applications without the need to concern themselves with anything related to Wayland.

Example EFL Gadget Sandbox

An example sandbox gadget written in EFL reads something like this:

#include < Elementary.h >
static Evas_Object * popup;
static Evas_Object * child;

static void
popup_del(void * data EINA_UNUSED, Evas * e EINA_UNUSED, Evas_Object * obj EINA_UNUSED, void * event_info EINA_UNUSED)
{
  popup = NULL;
}

static void
child_del(void * data EINA_UNUSED, Evas * e EINA_UNUSED, Evas_Object * obj EINA_UNUSED, void * event_info EINA_UNUSED)
{
  child = NULL;
}

static void
popup_unfocus(void * data EINA_UNUSED, Evas * e EINA_UNUSED, Evas_Object * obj, void * event_info EINA_UNUSED)
{
  evas_object_del(obj);
}

static void
mouse_button(void * data EINA_UNUSED, Evas * e EINA_UNUSED, Evas_Object * obj, void * event_info)
{
  Evas_Object * ic;
  Evas_Event_Mouse_Down * ev = event_info;
  char buf[PATH_MAX];
  int w, h;
  Evas_Object * win;
  Elm_Win_Type type = ELM_WIN_POPUP_MENU;
  if ((ev - > button != 1) && (ev - > button != 3)) return;
  if (ev - > button == 3)
  {
    type = ELM_WIN_BASIC;
    if (child)
    {
      evas_object_del(child);
      return;
    }
  } else
  {
    if (popup)
    {
      evas_object_del(popup);
      return;
    }
  }
  win = elm_win_add(elm_win_get(obj), "win", type);
  elm_win_alpha_set(win, 1);
  if (ev - > button == 3)
  {
    child = win;
    evas_object_event_callback_add(win, EVAS_CALLBACK_DEL, child_del, NULL);
  } else
  {
    popup = win;
    evas_object_event_callback_add(win, EVAS_CALLBACK_DEL, popup_del, NULL);
  }
  if (ev - > button == 3)
    evas_object_event_callback_add(win, EVAS_CALLBACK_FOCUS_OUT, popup_unfocus, NULL);
  ic = elm_icon_add(win);
  snprintf(buf, sizeof(buf), "%s/images/bubble.png", elm_app_data_dir_get());
  elm_image_file_set(ic, buf, NULL);
  elm_image_object_size_get(ic, & w, & h);
  evas_object_size_hint_aspect_set(win, EVAS_ASPECT_CONTROL_BOTH, w, h);
  if (ev - > button == 1)
  {
    elm_image_resizable_set(ic, EINA_FALSE, EINA_FALSE);
    elm_image_no_scale_set(ic, EINA_TRUE);
  }
  evas_object_size_hint_weight_set(ic, EVAS_HINT_EXPAND, EVAS_HINT_EXPAND);
  evas_object_size_hint_fill_set(ic, 0.5, 0.5);
  evas_object_size_hint_min_set(ic, 100, 100);
  elm_win_resize_object_add(win, ic);
  evas_object_show(ic);
  evas_object_show(win);
}

int
main(int argc, char * argv[])
{
  Evas_Object * win, * ic;
  char buf[PATH_MAX];
  int w, h;
  elm_init(argc, (char * * ) argv);
  elm_policy_set(ELM_POLICY_QUIT, ELM_POLICY_QUIT_LAST_WINDOW_CLOSED);
  elm_app_info_set(main, "elementary", "images/logo.png");
  win = elm_win_add(NULL, "icon-transparent", ELM_WIN_BASIC);
  elm_win_title_set(win, "Icon Transparent");
  elm_win_autodel_set(win, EINA_TRUE);
  elm_win_alpha_set(win, EINA_TRUE);
  ic = elm_icon_add(win);
  snprintf(buf, sizeof(buf), "%s/images/logo.png", elm_app_data_dir_get());
  elm_image_file_set(ic, buf, NULL);
  elm_image_object_size_get(ic, & w, & h);
  evas_object_size_hint_aspect_set(win, EVAS_ASPECT_CONTROL_BOTH, w, h);
  if (argc > 1)
  {
    elm_image_resizable_set(ic, EINA_FALSE, EINA_FALSE);
    elm_image_no_scale_set(ic, EINA_TRUE);
  }
  evas_object_size_hint_weight_set(ic, EVAS_HINT_EXPAND, EVAS_HINT_EXPAND);
  evas_object_size_hint_fill_set(ic, 0.5, 0.5);
  evas_object_size_hint_min_set(ic, 100, 100);
  elm_win_resize_object_add(win, ic);
  evas_object_show(ic);
  evas_object_event_callback_add(ic, EVAS_CALLBACK_MOUSE_DOWN, mouse_button, NULL);
  evas_object_show(win);
  ecore_main_loop_begin();
  return 0;
}

This code results in an E logo gadget like this:

How to Create an EFL Gadget Sandbox - gadget_sandbox1.png

Left clicking creates a popup window (xdg_popup) which the gadget provider extracts from the compositor widget to display like a normal gadget contextual popup.

How to Create an EFL Gadget Sandbox - gadget_sandbox2

Right clicking creates a child window (xdg_toplevel with parent) which the gadget provider extracts from the compositor widget to display like a standard gadget popup.

How to Create an EFL Gadget Sandbox - gadget_sandbox3

There are no toolkit or language restrictions for gadget applications using this functionality. The only requirement is that interacting with the compositor in any way, such as receiving the gadget properties, requires use of the Wayland protocol. Even without that, any framework or toolkit that has Wayland support can now have fully functional gadgets running inside Enlightenment.

The sandbox infrastructure can currently be found in the desksanity module, and is scheduled to be merged into the main tree after the E22 release.

Implementing a Multiseat Wayland Compositor into a Toolkit Widget

Wayland: there have been many blog posts, articles, and news items about the new Linux display protocol. There have been developers working to extend the protocol for new and extremely helpful use cases, but new frontiers continue to be explored: putting a multiseat Wayland compositor into a toolkit widget.

Due to Wayland’s philosophy of “build your own compositor,” there is no de-facto implementation of a Wayland display server that is analogous to Xorg for X11. Wayland implementations are written using the libwayland server API, and this is flexible enough to be used even in the case of a widget. Overall, the only noteworthy difference between putting a compositor in a widget and a normal nested compositor is that the output size is set to the size of the widget instead of the size of the window. With this in mind, a compositor widget can be manipulated just like any other widget, as it manages all protocol-related matters internally.

[iframe width=”560″ height=”315″ src=”https://www.youtube.com/embed/xZGYdg21IS0″ frameborder=”0″ allowfullscreen]

weston-terminal running in a widget in an X11 window.

Implementing Clipboard Sharing and Drag-N-Drop

This project had a number of difficulties along the way, most of them being related to clipboard and drag-n-drop support. A nested compositor of this type is significantly less useful if it lacks support for these things, and so it was necessary to add bridging for Wayland<->Wayland selection transfers as well as X11<->Wayland transfers. Clipboard selections were the easier part to handle in this case, as they do not require mouse/touch interactions and have no positioning data. For Wayland<->Wayland requests, all that’s needed is to transparently proxy all the data offer protocol exchanges to and from the parent compositor, and then pass any resulting file descriptors to the clients. Under X11, however, data must be manually read from or written to X11 window atoms and the selection requests need to be translated into Wayland equivalents.

[iframe width=”560″ height=”315″ src=”https://www.youtube.com/embed/o0a0I4Ao9Ng” frameborder=”0″ allowfullscreen]

Clipboard bridging under X11

Drag-N-Drop (DND) support was considerably more difficult. Not only does positioning affect the operation, but when the drag leaves the widget it becomes necessary to proxy the drag surface into a new window and initiate a global DND operation instead of working only within the widget. This leads to a new set of issues where buffer management for the drag surface must be adjusted to account for different render timings of the global drag window, which reuses the same buffer. The behavior is different between X11 and Wayland as well. Under X11, when dragging back to the originating widget, the global drag can be canceled. However, under Wayland the operation remains global and the result will be that the widget proxies a selection send through the outer compositor back to itself.

[iframe width=”560″ height=”315″ src=”https://www.youtube.com/embed/muTkLt1VNlw” frameborder=”0″ allowfullscreen]

DND bridging under X11

Adding full support for bridged DND also means that dragging between separate compositor instances also becomes possible.

[iframe width=”560″ height=”315″ src=”https://www.youtube.com/embed/lonwtDsz9VE” frameborder=”0″ allowfullscreen]

DND bridging between compositors under X11.

Another challenge with the process of putting Wayland into a canvas widget was supporting subsurfaces and other types of child surfaces. In Enlightenment, these are handled as regular objects which are stacked above the parent, but this makes the object tree more difficult to debug. The compositor widget does things a bit differently by making child surfaces into child objects of the parent surface, creating an object hierarchy which is much easier to read and debug.

[iframe width=”560″ height=”315″ src=”https://www.youtube.com/embed/u8bbTAVJirQ” frameborder=”0″ allowfullscreen]

Subsurfaces under X11.

Using the Compositor Widget to Sandbox Applications

The compositor widget has a number of useful features, but perhaps the most overlooked feature would be the ability to run applications in a sandbox. Any application launched inside has no ability to directly affect anything outside of the widget, meaning that questionable applications can be run in a safe environment from the display server’s perspective. While it has obvious uses in security, the more mundane usage could simply be embedding a web browser into an application without the need to connect to any browser engine’s embedding API.

Adding a compositor widget to an application can allow content to be embedded in isolation. It provides seamless integration with existing desktop interactions and requires no special handling to be placed into a UI layout. It also allows Wayland applications to run under X11, although this is perhaps not as useful since there are very few Wayland-only applications at present.

An Introduction to Relative and Constrained Pointers in Enlightenment

Recent development in Enlightenment’s Wayland compositor has focused on implementing cross-desktop protocols and improving stability. One of the recently handled protocol series has been relative and constrained pointers.

Relative pointer motion is a method for providing pointer movement deltas directly to applications. This is useful for a number of cases, though the easiest to imagine might be first person shooter games. In this case, the application receives movement deltas, allowing the player to endlessly scroll the screen in one direction without hitting the boundaries of the screen.

Under Enlightenment, this is handled in different ways depending on the output backend being used. For backends using libinput, e.g., DRM, it’s possible to get relative motion deltas directly from the events and then emit them for compositor use. Other backends, however, such as nested Wayland compositors, have no access to libinput’s hardware events. In this case, Enlightenment must manually calculate the deltas between motion events to pass to applications. This case is less precise, but for the case where there is no device directly attached to the compositor then there is no other method of getting relative motion data. In this way, all backends can now supply relative motion events when requested.

Pointer Confinement and Locking in Wayland

The pointer constraints protocol under Wayland consists of two separate concepts: pointer confinement and pointer locking. Pointer confinement, in this case, is where the pointer’s position is confined to a number of specified regions. Once confined, the pointer is unable to leave the specified region until the application releases the confinement or the compositor policy results in the confinement being released. Pointer locking, on the other hand, is when the pointer’s position is locked to a specific location, preventing motion events from being propagated to the client. A locked pointer similarly will not be unlocked either until the client releases the lock or the compositor policy requires the lock to end.

Confined pointers are implemented in Enlightenment primarily using calculation of motion deltas in relation to the confinement regions. By converting the deltas into simple linear equations, it becomes easy to determine whether the delta has occurred within the confined region as well as whether the pointer motion terminated within the confined region. If the motion would result in the pointer leaving the confined region, substitution into the equation allows for the calculation of the closest point to the end of the allowed confinement region, letting the pointer be confined as close to the hardware pointer position as possible.

An Introduction to Relative and Constrained Pointers in Enlightenment - confine

Locked pointers are perhaps the simplest to implement in any compositor: simply stop providing motion events to the pointer-focused surface once the lock begins.

All of these protocols are now implemented in Enlightenment as well as a number of other Wayland compositors, allowing for desktop applications to begin using the functionalities within the protocol.

Libdrm Event Handling: You’re Probably Doing it Wrong

I want to use this short article to point out a big problem in code that uses libdrm. This, or a variant of this, is lurking in a shocking amount of software:

drmEventContext evctx;

evctx.version = DRM_EVENT_CONTEXT_VERSION;
 evctx.page_flip_handler = pfhdlr;
 evctx.vblank_handler = vbnkhdr;

Stop doing this, it has always been a crash waiting to happen.

drmEventContext is part of libdrm’s ABI and there are strict promises that the structure won’t be randomly re-ordered. However, it may grow as the need arises when new versions of libdrm offer functionality which didn’t exist before.

libdrm 2.4.78 introduces another page flip handler, adding a new function pointer harmlessly at the end of the structure page_flip_handler2. It also bumps DRM_EVENT_CONTEXT_VERSION from 2 to 3, as that’s the current version of the structure in the header file.

As a result, the code above crashes… sometimes. What went wrong?

The Proper Way to Use drmEventContext

DrmEventContext.version is intended to tell the library what version of the event context functionality an application understands. The DRM_EVENT_CONTEXT_VERSION macro indicates the highest version the installed libdrm you compiled against understands.

The new page_flip_handler2 function will only be used if the calling program sets the version to 3, declaring it knows about page_flip_handler2 and has set it correctly. The snippet declared its version to always be the most recent version of the structure, yet left the new page_flip_handler2 pointer uninitialized, meaning it might be non-NULL and might be called as a function: potentially causing the program to crash.

EFL had a variant of this:

drmEventContext ctx;
 memset(&ctx, 0, sizeof(ctx));
 ...

This, at a glance, looks safe since the new pointer is always initialized to NULL. But, this is potentially broken too as the context version could, in theory, change something behavioral. So we’ve updated our code to be future proof, and any code you have that uses libdrm you should too! :)

The fix is trivially different:


 drmEventContext evctx;

evctx.version = 2;
 evctx.page_flip_handler = pfhdlr;
 evctx.vblank_handler = vbnkhdr;

Unless, of course, your code knows about page_flip_handler2 and sets it correctly, then go ahead and set version to 3.

(And we’re all left wondering why that macro exists at all…)

A Curious Wayland Bug

Enlightenment has a slightly unconventional architecture. For many of its internal dialogues (settings, the file manager, etc.) it uses what we call “internal windows.” These are simply regular toolkit windows that use regular rendering back-ends; in the good ol’ days, this meant a connection to the X server which resulted in a render to a standard X window. In this scenario, the compositor gets the result back the same way as all X client windows, and it composites all internal and external windows together.

Under Wayland, things get a bit scarier because now the compositor is the display server. Enlightenment makes a Wayland connection to itself; this gives us all manner of game winning capabilities we never had before. For example, let’s say the internal window wants to sleep and wait for a reply from the compositor (such as a released buffer to render the next frame into). We deadlock and fall down a hole. But I digress.

Problems with internal windows also led us to this problem recently where the compositor exited, logged nothing, and gave only a cryptic message:

invalid object (3), type (zwp_linux_dmabuf_v1), message enter(uoa)

This prompts two questions

  1. What does that even mean?
  2. How do I debug something like this?

Since both client and compositor errors are logged in the same file, the first surprise was learning that this is a client error message. An internal window connection was closing itself on the client end because it received something from the compositor that didn’t make any sense.

This error message means the client received a message of the form: foo.enter(unsigned integer, object, array) where the object’s id was 3: a zwp_linux_dmabuf_v1 object. The client knows what type “object” should have been because it knows what type “foo” is (even though it doesn’t bother to include this bit of information in the log), and that type isn’t zwp_linux_dmabuf_v1. So, the client doesn’t have any idea about what to do with this event. It then has an existential crisis to ponder: what if that event is important in discerning the meaning of all future events? The only thing left for it to do is quietly disconnect and exit(); since this particular client is the same process as the compositor, it takes the rest of the desktop with it.

How do we Debug this Mess?

Well, a reasonable first step might be to add more instrumentation to libwayland-client to actually tell us the type of the object that received the bad message, but let’s be lazy.

Between the base Wayland protocol and extensions from the wayland-protocols repository, there are currently 6 possible “enter” events in Wayland. It doesn’t take too terribly long to match the signature from the error message – the (uoa) part – in the log for the keyboard enter event in the protocol XML file.

We know the client is exiting, but the client is likely not the location of the bug; rather, it seems to be exiting in response to garbage from the compositor. One of the several wl_keyboard_send_enter() call sites in enlightenment is probably the root cause.

In fact, what was happening is that under certain circumstances a change of keyboard focus wasn’t cleaning up all internal states, and a keyboard enter event was being sent to the wrong client. In some other client, object id 3 corresponded to a wl_surface object, which is the second argument that’s expected in a keyboard enter. It was actually fairly easy to sort out, but made much more difficult by the limited information in the log and the confusing way the client took the hit for a compositor bug.

Now that I’ve explained how to debug this error pattern, I’ll mention in passing that nobody else will have to quite these lengths again. We’ve recently landed changes to the wayland-server library which validate events before sending them to ensure it’s not mixing up objects from different clients. Now, the compositor will log an error with a much more coherent message instead of making it appear that the client is responsible for the wrong doings. This makes it easier to start the debugging process in the right place, and should help make future debug marathons shorter. The client still gets disconnected which is a little disruptive, but it’s a clean disconnect rather than an abort(), so the client is free to attempt a reconnect. Then, the server side prints an error message with the actual object interface name, making it so this kind of bug should be a little easier to sort out in the future.