EFL Multiple Output Support with Wayland

Supporting multiple outputs is something most of us take for granted in the world of X11 because the Xorg team has had years to implement and perfect support for multiple outputs. While we have all enjoyed the fruits of their labor, this has not been the case for Wayland. There are two primary types of multiple output configurations that are relevant: cloned and extended outputs.

Cloned Outputs

Cloned output mode is the one that most people are familiar with. This means that the contents of the primary output will be duplicated on any additional outputs that are enabled.  If you have ever hot plugged an external monitor into your Windows laptop, then you have seen this mode in action.

EFL Multiple Output Support with Wayland - mirrored-display
An example of cloned output

Extended Outputs

Extended output mode is somewhat less common, yet still very important. In this mode, the desktop is extended to span across multiple outputs, while the primary output retains sole ownership of any panels, shelves, or gadgets that reside there. This enables the secondary monitor to act as an extended desktop space where other applications can be run.

EFL Multiple Output Support with Wayland - extended-display
An example of extended outputs

Multiple Output Support in Weston and EFL

Weston, the reference Wayland compositor, has had the ability to support multiple outputs in an extended configuration for quite some time now. This extended mode configuration allows each output to run with completely separate framebuffer and repaint loops. There are patches currently being worked on to enable cloned output support, but these have not been pushed upstream yet.

While EFL has had support for multiple outputs when running under X11 for quite some years now, the support for multiple outputs under Wayland has been missing. A major hurdle to implementing this support has been that Evas, the EFL rendering library, was not accounting for multiple outputs in it’s rendering update loop. With this commit, our very own Cedric Bail has removed that hurdle and enabled work to progress further on implementing this feature.

Complete Multiple Output Support is on the Way

While the current implementation of multiple output support in EFL Wayland only supports outputs in a cloned configuration, support for extended mode is in development and is expected to be available in upstream EFL soon. When these patches get published upstream, this will bring our EFL Wayland implementation much closer in parity to our X11 support. Thankfully there are little to no changes needed to add this support to our Enlightenment Wayland compositor.

EFL: Enabling Wayland Output Rotation

The Enlightenment Foundation Libraries have long supported the ability to do output rotation when running under X11 utilizing the XRandR (resize and rotate) extension. This functionality is exposed to the user by way of the Ecore_X library which provides API function calls that can be used to rotate a given output.

Wayland

While this functionality has been inside EFL for a long time, the ability to rotate an output while running under the Enlightenment Wayland Compositor has not been possible. This is due, in part, to the fact that the Wayland protocol does not provide any form of RandR extension. Normally this would have proved a challenge when implementing output rotation inside the Enlightnement Wayland compositor, however EFL already has the ability to do this.

Software-Based Rotation

EFL’s Ecore_Evas library, which is used as the base of the Enlightenment Compositor canvas, has the ability to perform software-based rotation. This means that when a user asks for the screen to be rotated via Enlightenment’s Screen Setup dialog, the Enlightenment Compositor will draw it’s canvas rotated to the desired degree using the internal Evas engine code. While this works for any given degree of rotation, it is not incredibly efficient considering that modern video cards can do rotation themselves.

Hardware-Based Rotation

Many modern video cards support some form of hardware-based rotation. This allows the hardware to rotate the final scanout buffer before displaying to the screen and is much more efficient than software-based rotation. The main drawback is that many will only support rotating to either 0 degrees or 180 degrees. While this is a much more practical and desired approach to output rotation, the lack of other available degrees of rotation make it a less than ideal method.

Hybrid Rotation

In order to facilitate a more ideal user experience, we have decided to implement something I call hybrid rotation. Hybrid rotation simply means that when asked to rotate the compositor canvas, we will first check if the hardware is able to handle the desired rotation itself. In the event that the hardware is unable to accommodate the necessary degree of rotation, it will then fall back to the software implementation to handle it. With this patch we can now handle any degree of output rotation in the most efficient way available.

The major benefit for application developers is that they do not need to know anything about the capabilities of the underlying hardware. They can continue to use their existing application code to rotate the canvas:

/**
 * Ecore example illustrating the basics of ecore evas rotation usage.
 *
 * You'll need at least one Evas engine built for it (excluding the
 * buffer one). See stdout/stderr for output.
 *
 * @verbatim
 * gcc -o ecore_evas_basics_example ecore_evas_basics_example.c `pkg-config --libs --cflags ecore evas ecore-evas`
 * @endverbatim
 */

#include <Ecore.h>
#include <Ecore_Evas.h>
#include <unistd.h>

static Eina_Bool
_stdin_cb(void *data EINA_UNUSED, Ecore_Fd_Handler *handler EINA_UNUSED)
{
   Eina_List *l;
   Ecore_Evas *ee;
   char c;

   int ret = scanf("%c", &c);
   if (ret < 1) return ECORE_CALLBACK_RENEW;

   if (c == 'h')
     EINA_LIST_FOREACH(ecore_evas_ecore_evas_list_get(), l, ee)
       ecore_evas_hide(ee);
   else if (c == 's')
     EINA_LIST_FOREACH(ecore_evas_ecore_evas_list_get(), l, ee)
       ecore_evas_show(ee);
   else if (c == 'r')
     EINA_LIST_FOREACH(ecore_evas_ecore_evas_list_get(), l, ee)
       ecore_evas_rotation_set(ee, 90);

   return ECORE_CALLBACK_RENEW;
}

static void
_on_delete(Ecore_Evas *ee)
{
   free(ecore_evas_data_get(ee, "key"));
   ecore_main_loop_quit();
}

int
main(void)
{
   Ecore_Evas *ee;
   Evas *canvas;
   Evas_Object *bg;
   Eina_List *engines, *l;
   char *data;

   if (ecore_evas_init() <= 0)
     return 1;

   engines = ecore_evas_engines_get();
   printf("Available engines:\n");
   EINA_LIST_FOREACH(engines, l, data)
     printf("%s\n", data);
   ecore_evas_engines_free(engines);

   ee = ecore_evas_new(NULL, 0, 0, 200, 200, NULL);
   ecore_evas_title_set(ee, "Ecore Evas basics Example");
   ecore_evas_show(ee);

   data = malloc(sizeof(char) * 6);
   sprintf(data, "%s", "hello");
   ecore_evas_data_set(ee, "key", data);
   ecore_evas_callback_delete_request_set(ee, _on_delete);

   printf("Using %s engine!\n", ecore_evas_engine_name_get(ee));

   canvas = ecore_evas_get(ee);
   if (ecore_evas_ecore_evas_get(canvas) == ee)
     printf("Everything is sane!\n");

   bg = evas_object_rectangle_add(canvas);
   evas_object_color_set(bg, 0, 0, 255, 255);
   evas_object_resize(bg, 200, 200);
   evas_object_show(bg);
   ecore_evas_object_associate(ee, bg, ECORE_EVAS_OBJECT_ASSOCIATE_BASE);

   ecore_main_fd_handler_add(STDIN_FILENO, ECORE_FD_READ, _stdin_cb, NULL, NULL, NULL);

   ecore_main_loop_begin();

   ecore_evas_free(ee);
   ecore_evas_shutdown();

   return 0;
}

 

Summary

While not all hardware can support the various degrees of output rotation, EFL provides the ability to rotate an output via software. This gives the Enlightenment Wayland Compositor the ability to handle output rotation in the most efficient method available while being transparent to the user.

The Wayland Zombie Apocalypse is Near

Quite some time ago I received a report of a nasty Wayland bug: under certain circumstances a Wayland event was being delivered with an incorrect file descriptor. The reporter dug deeper and determined the root cause of this; it wasn’t good.

When a client deletes a Wayland object, there might still be protocol events coming from the compositor destined for it (as a contrived example, I delete my keyboard object because I’m done processing keys for the day, but the user is still typing…). Once the compositor receives the delete request it knows it can’t send any more events, but due to the asynchronous nature of Wayland, there could still be some unprocessed events in the buffer destined for the recently deleted object.

The Zombie Scourge

This is handled by making the deleted object into a zombie, rather, THE zombie, as there is a singleton zombie object in the client library that gets assigned to receive and eat (like a yummy bowl of brains) events destined for deleted objects. Once the compositor receives the delete request it will respond with a delete event, and at that time the client object ceases to be a zombie, and ceases to exist at all. Any number of objects may have been replaced by the zombie, a pointer in the object map just points to the zombie instead of a live object.

When an event is received, it undergoes a process called “demarshalling” where it’s decoded: a tiny message header in the buffer indicates its length, its destination object, and its “op code.” The type of the destination object and the op code are used to look up the signature for that event, which is a list of its parameters (integers, file descriptors, arrays, etc). Even though file descriptors are integer values, for the purposes of the Wayland protocol, integer is distinct from file descriptor. This is because when a Wayland event contains a file descriptor, that file descriptor is sent in a sort of out-of-band buffer along side the data stream (called an ancillary control message), instead of in the stream like an integer.

The demarshalling process consumes the main data stream and the ancillary buffer as it parses the message signature. Once a complete message is demarshalled, it is dispatched (client callbacks for that object plus op code are passed as parameters, and the client program gets to do its thing). When an event is destined for the zombie object, this demarshalling process is skipped. The length of data from the header is simply used to determine how much data to discard, and we proceed to the next event in the queue.

Here Lies the Problem

The file descriptors aren’t in the main data stream, so simply consuming that many bytes does not remove them from the ancillary buffer. The signature for the object is required to know how many file descriptors must be removed from the ancillary buffer, and the singleton zombie doesn’t (and can’t) have any specific event signatures at all.

So, if an event carrying a file descriptor is destined for a zombie object:

  • At best, the file descriptor is leaked in the client, is unclosable, and counts towards the client’s maximum number of open file descriptors forever.
  • At worst, there is a different problem; Since the file descriptors are pulled from the ancillary buffer in the order they’re inserted, if there is a following event that carries a file descriptor for a live object, it will get the file descriptor the zombie didn’t eat. The client will have no idea that this has occurred, and no idea what the file descriptor it received is actually going to provide for it. Bad things happen.

Not the Fix

We can’t change the wire protocol (to indicate the number of fds in the message header) because this would break existing software. We can’t simply keep the old object alive and mark it as dead, the object interface that contains the signatures is in client code, possibly in some sort of plug-in system in the client, and the client is allowed to dispose of all copies of it after deleting the object.

The Fix? More Zombies!

I recently sent a new edition of a patch series to fix this (and other file descriptor leaks) to the Wayland mailing list. The singleton zombie is permanently put to rest and is now replaced by an army of bespoke zombies, one for any object that can receive file descriptors in events, created at the time the object is instantiated (see, you can’t create at time of object destruction because it requires memory allocation, and that would allow deletion to fail…).

When the object is no longer viable, its zombie will live on, consuming its unhandled file descriptors until such time as the compositor informs the client it will no longer send any.

Improving EFL Graphics With Wayland Application Redraws

One of the most heinous visual artifacts modern displays are capable of is tearing, on the desktop, there are two major potential sources of this:

  1. A compositor redraw during screen refresh – this leads to global artifacts such as application windows breaking up while being repositioned.
  2. Applications redrawing while the compositor is reading their content – the contents of a single window will be displayed in an indeterminate state.

Under X, application redraws are tricky to do without tearing because content can be updated at any chosen time with no clear feedback as to when the compositor will read it. EFL uses some clever tricks to this end (check out the state of the art X redraw timing for yourself), but it’s difficult to get right in all cases. For a lot of people this just works, or they’re not sensitive to the issue when it doesn’t.

Wayland Does Application Redraws Better

For the rest of us, Wayland has a different approach; prepare the buffer, attach it to a surface, and commit it for display. If you want the compositor to help with timing, you can request a frame callback before committing the surface; the compositor will display the frame on the next vertical retrace and will send a callback event at some point in the future. This callback informs the program that if it submits a buffer now, it has a reasonable chance to hit the upcoming vertical retrace.

Pro-tip: You can request a frame callback without submitting a buffer as well, and the compositor will still send a callback (which may be immediate) when it thinks you should commit a buffer for display.

The client shouldn’t expect the frame callback to be generated when the client is obscured or otherwise invisible. That is, the frame callback only tells you to draw when it’s useful for you to draw. This means the frame callback is not useful as a general purpose timing mechanism, it’s only for display.

Once the buffer is committed to the compositor, the compositor owns it. The Wayland compositor will send the buffer back when it’s done with it via a release event; until this point you’re simply not allowed to draw into it again. The result is undefined, and can cause your program to terminate as punishment for being poorly written.

Because of this commit/release paradigm, it’s very difficult to cause tearing artifacts under Wayland (it requires out of spec behavior that can lead to application termination).

Some people (notably gamers) will complain that synchronizing display with monitor refresh like this introduces input lag, and while I’m not going to wander into that firefight, I’d be remiss if I didn’t mention there’s no obligation to use frame callbacks for timing. If your goal is to render your frames as temporally proximal as possible to the screen retrace, then standard triple buffering is easily accomplished without using frame callbacks.

The client can render and commit frames as quickly as the CPU/GPU allows, and the compositor will release many of them undisplayed as soon as they’re replaced with a newer frame. Only the most recently-submitted frame at the time the compositor redraws the screen will be used. When an application doesn’t need twitchy response times though, using frame callbacks results in smooth animation with no wasted rendering.

Bringing This Improvement to EFL

New for the upcoming EFL 1.21 release (TBA soon, we promise) EFL applications now (finally!) drive animations directly from these frame callbacks. This has been a long time coming as it required major changes to our core timing code. Previously, we used a separate timer source that ran continuously in a thread, sending frame times through a pipe to the main thread. These “ticks” were then gated by a clever hack that prevented a redraw from occurring between buffer submission time and frame callback time. Prior to this, we simply triple buffered and threw away a few frames.

The most obvious immediate benefit of doing this properly has been that when the compositor blanks the screen, many clients stop consuming CPU cycles entirely. This is a departure from our previous timing implementation which burned many CPU cycles sending timestamps nothing cared about. It’s also a radical change from the behavior under X where animations continue despite the black screen, while X, enlightenment, and the client all continue to work needlessly.

So, here we have yet another case where Wayland does less than X.

New Features in Enlightenment 22

The E22 development cycle has been underway for over a year, and it has included over 1,500 patches to address nearly 200 tickets on our issue tracker. With this has come a number of new features and improvements.

Greatly Improved Wayland Support

The majority of development for this cycle has gone towards improving Wayland support. This covers, but is not limited to: adding support for xdg-shell v6, pointer constraints, and relative pointer motion protocols. These additions improve XWayland support and increase stability across all components running under Wayland.

Continued Improvements to New Gadget Infrastructure

As previous posts have indicated, a lot of work is being done in this area. The goal is to create a more robust infrastructure with a simpler and more intuitive EFL-based API, moving away from the legacy “gadcon” interface, which has its own API and currently only functions due to mountains of gadget-specific workarounds that make safely adding any new functionality nearly impossible.
Presently, almost all gadgets and functionality that existed for the legacy gadget interface have been ported or rewritten for the new system, with future improvements already underway.

sudo/ssh Password GUI

A decision was made to implement a GUI for the ASKPASS feature of sudo/ssh which will pop up a dialog any time a password needs to be entered for these services.
More information about this feature can be found by referencing the SUDO_ASKPASS or SSH_ASKPASS environment variables.

Meson Build System

Meson is a relatively new build system that is significantly faster than autotools while providing a similar set of features. All modern distributions should be compatible with this build system, and autotools is planned to be removed after the E22 release.

Tiling Window Policy Improvements

Many issues related to this have been resolved, and some new features – such as a nicer window dragging UI – have been added.

Per-Window Pulseaudio Volume Controls

Pulseaudio exports volume controls for each audio-producing application, and work has been done to bind this into a GUI control.

New Features in Enlightenment 22 - e22-mixer

Stay Tuned for More!

More work is planned to improve the desktop experience in future versions of Enlightenment. Check out our contact page to find out more or get involved.

How to Create an EFL Gadget Sandbox

The new gadget API and infrastructure for Enlightenment continue to undergo heavy development. In addition to improving and extending the base gadget UI, work has recently begun on creating a gadget provider with the new API to provide sandboxing and allow gadgets to be written as regular applications that don’t have or require access to compositor internals.

The primary enabler of the new sandboxing system is the efl-wl compositor widget. This allows the compositor to launch applications in isolation, and also provides the ability to add protocol extensions for only that specific instance of the compositor widget. Using these features, it becomes possible to add gadget-specific protocols and utilities on the compositor side that are passed through transparently to the client gadget application.

Currently, there is one base protocol in use: the e-gadget protocol, which looks like this:



  
    
      
      
      
    
    
      
      
      
      
      
      
    
    
      
      
      
      
      
    
    
      
    
    
      
    
    
      
    
  


The purpose of this is to mimic the gadget API. Applications can choose to listen for events on the window object with the listed event name and receive the corresponding property value. This is made possible because of a shared library that is preloaded into applications by the gadget provider using the LD_PRELOAD environment variable. This preloaded library intercepts calls to create EFL window objects and converts Wayland protocol events into callbacks on the window object. Gadget application developers can then write applications without the need to concern themselves with anything related to Wayland.

Example EFL Gadget Sandbox

An example sandbox gadget written in EFL reads something like this:

#include < Elementary.h >
static Evas_Object * popup;
static Evas_Object * child;

static void
popup_del(void * data EINA_UNUSED, Evas * e EINA_UNUSED, Evas_Object * obj EINA_UNUSED, void * event_info EINA_UNUSED)
{
  popup = NULL;
}

static void
child_del(void * data EINA_UNUSED, Evas * e EINA_UNUSED, Evas_Object * obj EINA_UNUSED, void * event_info EINA_UNUSED)
{
  child = NULL;
}

static void
popup_unfocus(void * data EINA_UNUSED, Evas * e EINA_UNUSED, Evas_Object * obj, void * event_info EINA_UNUSED)
{
  evas_object_del(obj);
}

static void
mouse_button(void * data EINA_UNUSED, Evas * e EINA_UNUSED, Evas_Object * obj, void * event_info)
{
  Evas_Object * ic;
  Evas_Event_Mouse_Down * ev = event_info;
  char buf[PATH_MAX];
  int w, h;
  Evas_Object * win;
  Elm_Win_Type type = ELM_WIN_POPUP_MENU;
  if ((ev - > button != 1) && (ev - > button != 3)) return;
  if (ev - > button == 3)
  {
    type = ELM_WIN_BASIC;
    if (child)
    {
      evas_object_del(child);
      return;
    }
  } else
  {
    if (popup)
    {
      evas_object_del(popup);
      return;
    }
  }
  win = elm_win_add(elm_win_get(obj), "win", type);
  elm_win_alpha_set(win, 1);
  if (ev - > button == 3)
  {
    child = win;
    evas_object_event_callback_add(win, EVAS_CALLBACK_DEL, child_del, NULL);
  } else
  {
    popup = win;
    evas_object_event_callback_add(win, EVAS_CALLBACK_DEL, popup_del, NULL);
  }
  if (ev - > button == 3)
    evas_object_event_callback_add(win, EVAS_CALLBACK_FOCUS_OUT, popup_unfocus, NULL);
  ic = elm_icon_add(win);
  snprintf(buf, sizeof(buf), "%s/images/bubble.png", elm_app_data_dir_get());
  elm_image_file_set(ic, buf, NULL);
  elm_image_object_size_get(ic, & w, & h);
  evas_object_size_hint_aspect_set(win, EVAS_ASPECT_CONTROL_BOTH, w, h);
  if (ev - > button == 1)
  {
    elm_image_resizable_set(ic, EINA_FALSE, EINA_FALSE);
    elm_image_no_scale_set(ic, EINA_TRUE);
  }
  evas_object_size_hint_weight_set(ic, EVAS_HINT_EXPAND, EVAS_HINT_EXPAND);
  evas_object_size_hint_fill_set(ic, 0.5, 0.5);
  evas_object_size_hint_min_set(ic, 100, 100);
  elm_win_resize_object_add(win, ic);
  evas_object_show(ic);
  evas_object_show(win);
}

int
main(int argc, char * argv[])
{
  Evas_Object * win, * ic;
  char buf[PATH_MAX];
  int w, h;
  elm_init(argc, (char * * ) argv);
  elm_policy_set(ELM_POLICY_QUIT, ELM_POLICY_QUIT_LAST_WINDOW_CLOSED);
  elm_app_info_set(main, "elementary", "images/logo.png");
  win = elm_win_add(NULL, "icon-transparent", ELM_WIN_BASIC);
  elm_win_title_set(win, "Icon Transparent");
  elm_win_autodel_set(win, EINA_TRUE);
  elm_win_alpha_set(win, EINA_TRUE);
  ic = elm_icon_add(win);
  snprintf(buf, sizeof(buf), "%s/images/logo.png", elm_app_data_dir_get());
  elm_image_file_set(ic, buf, NULL);
  elm_image_object_size_get(ic, & w, & h);
  evas_object_size_hint_aspect_set(win, EVAS_ASPECT_CONTROL_BOTH, w, h);
  if (argc > 1)
  {
    elm_image_resizable_set(ic, EINA_FALSE, EINA_FALSE);
    elm_image_no_scale_set(ic, EINA_TRUE);
  }
  evas_object_size_hint_weight_set(ic, EVAS_HINT_EXPAND, EVAS_HINT_EXPAND);
  evas_object_size_hint_fill_set(ic, 0.5, 0.5);
  evas_object_size_hint_min_set(ic, 100, 100);
  elm_win_resize_object_add(win, ic);
  evas_object_show(ic);
  evas_object_event_callback_add(ic, EVAS_CALLBACK_MOUSE_DOWN, mouse_button, NULL);
  evas_object_show(win);
  ecore_main_loop_begin();
  return 0;
}

This code results in an E logo gadget like this:

How to Create an EFL Gadget Sandbox - gadget_sandbox1.png

Left clicking creates a popup window (xdg_popup) which the gadget provider extracts from the compositor widget to display like a normal gadget contextual popup.

How to Create an EFL Gadget Sandbox - gadget_sandbox2

Right clicking creates a child window (xdg_toplevel with parent) which the gadget provider extracts from the compositor widget to display like a standard gadget popup.

How to Create an EFL Gadget Sandbox - gadget_sandbox3

There are no toolkit or language restrictions for gadget applications using this functionality. The only requirement is that interacting with the compositor in any way, such as receiving the gadget properties, requires use of the Wayland protocol. Even without that, any framework or toolkit that has Wayland support can now have fully functional gadgets running inside Enlightenment.

The sandbox infrastructure can currently be found in the desksanity module, and is scheduled to be merged into the main tree after the E22 release.

Implementing a Multiseat Wayland Compositor into a Toolkit Widget

Wayland: there have been many blog posts, articles, and news items about the new Linux display protocol. There have been developers working to extend the protocol for new and extremely helpful use cases, but new frontiers continue to be explored: putting a multiseat Wayland compositor into a toolkit widget.

Due to Wayland’s philosophy of “build your own compositor,” there is no de-facto implementation of a Wayland display server that is analogous to Xorg for X11. Wayland implementations are written using the libwayland server API, and this is flexible enough to be used even in the case of a widget. Overall, the only noteworthy difference between putting a compositor in a widget and a normal nested compositor is that the output size is set to the size of the widget instead of the size of the window. With this in mind, a compositor widget can be manipulated just like any other widget, as it manages all protocol-related matters internally.

[iframe width=”560″ height=”315″ src=”https://www.youtube.com/embed/xZGYdg21IS0″ frameborder=”0″ allowfullscreen]

weston-terminal running in a widget in an X11 window.

Implementing Clipboard Sharing and Drag-N-Drop

This project had a number of difficulties along the way, most of them being related to clipboard and drag-n-drop support. A nested compositor of this type is significantly less useful if it lacks support for these things, and so it was necessary to add bridging for Wayland<->Wayland selection transfers as well as X11<->Wayland transfers. Clipboard selections were the easier part to handle in this case, as they do not require mouse/touch interactions and have no positioning data. For Wayland<->Wayland requests, all that’s needed is to transparently proxy all the data offer protocol exchanges to and from the parent compositor, and then pass any resulting file descriptors to the clients. Under X11, however, data must be manually read from or written to X11 window atoms and the selection requests need to be translated into Wayland equivalents.

[iframe width=”560″ height=”315″ src=”https://www.youtube.com/embed/o0a0I4Ao9Ng” frameborder=”0″ allowfullscreen]

Clipboard bridging under X11

Drag-N-Drop (DND) support was considerably more difficult. Not only does positioning affect the operation, but when the drag leaves the widget it becomes necessary to proxy the drag surface into a new window and initiate a global DND operation instead of working only within the widget. This leads to a new set of issues where buffer management for the drag surface must be adjusted to account for different render timings of the global drag window, which reuses the same buffer. The behavior is different between X11 and Wayland as well. Under X11, when dragging back to the originating widget, the global drag can be canceled. However, under Wayland the operation remains global and the result will be that the widget proxies a selection send through the outer compositor back to itself.

[iframe width=”560″ height=”315″ src=”https://www.youtube.com/embed/muTkLt1VNlw” frameborder=”0″ allowfullscreen]

DND bridging under X11

Adding full support for bridged DND also means that dragging between separate compositor instances also becomes possible.

[iframe width=”560″ height=”315″ src=”https://www.youtube.com/embed/lonwtDsz9VE” frameborder=”0″ allowfullscreen]

DND bridging between compositors under X11.

Another challenge with the process of putting Wayland into a canvas widget was supporting subsurfaces and other types of child surfaces. In Enlightenment, these are handled as regular objects which are stacked above the parent, but this makes the object tree more difficult to debug. The compositor widget does things a bit differently by making child surfaces into child objects of the parent surface, creating an object hierarchy which is much easier to read and debug.

[iframe width=”560″ height=”315″ src=”https://www.youtube.com/embed/u8bbTAVJirQ” frameborder=”0″ allowfullscreen]

Subsurfaces under X11.

Using the Compositor Widget to Sandbox Applications

The compositor widget has a number of useful features, but perhaps the most overlooked feature would be the ability to run applications in a sandbox. Any application launched inside has no ability to directly affect anything outside of the widget, meaning that questionable applications can be run in a safe environment from the display server’s perspective. While it has obvious uses in security, the more mundane usage could simply be embedding a web browser into an application without the need to connect to any browser engine’s embedding API.

Adding a compositor widget to an application can allow content to be embedded in isolation. It provides seamless integration with existing desktop interactions and requires no special handling to be placed into a UI layout. It also allows Wayland applications to run under X11, although this is perhaps not as useful since there are very few Wayland-only applications at present.

An Introduction to Relative and Constrained Pointers in Enlightenment

Recent development in Enlightenment’s Wayland compositor has focused on implementing cross-desktop protocols and improving stability. One of the recently handled protocol series has been relative and constrained pointers.

Relative pointer motion is a method for providing pointer movement deltas directly to applications. This is useful for a number of cases, though the easiest to imagine might be first person shooter games. In this case, the application receives movement deltas, allowing the player to endlessly scroll the screen in one direction without hitting the boundaries of the screen.

Under Enlightenment, this is handled in different ways depending on the output backend being used. For backends using libinput, e.g., DRM, it’s possible to get relative motion deltas directly from the events and then emit them for compositor use. Other backends, however, such as nested Wayland compositors, have no access to libinput’s hardware events. In this case, Enlightenment must manually calculate the deltas between motion events to pass to applications. This case is less precise, but for the case where there is no device directly attached to the compositor then there is no other method of getting relative motion data. In this way, all backends can now supply relative motion events when requested.

Pointer Confinement and Locking in Wayland

The pointer constraints protocol under Wayland consists of two separate concepts: pointer confinement and pointer locking. Pointer confinement, in this case, is where the pointer’s position is confined to a number of specified regions. Once confined, the pointer is unable to leave the specified region until the application releases the confinement or the compositor policy results in the confinement being released. Pointer locking, on the other hand, is when the pointer’s position is locked to a specific location, preventing motion events from being propagated to the client. A locked pointer similarly will not be unlocked either until the client releases the lock or the compositor policy requires the lock to end.

Confined pointers are implemented in Enlightenment primarily using calculation of motion deltas in relation to the confinement regions. By converting the deltas into simple linear equations, it becomes easy to determine whether the delta has occurred within the confined region as well as whether the pointer motion terminated within the confined region. If the motion would result in the pointer leaving the confined region, substitution into the equation allows for the calculation of the closest point to the end of the allowed confinement region, letting the pointer be confined as close to the hardware pointer position as possible.

An Introduction to Relative and Constrained Pointers in Enlightenment - confine

Locked pointers are perhaps the simplest to implement in any compositor: simply stop providing motion events to the pointer-focused surface once the lock begins.

All of these protocols are now implemented in Enlightenment as well as a number of other Wayland compositors, allowing for desktop applications to begin using the functionalities within the protocol.

A Curious Wayland Bug

Enlightenment has a slightly unconventional architecture. For many of its internal dialogues (settings, the file manager, etc.) it uses what we call “internal windows.” These are simply regular toolkit windows that use regular rendering back-ends; in the good ol’ days, this meant a connection to the X server which resulted in a render to a standard X window. In this scenario, the compositor gets the result back the same way as all X client windows, and it composites all internal and external windows together.

Under Wayland, things get a bit scarier because now the compositor is the display server. Enlightenment makes a Wayland connection to itself; this gives us all manner of game winning capabilities we never had before. For example, let’s say the internal window wants to sleep and wait for a reply from the compositor (such as a released buffer to render the next frame into). We deadlock and fall down a hole. But I digress.

Problems with internal windows also led us to this problem recently where the compositor exited, logged nothing, and gave only a cryptic message:

invalid object (3), type (zwp_linux_dmabuf_v1), message enter(uoa)

This prompts two questions

  1. What does that even mean?
  2. How do I debug something like this?

Since both client and compositor errors are logged in the same file, the first surprise was learning that this is a client error message. An internal window connection was closing itself on the client end because it received something from the compositor that didn’t make any sense.

This error message means the client received a message of the form: foo.enter(unsigned integer, object, array) where the object’s id was 3: a zwp_linux_dmabuf_v1 object. The client knows what type “object” should have been because it knows what type “foo” is (even though it doesn’t bother to include this bit of information in the log), and that type isn’t zwp_linux_dmabuf_v1. So, the client doesn’t have any idea about what to do with this event. It then has an existential crisis to ponder: what if that event is important in discerning the meaning of all future events? The only thing left for it to do is quietly disconnect and exit(); since this particular client is the same process as the compositor, it takes the rest of the desktop with it.

How do we Debug this Mess?

Well, a reasonable first step might be to add more instrumentation to libwayland-client to actually tell us the type of the object that received the bad message, but let’s be lazy.

Between the base Wayland protocol and extensions from the wayland-protocols repository, there are currently 6 possible “enter” events in Wayland. It doesn’t take too terribly long to match the signature from the error message – the (uoa) part – in the log for the keyboard enter event in the protocol XML file.

We know the client is exiting, but the client is likely not the location of the bug; rather, it seems to be exiting in response to garbage from the compositor. One of the several wl_keyboard_send_enter() call sites in enlightenment is probably the root cause.

In fact, what was happening is that under certain circumstances a change of keyboard focus wasn’t cleaning up all internal states, and a keyboard enter event was being sent to the wrong client. In some other client, object id 3 corresponded to a wl_surface object, which is the second argument that’s expected in a keyboard enter. It was actually fairly easy to sort out, but made much more difficult by the limited information in the log and the confusing way the client took the hit for a compositor bug.

Now that I’ve explained how to debug this error pattern, I’ll mention in passing that nobody else will have to quite these lengths again. We’ve recently landed changes to the wayland-server library which validate events before sending them to ensure it’s not mixing up objects from different clients. Now, the compositor will log an error with a much more coherent message instead of making it appear that the client is responsible for the wrong doings. This makes it easier to start the debugging process in the right place, and should help make future debug marathons shorter. The client still gets disconnected which is a little disruptive, but it’s a clean disconnect rather than an abort(), so the client is free to attempt a reconnect. Then, the server side prints an error message with the actual object interface name, making it so this kind of bug should be a little easier to sort out in the future.

Ecore_Evas: Creating a Canvas for Wayland Applications

While there are some developers who are familiar with using Ecore_Evas to create a canvas for applications, we often find that new EFL users face some confusion when first trying to create an application. This article aims to provide a simple example of how to create your first EFL Wayland application.

For those not familiar with the Ecore_Evas library, it is a set of functions that make it easy to tie together Ecore’s main loop and input handling to Evas; as such, it’s a natural base for EFL applications. While this combination makes it easy to create the basic aspects all applications need, for normal applications (those that use buttons, checkboxes and layouts) one should consider using Elementary. Ecore_Evas is extremely well suited for applications that are not based on widgets. It has a main loop that delivers events, does basic window handling, and leaves all of the drawing up to the user. This works very well if used in conjunction with Edje or when doing custom drawing such as what’s done for games, for example.

A typical example of an Ecore_Evas application may look like this

/**
 * Ecore example illustrating the basics of ecore evas usage.
 *
 * You'll need at least one Evas engine built for it (excluding the
 * buffer one). See stdout/stderr for output.
 *
 * @verbatim
 * gcc -o ecore_evas_basics_example ecore_evas_basics_example.c `pkg-config --libs --cflags ecore evas ecore-evas`
 * @endverbatim
 */

#include 
#include 
#include 

static Eina_Bool
_stdin_cb(void *data EINA_UNUSED, Ecore_Fd_Handler *handler EINA_UNUSED)
{
   Eina_List *l;
   Ecore_Evas *ee;
   char c;

   int ret = scanf("%c", &c);
   if (ret < 1) return ECORE_CALLBACK_RENEW;

   if (c == 'h')
     EINA_LIST_FOREACH(ecore_evas_ecore_evas_list_get(), l, ee)
       ecore_evas_hide(ee);
   else if (c == 's')
     EINA_LIST_FOREACH(ecore_evas_ecore_evas_list_get(), l, ee)
       ecore_evas_show(ee);

   return ECORE_CALLBACK_RENEW;
}

static void
_on_delete(Ecore_Evas *ee)
{
   free(ecore_evas_data_get(ee, "key"));
   ecore_main_loop_quit();
}

int
main(void)
{
   Ecore_Evas *ee;
   Evas *canvas;
   Evas_Object *bg;
   Eina_List *engines, *l;
   char *data;

   if (ecore_evas_init() <= 0)
     return 1;

   engines = ecore_evas_engines_get();
   printf("Available engines:\n");
   EINA_LIST_FOREACH(engines, l, data)
     printf("%s\n", data);
   ecore_evas_engines_free(engines);

   ee = ecore_evas_new(NULL, 0, 0, 200, 200, NULL);
   ecore_evas_title_set(ee, "Ecore Evas basics Example");
   ecore_evas_show(ee);

   data = malloc(sizeof(char) * 6);
   sprintf(data, "%s", "hello");
   ecore_evas_data_set(ee, "key", data);
   ecore_evas_callback_delete_request_set(ee, _on_delete);

   printf("Using %s engine!\n", ecore_evas_engine_name_get(ee));

   canvas = ecore_evas_get(ee);
   if (ecore_evas_ecore_evas_get(canvas) == ee)
     printf("Everything is sane!\n");

   bg = evas_object_rectangle_add(canvas);
   evas_object_color_set(bg, 0, 0, 255, 255);
   evas_object_resize(bg, 200, 200);
   evas_object_show(bg);
   ecore_evas_object_associate(ee, bg, ECORE_EVAS_OBJECT_ASSOCIATE_BASE);

   ecore_main_fd_handler_add(STDIN_FILENO, ECORE_FD_READ, _stdin_cb, NULL, NULL, NULL);

   ecore_main_loop_begin();

   ecore_evas_free(ee);
   ecore_evas_shutdown();

   return 0;
}

This example application will use the first available Ecore_Evas engine to create a canvas that contains a blue rectangle.

Ecore_Evas Creating a Canvas for Wayland Applications - shot-2016-12-13_08-42-35

While this is not a very exciting example, it does illustrate how to create a basic Ecore_Evas application quite well.

How to Use Wayland with Ecore_Evas

With a simple modification of one line, this example application can run in a Wayland environment. The following line is changed to facilitate this.

ee = ecore_evas_new(NULL, 0, 0, 200, 200, NULL);

To run in Wayland, it should read

ee = ecore_evas_new("wayland_shm", 0, 0, 200, 200, NULL);

This essentially tells ecore_evas to utilize the Wayland_Shm engine to create a new canvas; this is an Evas engine that uses Wayland and Shared-Memory for drawing. There are other Evas Wayland engines available, such as the "wayland_egl" engine which uses Hardware Accelerated drawing (namely EGL).

What many new EFL developers do not realize is that there is an even easier way to make this application run in a Wayland environment, without changing a single line of code! While running in a Wayland environment (Weston, Enlightenment-Wayland, Gnome-Wayland, etc) the following command should be run from the terminal

export ECORE_EVAS_ENGINE=wayland_shm

Then, the application can be run, and when ecore_evas_new executes it will check for the ECORE_EVAS_ENGINE environment variable and attempt to use the engine specified there.

This means application developers who utilize Ecore_Evas can easily make any application run in a Wayland environment. It's possible to choose to make it Wayland specific by modifying the function call that creates the canvas, or it's possible to simply export an environment variable to change the engine that's in use. Personally, I have always found it easier to set the environment variable because this makes it possible to test the application under various windowing systems (such as X11, Wayland, Framebuffer) without modifying and recompiling the code.

For more information on how to utilize Ecore_Evas to create a basic EFL application, please see the Ecore_Evas documentation.