How to Create an EFL Gadget Sandbox

The new gadget API and infrastructure for Enlightenment continue to undergo heavy development. In addition to improving and extending the base gadget UI, work has recently begun on creating a gadget provider with the new API to provide sandboxing and allow gadgets to be written as regular applications that don’t have or require access to compositor internals.

The primary enabler of the new sandboxing system is the efl-wl compositor widget. This allows the compositor to launch applications in isolation, and also provides the ability to add protocol extensions for only that specific instance of the compositor widget. Using these features, it becomes possible to add gadget-specific protocols and utilities on the compositor side that are passed through transparently to the client gadget application.

Currently, there is one base protocol in use: the e-gadget protocol, which looks like this:



  
    
      
      
      
    
    
      
      
      
      
      
      
    
    
      
      
      
      
      
    
    
      
    
    
      
    
    
      
    
  


The purpose of this is to mimic the gadget API. Applications can choose to listen for events on the window object with the listed event name and receive the corresponding property value. This is made possible because of a shared library that is preloaded into applications by the gadget provider using the LD_PRELOAD environment variable. This preloaded library intercepts calls to create EFL window objects and converts Wayland protocol events into callbacks on the window object. Gadget application developers can then write applications without the need to concern themselves with anything related to Wayland.

Example EFL Gadget Sandbox

An example sandbox gadget written in EFL reads something like this:

#include < Elementary.h >
static Evas_Object * popup;
static Evas_Object * child;

static void
popup_del(void * data EINA_UNUSED, Evas * e EINA_UNUSED, Evas_Object * obj EINA_UNUSED, void * event_info EINA_UNUSED)
{
  popup = NULL;
}

static void
child_del(void * data EINA_UNUSED, Evas * e EINA_UNUSED, Evas_Object * obj EINA_UNUSED, void * event_info EINA_UNUSED)
{
  child = NULL;
}

static void
popup_unfocus(void * data EINA_UNUSED, Evas * e EINA_UNUSED, Evas_Object * obj, void * event_info EINA_UNUSED)
{
  evas_object_del(obj);
}

static void
mouse_button(void * data EINA_UNUSED, Evas * e EINA_UNUSED, Evas_Object * obj, void * event_info)
{
  Evas_Object * ic;
  Evas_Event_Mouse_Down * ev = event_info;
  char buf[PATH_MAX];
  int w, h;
  Evas_Object * win;
  Elm_Win_Type type = ELM_WIN_POPUP_MENU;
  if ((ev - > button != 1) && (ev - > button != 3)) return;
  if (ev - > button == 3)
  {
    type = ELM_WIN_BASIC;
    if (child)
    {
      evas_object_del(child);
      return;
    }
  } else
  {
    if (popup)
    {
      evas_object_del(popup);
      return;
    }
  }
  win = elm_win_add(elm_win_get(obj), "win", type);
  elm_win_alpha_set(win, 1);
  if (ev - > button == 3)
  {
    child = win;
    evas_object_event_callback_add(win, EVAS_CALLBACK_DEL, child_del, NULL);
  } else
  {
    popup = win;
    evas_object_event_callback_add(win, EVAS_CALLBACK_DEL, popup_del, NULL);
  }
  if (ev - > button == 3)
    evas_object_event_callback_add(win, EVAS_CALLBACK_FOCUS_OUT, popup_unfocus, NULL);
  ic = elm_icon_add(win);
  snprintf(buf, sizeof(buf), "%s/images/bubble.png", elm_app_data_dir_get());
  elm_image_file_set(ic, buf, NULL);
  elm_image_object_size_get(ic, & w, & h);
  evas_object_size_hint_aspect_set(win, EVAS_ASPECT_CONTROL_BOTH, w, h);
  if (ev - > button == 1)
  {
    elm_image_resizable_set(ic, EINA_FALSE, EINA_FALSE);
    elm_image_no_scale_set(ic, EINA_TRUE);
  }
  evas_object_size_hint_weight_set(ic, EVAS_HINT_EXPAND, EVAS_HINT_EXPAND);
  evas_object_size_hint_fill_set(ic, 0.5, 0.5);
  evas_object_size_hint_min_set(ic, 100, 100);
  elm_win_resize_object_add(win, ic);
  evas_object_show(ic);
  evas_object_show(win);
}

int
main(int argc, char * argv[])
{
  Evas_Object * win, * ic;
  char buf[PATH_MAX];
  int w, h;
  elm_init(argc, (char * * ) argv);
  elm_policy_set(ELM_POLICY_QUIT, ELM_POLICY_QUIT_LAST_WINDOW_CLOSED);
  elm_app_info_set(main, "elementary", "images/logo.png");
  win = elm_win_add(NULL, "icon-transparent", ELM_WIN_BASIC);
  elm_win_title_set(win, "Icon Transparent");
  elm_win_autodel_set(win, EINA_TRUE);
  elm_win_alpha_set(win, EINA_TRUE);
  ic = elm_icon_add(win);
  snprintf(buf, sizeof(buf), "%s/images/logo.png", elm_app_data_dir_get());
  elm_image_file_set(ic, buf, NULL);
  elm_image_object_size_get(ic, & w, & h);
  evas_object_size_hint_aspect_set(win, EVAS_ASPECT_CONTROL_BOTH, w, h);
  if (argc > 1)
  {
    elm_image_resizable_set(ic, EINA_FALSE, EINA_FALSE);
    elm_image_no_scale_set(ic, EINA_TRUE);
  }
  evas_object_size_hint_weight_set(ic, EVAS_HINT_EXPAND, EVAS_HINT_EXPAND);
  evas_object_size_hint_fill_set(ic, 0.5, 0.5);
  evas_object_size_hint_min_set(ic, 100, 100);
  elm_win_resize_object_add(win, ic);
  evas_object_show(ic);
  evas_object_event_callback_add(ic, EVAS_CALLBACK_MOUSE_DOWN, mouse_button, NULL);
  evas_object_show(win);
  ecore_main_loop_begin();
  return 0;
}

This code results in an E logo gadget like this:

How to Create an EFL Gadget Sandbox - gadget_sandbox1.png

Left clicking creates a popup window (xdg_popup) which the gadget provider extracts from the compositor widget to display like a normal gadget contextual popup.

How to Create an EFL Gadget Sandbox - gadget_sandbox2

Right clicking creates a child window (xdg_toplevel with parent) which the gadget provider extracts from the compositor widget to display like a standard gadget popup.

How to Create an EFL Gadget Sandbox - gadget_sandbox3

There are no toolkit or language restrictions for gadget applications using this functionality. The only requirement is that interacting with the compositor in any way, such as receiving the gadget properties, requires use of the Wayland protocol. Even without that, any framework or toolkit that has Wayland support can now have fully functional gadgets running inside Enlightenment.

The sandbox infrastructure can currently be found in the desksanity module, and is scheduled to be merged into the main tree after the E22 release.

Ecore_Wl2: An EFL Library for Wayland Applications

Throughout the years of developing Wayland support for EFL, few EFL libraries have had as much impact on EFL Wayland applications as the Ecore_Wayland library has. This library was one of the first to make it possible to truly run EFL applications in a Wayland environment. As the years progressed, it became apparent that Ecore_Wayland had some shortcomings; this blog post will introduce you to the replacement for Ecore_Wayland, called Ecore_Wl2.

Ecore_Wayland’s Shortcomings

While testing our first Wayland implementation, it became apparent that the initial implementation of the Ecore_Wayland library had some drawbacks. Publicly exposed structures could not be changed easily without breaking existing applications, and any changes to existing Wayland protocols would require significant changes to our Ecore_Wayland library. It was also discovered that when an EFL Wayland application creates a new window, the backend library also creates an entirely new display and connection to the Wayland server. This resulted in high memory usage in EFL applications.

Due to these shortcomings, the community decided that EFL should have a better library for integrating with Wayland, and thus Ecore_Wl2 was born. Determined not to repeat the mistakes of the past, I began the long and arduous task of giving birth to a new library.

Building a More Capable Library

One of the first tasks with the new library was to move structures fields out of the public API so they could later be changed without breaking existing applications. As a result, in Ecore_Wl2 there are some publicly exposed structures (via Ecore_Wl2.h header file), however, the individual fields of these structures are not directly accessible. There are now public API functions to retrieve structure members to allow some access to these fields; this makes it possible to make necessary changes in the future while also maintaining existing applications.

The next area of focus for the new library was to reduce memory usage. In the older library, every window of every application would create it’s own Ecore_Wayland_Display: a huge waste of resources. I decided to implement display caching; this means the library code now maintains a cache of existing displays. When an application requests a display it calls ecore_wl2_display_connect():

EAPI Ecore_Wl2_Display *
ecore_wl2_display_connect(const char *name)

However, behind the scenes the library checks for any existing displays in the cache that match the given display name and return it. If no display is found in the internal cache, a new one is created and cached. This allows for display reuse without added memory overhead, especially for an application that intends to create more than one window. Since we use this new library in the server implementation, caching server displays has also been implemented.

While this blog post has briefly touched on the new Ecore_Wl2 library, I invite everyone to keep an eye open for another upcoming post concerning the Ecore_Wl2 library. It will go into more detail related to the improvements the library provides and how to make use of it.

Ecore_Drm2: How to Use Atomic Modesetting

In a previous article, I briefly discussed how the Ecore_Drm2 library came into being. This article will expand on that article and provide a brief introduction to the Atomic Modesetting and Nuclear Pageflip features inside the new Ecore_Drm2 library.

What Makes Atomic Modesetting and Nuclear Pageflip so Great?

For those that are unaware of what “modesetting” is, you may read more about it here. Atomic Modesetting is a feature that allows for output modes (resolutions, refresh rate, etc) to be tested in advance on a single screen or on multiple outputs. A benefit of this feature is that the given mode may be tested prior to being applied. If the test of a given output mode fails, the screen image doesn’t need to be changed to confirm a given mode works or not, thus reducing screen flickering.

Atomic/Nuclear Pageflipping allows for a given scanout framebuffer object and/or one or more hardware plane framebuffers to be updated in a single atomic operation. This means that updates to the screen output, whether they be updates of a given hardware plane, updates to a CRTC framebuffer, or even updates to rotation can happen in a single vblank interrupt. For the end user, this provides a much more friendly screen update by reducing redraws and flicker.

One of the things we strive to do with EFL is to keep our API simple to use. With that in mind, you will not notice any new API functions related to using Atomic Modesetting or Nuclear Pageflipping inside the Ecore_Drm2 library. You may be asking yourself right now: “Well, how do I make use of this then?” I have great news, you may already be using it :-). Rather than add new API functions, which need to be explicitly called for these features, I decided early that these benefits should come without any cost so that existing applications can make use of them without having to be modified.

The Technical Details

Inside the Ecore_Drm2 library there are various API functions that allow for setting output modes, or even sending a framebuffer object to scanout. These existing API functions have been modified to make use of Atomic Modesetting/Nuclear Pageflipping automatically where applicable. This means applications that are already using the various APIs of the Ecore_Drm2 library do not need to be changed.

Behind the scenes, the usage of Atomic Modesetting/Pageflipping is determined at runtime. This means that when the Ecore_Drm2 library gets initialized, it will make some checks to see if atomic usage is supported. Currently, these features are only supported on an Intel (i915) graphics card with a linux kernel version >= 4.8.0. If the Ecore_Drm2 initialization finds the proper driver and kernel version, it will then attempt to enable DRM_CLIENT_CAP_ATOMIC to support Atomic Modesetting/Pageflipping. If the client capability is successfully set, it will then attempt to enable DRM_CLIENT_CAP_UNIVERSAL_PLANES in order to enable hardware plane support.

Assuming all of the internal checks pass and the client capabilities are set successfully, the application will gain the added benefits of these features without any added work :-). There is one more thing that May be needed on some systems to get this working: it has been my experience that some boxes still need kernel boot-time options enabled for atomic to function. The necessary options (if required) are: drm.atomic=1, and i915.nuclear_pageflip=1.

While these features may not yet be available for all end-users to enjoy, the benefits they provide should be something that everyone can look forward to. With the continued hard work of linux kernel developers and graphics driver developers, we may all be able to enjoy these redraws soon :-)

Introducing the New & Improved Ecore_Drm2 Library

In the early days of developing Wayland support in EFL/Enlightenment, it was quickly apparent that EFL would need an abstraction library to interface with libdrm. We wanted users of EFL to be able to call simple functions without having to know about the underlying internals of libdrm, thus the original Ecore_Drm library was born. First, efforts to develop this library were undertaken with much enthusiasm and little fan-fare. After the birth of Ecore_Drm, we then proceeded to integrate it’s usage into some new Evas and Ecore_Evas engines so that the Enlightenment Desktop Shell could make use of it and render our first standalone Wayland desktop implementation.

After kicking the tires of our Wayland desktop for a while, we came to realize some shortcomings of the existing Ecore_Drm implementation. For starters, it would create it’s own Ecore_Drm_Device structure when launching the Enlightenment Wayland desktop (this structure was a representation of the physical connection to the /dev/dri card used in rendering). Along with this structure came lists of outputs, inputs, seats, planes, etc. In short, this structure was not very light and memory was being used up needlessly. Other shortcomings of the Ecore_Drm library included things such as structures being declared in public headers that couldn’t be easily changed, input device handling, systemd/logind support being coded directly into the library, and several other issues.

Redesigning From the Ground up

The community came to the easy decision that a better implementation was needed, thus we started efforts to create a new implementation and the Ecore_Drm2 library was born. This new library was developed slowly, while taking into account the shortcomings of the first iteration because we didn’t want to repeat the mistakes of the past. To help avoid these mistakes, the Ecore_Drm2 library API is currently marked as a ‘beta’ API which means it’s susceptible to changes at any time until the beta API marking is removed.

After several months of careful development and several months of testing, Ecore_Drm2 was deemed stable and complete enough to be pushed upstream. Efforts then ensued to modify the existing Evas and Ecore_Evas engines to use this shiny new library.

The new Ecore_Drm2 library fixes all of the shortcomings of the previous library. The community decided at inception that input handling should reside in it’s own library (thus the elput library was created). This removed a lot of internal overhead inside the Ecore_Drm2 library while also removing the need for the library itself to interface with systemd/logind. Public structures exposed via the Ecore_Drm2 library now have their actual implementations kept private so that changes could be made in the future if necessary. Internally, the size of the Ecore_Drm2_Device structure has been greatly reduced, saving memory.

While many improvements have been made to this new library (compared to the old one), one of the greatest overall improvements came as a result of it’s existence. With the new version of this library, we realized that the Evas and Ecore_Evas drm engines were also ripe for improvement. These rendering improvements did not happen by accident, but rather are the result of some great efforts, led by Derek Foreman, that provided tangible benefits by reducing tearing, decreasing time to first frame, improving rendering speed, and reducing memory usage. We believe we will see many more improvements just like this as a result of Ecore_Drm2.

Building Technical Talent in the Enlightenment Community by Solving Bugs

The past week or so has seen a significant amount of progress in the gadget backend of Enlightenment, due in no small part to the constant poking and prodding from Stephen Houston: our newest Samsung OSG Intern. As he mentioned in his post, we’ve known each other for quite some time now, so mentoring him has allowed both of us to skip over most of the pleasantries and get down to the code.

Establishing a Mutually-Beneficial Partnership

This type of internship is certainly new to me; seldom do I get the opportunity to sponsor and mentor a member of the community who has already been a contributor for such a long time. Given that I’d been the only one to use the new gadget API released in Enlightenment v21, I was curious to see what others would think after spending some time developing on top of it; soliciting feedback on the API during its development, as I did during my initial development work, hardly substitutes for the experience of spending time working with it in depth.

Before starting on his main gadget projects, Stephen decided he would try to adapt an existing gadget to the new API to test it out. We discussed this for a bit, and agreed it was a sound idea, so he set off to work. I think the first time I heard back from him was a few days after (keep in mind that he only works on this part-time in addition to his real job), and he was nearly done. He had found the new API to be satisfactory and easy to use, which was certainly a relief to me! This was, however, the point at which he brought up the sizing issue he posted a video of. We managed to work through it after a brief discussion of the problem, and he went on to tackle the last aspect of the gadget conversion: drag-n-drop (DND).

Solving Technical Problems While Growing Community Talent

Anyone who knows me or my past blogging knows that DND is not one of my favorite things to work on. While I typically have some choice words to describe it, the fact is that it’s a challenging subsystem to work within due to the necessary message passing between the drag initiator (the user) and the drop receiver. This leads to multiple places requiring debugging and testing anytime problems occur, adding to the complexity of solving problems. Solving DND bugs in a given area, however, is something like a rite of passage, and the tasks can only get easier from there… usually.

The DND issues here were numerous, and it turned out they were all either gadget infrastructure bugs or internal compositor bugs; in other words, I should be solving them to allow Stephen to continue progressing with his own work. I believe this is one of the great benefits of the way we’re running the internship program: in addition to sponsoring community members to continue doing the great work they have difficulty finding time for, we’re also providing the manpower behind the scenes to support that work and allow them to progress at their own pace, minimally hindered by any bugs outside the direct scope of their projects.

At this point, testing has been completed and Stephen’s new pager gadget has been merged into the upstream repository and is available for general use. He’s now already quite deep into the brand new launcher gadget, and I think we’ll have some exciting info to post about that soon. Stay tuned for more updates, as we have some new members of the Samsung OSG Internship program starting their projects within the next couple weeks!

Use the Sparse Semantic Parser to Spot Problems With Your Code

During my LinuxCon EU talk last year I briefly touched on the sparse semantic parser tool started by Linus Torvalds in 2003 (slide 7). While it might not be as powerful as other static analyzers I described, it still might be worthwhile to run on your code. Many distributions ship a sparse package already, which makes it easy to test. If not you might want to grab the latest tarball and build it yourself.

Once you have sparse installed, running it on your code should be easy as it provides a build wrapper around the CC environment variable. If you do not have any special requirements for CC in your build setup you should be able to run sparse like this:

make CC=cgcc

Use Filters to Find What Matters

Depending on your code, you might be overwhelmed by the amount of warnings and maybe errors sparse is producing. While you should have a look at all of them it helps on the practical side to filter out some warning classes to make sure you do not miss others. Sparse allows filtering with -Wsomething and -Wno-something command line options to enable or disable specific checks. To pass these options to sparse while using the build-wrapper we can put them into our CFLAGS.

CFLAGS="$CFLAGS -Wno-decl"

After this you need to make sure that the new CFLAGS are picked up by your build system, and after that a new sparse run should act accordingly to your newly set options.

For example a run in the EFL codebase showed a lot warnings about functions that should be static.

warning: symbol 'XXX' was not declared. Should it be static?

It was hard to find other warnings due to the sheer amount of those. After adding -Wno-decl to the CFLAGS for this run it was easier to spot others and allowing me to re-visit the functions which might need to be static later on.

Some Examples of Warnings You Might Find

When I ran this on EFL, one warning I hit was “Using plain integer as NULL pointer” which pointed me to two cases in our code where we used the integer 0 as the address we pointed mmap to.

mmap(0, ...)

instead of

mmap(NULL, ...)

I have not checked the mmap() internals about how such a case would be handled, but we should use it as intended. Interestingly these cases have been 2 out of 31 actual uses of mmap() from our code.

Another warning we saw in our code base was:

warning: non-ANSI function declaration of function

It points out that we declared functions without parameters in some cases:

foobar();

instead of

foobar(void);

For a definition this would mean that the function takes no parameters, but for a declaration this has a different meaning. The former one takes an unspecified number of parameters. You could, for example, have something like this:

foobar();

foobar(int param) { };

If you want to make sure your function really takes no parameters. you need to make sure to use the (void) syntax for declaration. In C++ the meaning is different, but my examples here refer to plain C code.

Make the Sparse Semantic Parser Work for You

While it’s possible to filter warnings with the compiler options, not all warnings can be disabled like this. The sparse man page covers warning options that are available. The signal to noise ratio is still a bit too much on the noise side for my own taste, but at least some things can be filtered out. Given the output is plain text, some shell magic could be used to filter the output to find the bits you are interested in.

In one of the next installations of this tools series I will look into the smatch tool, which uses sparse and was also developed for the Linux Kernel but can be used on other projects as well. Stay tuned and I hope you enjoyed this one and maybe even found some bugs in your code you could fix now.

Releasing Enlightenment Foundation Libraries 1.14

After three month of pleasure and pain, version 1.14 of the Enlightenment Foundation Libraries has finally been released. This is the sixth release I’ve managed as well as the sixth release to follow our time based release schedule.

How We Got Here

We aim for two months of development and one month of final stabilization. This can get problematic if we find problems late in the process, so we allow for some leeway here regarding the final release. We try to keep the delay within a week, and for 1.14 we have been two days late while chasing some of the bugs we considered to be show stoppers.

Setting such a short release cycle helps our users get quicker access to the newest features and fixes, and after 18 months of following this schedule, we seem to have found a good balance. To be successful with such a rapid release schedule, we continuously push our QA tools and infrastructure to keep up so we can find problems before we release, but this is a topic I’ll leave for another blog post.

The 1.14 release of EFL is the cooperative work of 77 authors who contributed over 1200 commits in a 12 week period to EFL in addition to another 300 commits to Elementary. Given the nature of time-based releases, we no longer wait for one big feature to get developed and finished before we finally release. Instead, we go with whatever we have ready at the given time.

Making Things Better

This release integrates Ector, a new retained rendering library that Evas will use to provide a vector graphics scene graph following the SVG specification. While not fully fleshed-out, the foundations are laid and we hope to get this working for the next releases.

Another area that has received significant changes is our Ecore DRM backend which speaks directly to libdrm. Its primary use is in Enlightenment for the Wayland compositor, and functionality has been added to support DPMS on outputs, set output, mark framebuffers as dirty, disable an output, output EDID parsing, and support libinput 0.8 and higher for the compositor.

Portability has also been improved, primarily for Windows and Mac OSX. While this is still a weak area for EFL, we have improved considerably over the last few releases. This could also be said for unit testing, where we are approaching a 40% function coverage with our current tests and are making plans to increase this significantly over the course of this year.

The Road Ahead

The merge window for 1.15 is already open and in less than a week we’ve received 236 commits, which leads me to believe this will be another busy release. These changes include improvements to the Elua library, which allows developers to use the Lua programming language in EFL applications. Lua is a simple scripting language that is easy to embed and allows developers to extend their application functionality.

Furthermore, numerous NEON intrinsics have been implemented to speed up various Evas operations on ARM hardware. Additionally, more changes to the Ecore DRM backend should allow the Enlightenment Wayland compositor to become more feature complete.

Last, but not least, a better debug infrastructure for backtraces and foundations for application profiling have been added to our code base and will hopefully be extended over the 1.15 release cycle. We aim for the first Monday of every third month for release, which makes the 3rd of August our goal for 1.15.