WOW, Wayland Over Wire!

A common complaint about Wayland is that it isn’t network transparent. X allows you to run an application on one computer and display its output on a different computer as long as the application doesn’t depend on certain “modern” features (such as the shared memory extension). Applications are forwarded individually and are indistinguishable from apps on the local desktop (network performance considerations aside). This is different than remote desktop protocols like VNC or RDP which provide control of an entire operating system.

The Hurdles to Wayland Network Transparency

Obviously, if we intend to make Wayland a replacement for X, we need to duplicate this functionality. However, some Wayland design decisions make it difficult to implement this kind of network transparency:

  • File descriptor passing is used extensively in the protocol. For example, keymaps are passed from the compositor to the client as file descriptors; the client is supposed to mmap() the file descriptor and treat it like an array. This sounds a little silly at first, but a keymap is actually a fairly large chunk of data that’s too large to fit in a single Wayland message packet (they can’t be larger than 4Kb). Obviously you can’t (usefully) pass a file descriptor over the network.
  • Keyboard repeat is handled on the client side, so if you have a dodgy network connection and a key press packet arrives but the key release packet is delayed, the client would start repeating keys.
  • Buffers are shared between the client and compositor… somehow. They could be shared memory through mmap or dmabuf, or it could be mesa’s buffer extension. In any event, image data is never pushed to the compositor through a network socket, it always takes the form of some kind of handle that both the client and compositor understand.  These handles are local only and are meaningless over a network.

Otherwise, there’s really not a lot about the Wayland protocol that makes it local only. In fact, it’s designed to be asynchronous and minimize round trips; this is great for remote connections.

Wow! Introducing Wayland Over Wire!

So, I’ve prepared a really simplistic implementation of Wayland network transparency, or Wayland Over Wire, which essentially spits the Wayland protocol over a TCP/IP socket. Since Wayland only carries handles to image data in its messages, I’ve added some new messages to take care of this. I’ve also taught libwayland-client about buffers, which is probably a controversial move. I’ve done this by adding client side “hooks” to certain requests.

A hook is a function that’s called just before the request is sent, and passed the same parameters. This lets libwayland-client itself track the damage instead of just being a transparent proxy between the compositor and client. The end result is that any compositor (I’ve tried both Weston and Enlightenment) can use this to provide remote apps with only 1 line of code added to the compositor:

wl_display_add_remote_socket(display, "foo");

Since this is an epic, gaping security hole, I’m not going to publish a patch to enable this, but feel free to add that line to your compositor and give it a try  (hint: in Weston it’s right before load_modules in main). Just keep in mind that this opens a network port that lets anyone connect directly to your compositor: something that could potentially create major security risks.

Work Left to be Completed

Since this is a proof of concept/request for comments, I’ve bailed on some of the less interesting (though hugely important) problems like authentication, port selection, ipv6 support. Additionally, cut and paste is not yet implemented.  There’s also a list of other minor (and not so minor) issues, including:

  • Clients must use the wl_shm buffer type to work, meaning EGL and dmabuf won’t work.
  • Damage tracking is terrible. Instead of doing anything at all intelligent I simply use a rectangle that contains all damage. If two points at opposite corners of the window change at the same time, the whole frame will be sent.
  • There’s no compression on image data, so this will only work reasonably on relatively fast network connections. I must confess to have only tested it on Gigabit-Ethernet.

Those issues can be resolved and refined, but there’s still something that’s always going to suck with this approach: I’m not sure how to get around the client side key repeat problems. Very clever toolkits could hold off on generating repeated keys until they get a frame callback, but this could result in deferring a lot of processing until it’s too late to finish the rendering before the next screen refresh.

I’ve just recently posted the patch series on the Wayland mailing list, so feel free to give it a try, but please keep in mind this is definitely “use at your own risk” stuff. As I said, anyone can connect to that network port so be careful how you use it, and I don’t recommend using it in production. If you have any feedback to provide, feel free to share it on the mailing list, or chat with us in #wayland on Freenode.

Author: Derek Foreman

Derek has a long history of writing graphics software and has spent much of his career working in open source.

12 thoughts on “WOW, Wayland Over Wire!”

  1. The easiest solution to key repeat is disabling key autorepeating when remoting. That’s what x11vnc did. But then toolkits must be aware that the display is a remote one.

    1. Glad to see it! I use X’s networking transparency quite a bit. I hope that someone will finish the task, e.g., creating a secure it-just-works by default version of this.

      Good luck!

  2. About the security: What if unix socket is used instead of TCP socket? SSH can forward unix sockets too and it will solve all security and network related concerns you mentioned (authentication, port selection, ipv6 support), since the unix socket is protected by unix permissions just like any other files. SSH can also handle compression (at least until a better solution is implemented).

    1. I didn’t know you could forward unix sockets with ssh – that’s really cool. :)

      That would be a nice way to pass on all the security concerns to someone else. We’d probably still need a separate socket explicitly for remote connections (so the compositor knows the client can’t do anything useful with file descriptors), but socket forwarding would definitely fix a lot of the missing bits.

      I have already played a little with forwarding the tcp connection via ssh -C, and it does help a bit.

  3. Yes, PLEASE look into passing it over SSH. NX solved some of the issues by sticking a proxy between X and the wire. I get great performance from X2GO, I really like that the clients have the SSH library built in on certain platforms. Makes it easy to remote into linux boxes from windows without subpar performance.

  4. As an heavy user of X11 over network (with “ssh -XC” magics), many thanks for pushing this! It’s a big and IMO underestimated drawback in current Wayland design: How such a basic component could be successful when any unices since the 70’s are built around the network stack?

    1. Wayland network transparency will likely be handled at the compositor level, and not in libwayland, so this work never evolved beyond a proof of concept.

Comments are closed.