Web, Internet of Things and GDPR

From the 25th  of May 2018, the European General Data Protection Regulation, also known as GDPR, has come into force across European Union (EU) and the European Economic Area (EEA).  This new European Regulation on privacy has far-reaching consequences on how personal data is collected, stored and used in the  Internet of Things (IoT) world and across the web. For IoT and web developers, it has a significant impact on how we start to think about technologies we deploy and services we build. One promise of the open web platform is enhanced privacy in comparison to other technology stacks. Can some of the inherent privacy issues in the currently deployed IoT architectures be mitigated by the use of web technology? We think it can.

In this blog post we will look at The Web of Things (WoT) technologies we have been involved in. By walking through the options from local device discovery to a gateway framework that enables remote access, we hope to give you a picture of a loosely coupled IoT solution using Web of Things technologies. At each stage, GDPR compliance of these technologies will be discussed.

GDPR and the Internet of Things

The 88 pages long GDPR regulation document has 99 articles on the rights for all individuals and the obligations placed on organizations. These new requirements represent a significant change on how data will be handled across businesses and industries. When it comes to the Internet of Things, GDPR compliance is particularly challenging. Concerns about the risks that the Internet of Things has posed on data protection and personal privacy have been raised for a few years.  An study from ICO in 2016 stated that “Six in ten Internet of Things devices don’t properly tell customers how their personal information is being used”. The good news is that industry and governments are aware of the problem and taking actions to change this.

With the Internet of Things growing faster and smarter, a lot of new devices and new solutions will be introduced in the coming years. When looking into technical solutions and innovation, we need to have “GDPR awareness”, especially “data protection by design” that GDPR advocates, in mind.  “Data protection by design“, previously known as “privacy by design”, encourages business to implement technical and organizational measures, safeguard privacy and data protection principles within the full life cycle right from the start.

The Web of Things

Interoperability is one of the major challenges in the Internet of Things. The Web of Things addresses this challenge by providing  a universal application layer protocol for the Things to talk to each other regardless of the physical and transport layer protocols used. Rather than reinventing the wheel, the Web of Things reuses existing and well-known web standards and web technologies. The Web of Things addresses Things via URLs and following standard APIs. Using the web as the framework makes things discoverable and linkable, and provides web developers with an opportunity to let people interact via a wide range of interfaces: screens, voice, gesture, augmented reality… even without an Internet connection!

Web of Things has been referred to as the application layer of the Internet of Things on many occasions. Bearing in mind though, the scope of IoT applications is broader and includes systems that are not necessarily accessible through the web. We prefer to think  that the Web of Things is an option of an application layer added over the network layer of the traditional Internet of Things architecture.

The Web of Things is growing fast in its standardization and implementations. The W3C has launched the Web of Things Working Group to take a leading role at addressing the challenge of fragmentation of the IoT through standards. Early this year Mozilla announced the “Project Things” framework – an open framework of software and services that bridges the gap between devices via the web. The “Project Things” is a good reflection of Mozilla’s involvement in creating an open standard with the W3C around the Web of Things, and practical implementations that provide open interfaces to devices from different vendors.

For developers, the fast growth of the Web of Things has opened up a lot opportunities when we search for solutions for the Internet of Things.  Let’s walk through an example to see what this means.

Imagine that you are a homeowner and just bought a new washing machine. To have the new device integrated into your smart home and be able to control or access it, here are some optional solutions that the Web of Things can offer:

  • Discover, pair and connect new devices around you via Physical Web and Web Bluetooth.
  • Control devices via a Progressive Web Application
  • Communicate with your device via an end-to-end encrypted channel provided by Mozilla Web of Things Gateway, locally and remotely.

So what are these technologies about? How do they address privacy and security concerns? Let’s walk through them in a bit more detail.

Physical Web and Web Bluetooth

The Physical Web is a discovery service based on Bluetooth Low Energy (B.L.E.) technology. In the Physical Web concept, any object or location can be a source of content and addressed using a URL. The idea is that smart objects broadcast relevant URLs to nearby devices by way of a BLE beacon.

The Physical Web has been brought into Samsung’s web browser, Samsung Internet, as an extension called CloseBy. When the browser handles the URL information received on the phone,  “no personal information is included and our servers do not collect any information about users at all,” stated my colleague  Peter O’Shaughnessy in his article on “Bringing the real world to your browser with CloseBy”.

Web Bluetooth is another technology based on Bluetooth Low Energy. Alongside efforts like the Physical Web, it provides a set of APIs to connect and control B.L.E. devices directly from web. With this web API, developers are able to build one solution that could work across all platforms. Although the API is still under development, there are already a few very cool projects and applications around. My colleague Peter has produced  a Web Bluetooth Parrot Drone demo to give you a sense of controlling a physical device using a web browser. And I promise you that playing Jo’s Hedgehog Curling game is simply light and fun!

The Web Bluetooth Community Group aims to provide APIs that allow websites to communicate with devices in a secure and privacy-preserving way. Although some security features are already in place, for example, the site must be served over a secure connection (HTTPS) and discovering Bluetooth devices must be triggered by a user gesture , there are still lots security implications to consider as per the Web Bluetooth security model by Jeffrey Yasskin.

Samsung Internet had Web Bluetooth enabled for developers by default since v6.4 stable Release.  With Physical Web and Web Bluetooth available, it is possible to have device on-boarding “just a tap away”.

Progressive Web Application

Progressive Web Apps (PWAs) are websites that deliver native app-like user experiences. They address issues in native mobile applications and websites with new design concepts and new Web APIs. Some key features of PWAs include:

  • “Add to Home Screen” prompts
  • Offline functionality
  • Fast loading from cache
  • (Optionally) web push notifications

These features are achieved by deploying a collection of technologies. Service Workers are the core of Progressive Web Apps and work in the background to power the offline functionality, push notifications and other striking features. With the app shell concept, PWAs can achieve a fast loading time by caching the basic shell of an application UI, while still loading fresh content when possible. The native install banner installs the mobile website to the homescreen with Web App Manifest support.

In PWAs, you can only register service workers on pages served over HTTPS. Since service workers process all network requests from the web app in the background, this is essential to prevent a man-in-the-middle attack. PWAs can work offline as we mentioned earlier. From privacy perspective, this potentially offers us possibilities to:

  • Minimize Collecting, storing, and using user data as much as possible.
  • Know where the data resides.

Since the term “Progressive Web Apps”  was first coined by Google in 2015,  we have seen big brands such as FT, Forbes and Twitter switching to PWAs in the last few years. Our Samsung Internet Developer Advocacy team has been actively contributing to and promoting this new technology [1] [2] [3] [4] [5], and even designed a community-approved Progressive Web Apps logo! Want to have hands-on experiences of PWAs? Check our demos on Podle, Snapwat, Airhorn VR, PWAcman and Biscuit Tin-der.

Mozilla “Project Things”

Mozilla “Project Things” aims at “building a decentralized ‘Internet of Things’ that is focused on security, privacy, and interoperability”, as stated by the company. The structure of the framework is shown as below –

  • Things Cloud. It provides a collection of IoT cloud services including supports for setup, backup, updates, 3rd party applications and services integration, and remote encrypted tunneling.
  • Things Gateway. Generally speaking the Things Gateway is the IoT connectivity hub for your IoT network.
  • Things Controllers. Smart devices such as smart speakers, tablets/phones, AR headsets etc, to control the connected Things.
  • Things Device Framework. It consists of a collection of sensors and actuators, or “Things” in the context of the Web of Things.

The “Things Project” has introduced an add-on system, which is loosely modeled after the add-on system in Firefox, to allow for the addition of new features or devices such as an adapter to the Things Gateway. My colleague Phil Coval has posted a blog explaining how to get started and establish basic automation using I2C sensors and actuators on a gateway’s device.

The framework has provided security and privacy solutions such as:

  • Secure remote access is achieved using HTTPS via encrypted tunneling. Basically, the “Things Project” provides a TLS tunnelling service via its registration server to allow people to easily set up a secure subdomain during first time setup . An SSL certificate is generated via LetsEncrypt and a secure tunnel from a Mozilla cloud server to the gateway is set up using PageKite.
  • The Things Gateway provides a system for safely authorizing third-party applications using the de-facto authorization standard OAuth 2.0. When a third-party application needs to access or control another person’s Things, it always requires consent from the Things’ owner. The owner can decide the scope of the access token granted to the third-party application. Things’ owner also has options to delete or revoke the tokens that are assigned to the third-party application. Details on this have been discussed at our recent blog “An End-to-End Web IoT Demo Using Mozilla Gateway” and talk “The Complex IoT Equation.

Future work

GDPR challenges us all to re-prioritize digital privacy and to reconsider how and when we need to collect, manage and store people’s data. This is an opportunity for the the Web of Things. As the technology develops, some of the security and privacy issues have been or being addressed. Still, building the Web of Things has various challenges ahead. For developers, using the right technologies are a way forward to make the Internet of Things a better place. Join us on this exciting journey!


An Introduction to IoT.js Architecture

IoT.js is a lightweight JavaScript platform for the Internet of Things. The platform keeps interoperable services at the forefront, and is designed to bring the success of Node.js to IoT devices like micro-controllers and other devices that are constrained to limited storage and only a few kilobytes of RAM. IoT.js is built on top of JerryScript: a lightweight JavaScript interpreter, and libtuv: an event driven (non-blocking I/O model) library. The project is open source under the Apache 2.0 license.

This article will introduce you to the architecture of IoT.js and the fundamentals of writing applications for it.

IoT.js Architecture

An Introduction to IoT.js Architecture - iotjs_arch

JerryScript – ECMAScript binding

JerryScript is the kernel for IoT.js on an ECMAScript binding, it’s an ultra lightweight JavaScript engine that was written from scratch at Samsung. The name “Jerry” comes from the popular character in Tom and Jerry, who’s small, smart, and fast! Since the engine is an interpreter only, it might be more precise to say JerryScript is a JavaScript interpreter.

Optimizations for memory footprint and performance have always been the top priorities of the project. The tiny engine has a base RAM footprint of less than 64KB, and the binary can accommodate less than 200KB of ROM. Amazingly, it implements the full ECMAScript 5.1 standard, and work has been ongoing recently to introduce new ES6 features such as promise, TypedArray, and more.

Like IoT.js, JerryScript was also released under the Apache 2.0 license. The community has experienced very rapid growth, especially in last couple years, and in 2016 the project was transferred to the JavaScript Foundation. A recent JavaScript Foundation press release mentioned how JerryScript was adopted in Fitbit’s latest product: Ionic.

JerryScript provides a good set of embedding APIs to compile and execute JavaScript programs, access JavaScript objects and their values, handle errors, manage the lifestyles of objects, and more. IoT.js uses these API’s to create the builtin module and native handler in IoT.js native core.

libtuv – I/O Event binding

Asynchronous I/O and threading in IoT.js are handled with libtuv: a library that focuses on asynchronous I/O and was primarily developed for use with Node.js. Samsung launched this open source project under the Apache 2.0 license, and it’s a multi-platform tiny event library that’s refactored from libuv to better serve IoT.js and on embedded systems.

Libtuv’s features include a loop, timer, poll, tcp & udp, fs event, thread, worker, and more. The platforms this library supports include i686-linux, arm-linux, arm-nuttx and arm-mbed.

IoT Binding

In the IoT.js community, there have been discussions about binding to an existing IoT platform or specification, such as IoTivity: The Open Connectivity Foundation‘s open source project. If this were to happen it would certainly add more dimension to supporting interoperability with other platforms.

IoT.js C Core

The IoT.js core layer is located above the binding layer, and it provides upper layer functionality to interact with the JavaScript engine, including running main event loops, managing I/O resources, etc. It also provides a set of builtin modules and native handlers.

Builtin modules are the basic and extended modules that are included in the IoT.js binary. Basically, these builtin modules are JavaScript objects implemented in C using the embedding API JerryScript provides, in either JavaScript or both languages. The native components of builtin modules are implemented as a native handle to access underlying systems via event handling, a C library, or system calls.

The life cycle of IoT.js is shown below:
An Introduction to IoT.js Architecture - iotjs_lifecycle-1

IoT.js ECMAScript API and JavaScript Modules

Like Node.js, IoT.js is a module-based system. Each module in IoT.js has its own context and provides a set of API’s associated with the module’s functionality.

IoT.js offers basic API modules and extended API modules. The basic API modules are based on Node.js and follow same form for compatibility reasons. Basic API modules include File System, Net, HTTP, Process, etc. Pretty much all application code that calls these API’s can be run in a Node.js environment without any modification.

The extended modules, on the other hand, are more IoT.js specific, and they are currently mostly hardware related (e.g. GPIO, Bluetooth Low Energy (BLE), I2C, SPI, UART, PWM, etc.). Many contributors are interested in adding new extended API modules to support their own specific hardware, so to maintain consistent usability, the IoT.js community has set guidelines and rules for introducing extended API’s.

Enabling JavaScript on the Internet of Things

The overall architecture of IoT.js is very friendly to Node.js, as a result of the asynchronous I/O and threading library, and the subset of Node.js compatible modules; it has reflected the design philosophy of providing a lightweight version of Node.js along with an inter-operable service platform. IoT.js has opened a great opportunity for JavaScript developers to develop applications for the Internet of Things, so it’s definitely an IoT platform to watch!

How We Simplified X.org Foundation Administration with SPI

The X.org Foundation is a non-profit governance entity charged with overseeing core components of the open source graphics community. X.org had been structured as a legal (non-profit) corporate entity registered in the state of Delaware for some years, which provided tax deduction on donations and other such benefits.

Unfortunately, being a non-profit is not cheap and entails various administrative tasks – filing annual reports, maintaining a bank account, dealing with donations and expenses, and so on – so the overhead of being an independent non-profit was deemed not worth the benefits, and in 2016 the members voted to join Software in the Public Interest (SPI). Joining SPI made a lot of sense; primarily, it would relieve X.org of administrative burdens while preserving the benefits of non-profit status. The costs of being in SPI are offset by the savings of not having to pay the various fees required to upkeep the corporate entity ourselves.

In that 2016 vote, I was elected to a position on the board to serve in the role of treasurer. I took on the tasks of establishing our financial services with SPI and closing down our Delaware registration. These seemingly straightforward administrative tasks proved to turn into a much more involved ordeal.

The Challenges of Dissolving a Non-Profit

Initiating things with SPI was complicated by the fact that I was new to the board, as well as the usual response delay endemic with volunteer-driven efforts. With some email back-and-forthing, we were added to SPI’s website, established an X.org “Click&Pledge” bucket for credit card donations, and later established Paypal donations as well.

The X.org Foundation’s cash assets were held in a Bank of America (BofA) account registered under the name of a previous treasurer. Unfortunately, BofA’s security policies meant that only the old treasurer could close out the account; fortunately, he was still available and able to do this. A lesson learned for other non-profits would be to ensure multiple people are registered for the organization’s bank accounts.

Once the assets were transferred it was time to close down the foundation itself, or “dissolution” in corporate terms. Delaware had a byzantine array of forms, fees, and procedures, and with this all being completely foreign to my experience, mistakes were inevitable. They kindly provide PDFs with fillable fields, however while some forms can be filed online, the PDFs have to be printed out and sent by mail. I was confused on this point and sent the filled in PDF to them online; unfortunately their system’s PDF processing system doesn’t support fillable fields and strips them out, resulting in invalid submissions.

Another lesson I learned is that the Division of Corporations does not notify you in case of errors, so I did not discern the failure for months and it took a number of phone calls to identify the problems and correct them. What brought the failure to my attention was when we attempted to terminate services with our registered agent. I’m still not exactly sure what a registered agent is, but corporate registration in Delaware requires it. X.org had a contract with National Registered Agents, Inc., who (once I’d squared away the dissolution paperwork) were pleasant to work with to close out our account with them.

It’s hard not to think of this transition as the end of one era and the beginning of another, and I’m very optimistic about the new capacity that the X.org Foundation has via SPI for receiving and processing donations. We’re now well positioned for organizing fund-raising activities to support X.org’s events and student sponsorships, and hopefully the information provided in this article will be beneficial to other open source communities facing similar problems.

How to Use an External Server with OpenVPN to Escape NATs

I have recently moved to a new flat and I love it, though unfortunately not all is perfect. One might expect central London (literally 500m away from the actual geographical centre) to have the best internet connection London has to offer. Well, one would be wrong; I am no longer able to get anything better than lousy ADSL2+, and this amazing offering comes with a high price, a long contract, and a month’s wait for the installation. This has led me to choose a 4G internet provider; the connection is usually better than what I would have expected to get with any of the landline providers, is much cheaper, and I had it up and running less than 24hrs after I joined.

However, there is a problem with this: the 4G provider uses a carrier-grade NAT, that made impossible to access my home server from outside my home network. Luckily, I have a VPS that is not behind a NAT so the solution was obvious: tunnel everything through it.

The Solution

One easy solution to my problem could be to run a VPN, connect my home server to it, and then connect to the VPN every time I want to access my home server. This solution works, but it’s annoying to connect to the VPN every time and even more annoying to give VPN access to anyone who needs access to my private Git server. This led to me choosing a slightly more complex solution that solves this issue: forward specific ports from a VPN that has a unique external IP address to my home server.

Setup OpenVPN

From now on I’ll call the externally available server EXT and the one I’ll expose HOME. First, it’s necessary to setup an OpenVPN server on EXT, however, setting up OpenVPN is beyond the scope of this post. I recommend following the Arch Linux guide, and the OpenVPN official documentation.

With that being said, there are a few important OpenVPN configurations you should make sure to have in your OpenVPN server (EXT):

# ... SNIP ...
# Allow client to client communication
# Send pings to keep connection alive
keepalive 15 60
# Persistent ip addresses
ifconfig-pool-persist /etc/openvpn/ipp

At this point you should make sure you can connect to the VPN and access services running on HOME from EXT.

Setup Port Forwarding

The first thing to do when setting up port forwarding, is to enable it in the kernel (on EXT). Sadly, since we still live in an IPv4 world (this is why we have CGNAT in the first place), I’ll only deal with the IPv4 case.

As root:

# Enable on the running system (temporary)
$ sysctl -w net.ipv4.ip_forward=1
# Enable on boot (depends on your distribution)
$ echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.d/51-ip.conf

The last step is to tell iptables which ports should be forwarded.  For the sake of the example, I’ll forward TCP port 2222 for ssh and UDP ports 61001-62000 for Mosh. The first step is to check HOME’s ip address using the command ip addr show.

Then, complete the following commands as root, for this example HOME’s ip address is

# Allow IP forwarding
# Forward the relevant ports to HOME
$ iptables -A PREROUTING -p tcp --dport 2222 -j DNAT --to-destination
$ iptables -A PREROUTING -p udp --dport 61001 -j DNAT --to-destination

# Enable connections to and from the machine (in case restricted)
$ iptables -A FORWARD -d -p tcp --dport 22 -j ACCEPT
$ iptables -A FORWARD -s -p tcp --sport 22 -j ACCEPT
$ iptables -A FORWARD -s -p udp -m state --state RELATED,ESTABLISHED --sport 61001:62000 -j ACCEPT
$ iptables -A FORWARD -d -p udp --dport 61001:62000 -j ACCEPT

Important note: commands like sysctl and iptables only last until the next boot, so don’t forget to make them persistent in whatever way is recommended for your distribution.

Verify Functionality

Connect to a network different than the one your home server is on, and try to connect to ssh using port 2222 with EXT’s ip/hostname. For example:

$ ssh -p 2222 user@EXT

You should now be successfully logged into HOME.

One Additional Trick

One annoyance that comes with this method is that all traffic to HOME is passed through EXT, even if the user’s device has direct access to HOME by being on the same LAN. This is due to the fact that the DNS name and ip address you are now using to connect to HOME is actually EXT’s.

One simple solution is to use a different domain name depending on whether or not the device has direct access. This is awfully annoying, especially when using git.

The better solution I chose for my setup is to configure DNS to resolve differently inside my home network, many routers support configuration for this. So for example, I use git.stosb.com for my git server. The public DNS is a CNAME to stosb.com, but inside my network, it resolves to HOME’s local address, which means everything will just connect to it directly.

Final Notes

I would love to hear if you have any corrections or better ideas on how to implement this. I would especially love to hear if you have any comments on how to harden this setup even further. I have a feeling the iptables rules could be a bit more strict. Please let me know.

This article was originally posted on Tom’s personal blog, and has been approved to be posted here.

How to Retain History When Moving Files Between Git Repositories

As my friends and colleagues know, I think the history of a project is very important for development. Being able to bisect, blame or read the log have proven very useful for finding bugs and understanding why a piece of code was written the way it was. Therefore, it makes sense to do whatever is possible to make sure history is preserved when moving files across repositories.

Luckily for us, git has made it extremely easy.

Merging Repositories

Merging a repository (bar) into another repository (foo) is easy.

$ cd /path/to/foo
$ # To use a local copy, replay the url with: file:///path/to/bar/.git
$ git remote add bar https://git.domain.com/bar.git
$ git fetch bar
$ git merge bar/master

This is it. It is very simple and retains all of the history from bar while maintaining the same commit hashes! This means that for example daed567e will point to the same commit in both foo and bar.

Unfortunately it is not always that simple. Sometimes you may face conflicts, if for example you had a README file in both repositories, the merge operation will fail. Luckily, this is also easy to solve.

First, abort the failed merge (if you already tried to merge):

$ git merge --abort

Now switch to a temporary branch that holds bar:

$ git checkout -b barmaster bar/master

Now you can deal with the conflicting files by either removing them, moving all of bar into a directory such as bar_directory or renaming them individually.

We can finally switch back to master and merge our branch again:

$ git checkout master
$ git merge barmaster

We’re done! Do not forget to push your changes.

Splitting Repositories

Splitting repositories is slightly more involved compared to merging them because we would like to remove all of the unrelated files and commits from history so our new repository is clean.

There are two approaches for this stage. The whitelist (we only keep a list of files) and the blacklist (we keep everything except for the list of files). I prefer the whitelist approach, so I will only cover it in this article.

For this example we will split bar out of foobar.

Let us first start by switching to a temporary branch we can work on.

$ git checkout -b tmp

Now we need to decide which files we would like to preserve.

Optional: Retain files that have been renamed throughout history.

If we have a file called a that has been renamed to b at some point in history, we would like to preserve both a and b. A useful command to find all of the past names of a file is:

$ git log --name-only --format=format: --follow -- path/to/file | sort -u

Just add both names of the file into the script we will create below.


Now we will create a script that moves the correct files into a new temporary directory and run it on all of our repository’s history.


mkdir -p newroot/

# Pipe output to silence "file not found" warnings.
mv README.md newroot/ 2>/dev/null
mv src newroot/ 2>/dev/null


Run this script on the project history:

$ git filter-branch -f --prune-empty --tree-filter /path/to/script HEAD

After that, we should have a new repository with a directory called newroot that contains all of the files we wish to preserve. If we spotted an issue, we can just reset our branch to the initial state (git reset --hard master) and try again, otherwise, we can move to the next step: filtering the repository to be only this directory.

$ git filter-branch --prune-empty -f --subdirectory-filter newroot

Assuming everything is correct we can go on and push it to our new repository as master.

$ git remote add bar git+ssh://git@git.domain.com/bar.git
$ git push bar tmp:master

That’s it! You have now split bar out of foo. The last remaining thing to do is to delete the remaining bar related files from our foobar repository and commit the changes.

Moving arbitrary files between repositories

Moving arbitrary files is very easy when you consider it is just a split from one repository followed by a merge to another. For this reason I will not elaborate further, just follow the two sections above.

Finishing notes

This is a very simple guide. In some more complex cases you will probably have to write more complex scripts or use some optimization techniques. I suggest you also take a look at the slides for a talk I gave about migrating the Enlightenment project from SVN to git. They contain some useful tips and tricks. Especially if you have a big project with a very rich history.

Please let me know if you encountered any issues or have any suggestions.

This article was originally posted on Tom’s personal blog, and has been approved to be posted here.

Running letsencrypt as an Unprivileged User

Running letsencrypt as an unprivileged user (non-root) is surprisingly easy, and even more surprisingly undocumented. There is no mention in the official documentation, nor was I able to find anything online. There are alternative clients that were designed to be run as unprivileged, but they are not as beginner-friendly as the official one. Personally, I’ve switched to acme-tiny (and created an AUR package for it). Its much smaller and lets me have an even more secure setup.

Why would you want to bother with this? One word: security. You should always strive to run every process with the lowest privileges possible because this reduces the chances of data loss as a result of a bug. More importantly, this reduces the chances of your server being compromised and thus improves overall security.


In this tutorial we will setup letsencrypt to run as an unprivileged user using the webroot plugin. This tutorial uses basic letsencrypt commands for simplicity. Refer to the official documentation for more advance usage.

Definitions and assumptions:

  • The domain: example.com
  • The web server’s web root: /srv/http/example.com
  • Commands preceded by # should be run as root.
  • Commands preceded by $ should be run as the letsencrypt user.

Prepare the Environment

First we need to create an unprivileged user for letsencrypt; I chose letsencrypt. The following command will create a new system user without a logging shell or a home directory.

# useradd --shell /bin/false --system letsencrypt

Now we will create the needed directory for the webroot plugin, and set the correct permissions.

# mkdir -p /srv/http/example.com/.well-known/acme-challenge
# chown -R letsencrypt: /srv/http/example.com/.well-known

Optional: verify the web server can serve files created by our user:

$ echo "works!" > /srv/http/example.com/.well-known/acme-challenge/test.txt
$ curl example.com/.well-known/acme-challenge/test.txt

If the last command printed “works!”, everything works. Otherwise, something is wrong with your configuration. This is unfortunately out of scope for this tutorial, but feel free to contact me, I might be able tohelp.

Setup letsencrypt

There are two options for this step. The first option is easier, and is best if you already have a working letsencrypt setup. The second option is more correct and is preferred if you are starting fresh.

Option 1: Update the Permissions of the Default Paths

By default letsencrypt (at least on Arch Linux) uses these three paths:

  • logs-dir: /var/log/letsencrypt
  • config-dir: /etc/letsencrypt
  • work-dir: /var/lib/letsencrypt

We need to change these directories to be owned by our user and group:

# chown -R letsencrypt: /var/log/letsencrypt /etc/letsencrypt /var/lib/letsencrypt

Now we need to run letsencrypt so it creates the initial configuration and our first certificate.

$ letsencrypt certonly --webroot -w /srv/http/example.com -d example.com -d www.example.com

At this stage letsencrypt will complain about not running as root, that is fine. Ignore it. Just follow the steps and answer the questions.

Option 2: Create New Directories for letsencrypt

Letsencrypt supports a few undocumented flags that let you change the running environment.

First we need to create the relevant directory structure, for simplicity I chose /home/letsencrypt as the base directory and the rest as subdirectories:

# mkdir /home/letsencrypt
# chown letsencrypt: /home/letsencrypt

And as the user:

$ cd /home/letsencrypt
$ mkdir logs config work

Now we can run letsencrypt as we normally do, just with the addition of the --logs-dir, --config-dir and the --work-dir flags.

$ letsencrypt certonly --config-dir /home/letsencrypt/config --work-dir /home/letsencrypt/work \
 --logs-dir /home/letsencrypt/logs/ --webroot -w /srv/http/example.com -d example.com -d www.example.com

At this stage letsencrypt will complain about not running as root, that is fine. Ignore it. Just follow the steps and answer the questions.

Verify Functionality

If you got here, you should already have your certificate issued.

You can verify this by running:

Verify Option 1:

$ ls /etc/letsencrypt/live/example.com

Verify Option 2:

$ ls /home/letsencrypt/config/live/example.com

This should output cert.pem chain.pem fullchain.pem privkey.pem

Certificate Renewal

Certificates need to be renewed before they expire or users will receive ominous warnings when visiting your site. You should run letsencrypt in a cron job so the certificate is renewed before it expires (at the time of writing, letsencrypt certificates are valid for 3 months). I have a cron job running once a month.

When renewing, you should run:

Renew Option 1:

$ letsencrypt certonly --agree-tos --renew-by-default --webroot -w /srv/http/example.com -d example.com \
 -d www.example.com

Renew Option 2:

$ letsencrypt certonly --agree-tos --renew-by-default --config-dir /home/letsencrypt/config \
 --work-dir /home/letsencrypt/work --logs-dir /home/letsencrypt/logs/ --webroot -w /srv/http/example.com \
 -d example.com -d www.example.com

Important: do not forget to make the server reload the certificates after they are renewed. Nginx for example, does not do this automatically.

Some Final Notes

For more information about letsencrypt, please refer to the official documentation.

This tutorial does not cover setting up your web server to use the new certificates. This is very simple and covered at length elsewhere.

However, here is an example for nginx:

server {
     server_name  example.com;

     ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
     ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;


Letsencrypt is an incredibly important tool in providing better security on the web, so if you have site that doesn’t currently offer HTTPS encryption, I highly encourage you to follow this guide. Please let me know if you encountered any issues or have any suggestions, or feel free to leave a comment on this article.

As originally posted on my blog.

An Introduction to Installing Your First Let’s Encrypt HTTPS Certificate

The usage of https has been so far somewhat restricted on open source projects, because of the cost of acquiring and maintaining certificates. As a result of this and the need to improve Internet security, several projects are working on providing free valid certificates. Among those projects, Let’s Encrypt launched a public beta last week on December, 3 2015.

The Let’s Encrypt Approach

Let’s Encrypt is a Linux Foundation Collaborative project that started to fulfill an Electronic Frontier Foundation – EFF long-term mission to Encrypt the Web. According with EFF, the “aim is to switch hypertext from insecure HTTP to secure HTTPS. That protection is essential in order to defend Internet users against surveillance of the content of their communications; cookie theft, account hijacking and other web security flaws; cookie and ad injection; and some forms of Internet censorship.”.

With that goal in mind, the Let’s Encrypt project is providing free certificates, valid for 90 days. The certificate renewals are also free, and the enrollment procedure is meant to be simple and scriptable. They have proposed an RFC to the Internet Engineering Task Force – IETF for an automatic protocol to manage https certificates, called Automatic Certificate Management Environment (ACME) protocol.

There are several clients that support the ACME protocol, we chose to use letsencrypt. As we’ve just upgraded the LinuxTV server last week, I decided to pioneer the install of the Let’s Encrypt certificates.

How to Use Letsencrypt to Get an https Certificate

The process is actually really simple.

The first step is to clone the letsencrypt script from https://github.com/letsencrypt/letsencrypt with:

$ git clone https://github.com/letsencrypt/letsencrypt

The first time it runs, it will install python dependencies. The script is smart enough to identify the distribution and do the right thing in most cases. I tested it on both Fedora 23 and Debian with similar results, but some distributions like SUSE might require more work:

# cd letsencrypt
# ./letsencrypt-auto
Bootstrapping dependencies for Debian-based OSes...

And, after installing the packages:

And check for the missing dependencies, installing them.
Creating virtual environment...
Updating letsencrypt and virtual environment dependencies.......
Running with virtualenv: sudo /home/mchehab/.local/share/letsencrypt/bin/letsencrypt

It will then proceed to the next step of asking for the e-mail of the admin:

An Introduction to Installing Your First Lets Encrypt HTTPS Certificate - email
It then asks you to agree to the license terms, everything seemed fine to me, so I accepted it:
An Introduction to Installing Your First Lets Encrypt HTTPS Certificate - encrypt-sla

If Let’s Encrypt successfully detects the domains on your server, it will present you with a set of checkboxes to select the domains you want to serve over https.

An Introduction to Installing Your First Lets Encrypt HTTPS Certificate - https_select

If the script can’t detect the domains on the server, it will ask you to type them in, separated by a space:

An Introduction to Installing Your First Lets Encrypt HTTPS Certificate - domains

NOTE: It should be noted that the script needs either root access or sudo access in order to install the needed dependencies and set up the apache server. It also needs to run on the server where the certificates will be installed. Trying to run it on a different machine would cause an error:
Failed authorization procedure.
www.linuxtv.org (http-01): urn:acme:error:unauthorized ::
The client lacks sufficient authorization ::
Invalid response from http://www.linuxtv.org/.well-known/acme-challenge/03ocs4YOeW32134wH3Oo911sv-aJ_SK0B1R_YVCGk []: 404, git.linuxtv.org (http-01):
urn:acme:error:unauthorized ::
The client lacks sufficient authorization ::
Invalid response from http://git.linuxtv.org/.well-known/acme-challenge/oPcUtwer423oc2dVqElgVc0HxTjJfuVv1cwk1A-F0 []: 404, linuxtv.org (http-01):
urn:acme:error:unauthorized :: The client lacks sufficient authorization ::
Invalid response from http://linuxtv.org/.well-known/acme-challenge/ZuPCq4geW36d6GxcIK_GhIfaH35l1mCNOS9X67HU4 []:
404, patchwork.linuxtv.org (tls-sni-01): urn:acme:error:unauthorized ::
The client lacks sufficient authorization :: Correct zName not found for TLS SNI challenge. Found

It then asked me if I wanted to allow both http and https or just https. I chose to allow both, but if your site communicates sensitive information like passwords or personal data, you might consider forcing all connections to use https:

An Introduction to Installing Your First Lets Encrypt HTTPS Certificate - https_type

After that, it created the certificate and updated the /etc/apache2 configurations for all the sites that were enabled:
An Introduction to Installing Your First Lets Encrypt HTTPS Certificate - https_congrats

Starting Using the New Certificates

That’s the most exciting part of the letsencrypt tool: the script adjusted all the configurations on my apache2 server and auto-reloaded it, so there’s no need to do anything to start using it! Ubuntu, Debian, Centos 7, and Fedora are currently the only Linux distros that support automatic configuration, other distributions will likely require manual configurations.

After running the script my apache server was running with the new certs with no downtime! Now visitors to Linux TV can now use https to access the site securely. We are currently working on implementing Let’s Encrypt on our blog and other internal resources here at the OSG. Here’s to a safer and more secure web!

Even More Open Source Medicine and Machine Learning, in This Week’s Wrap Up

Open Source Wrap Up: November 14-20, 2015

Group of Biohackers Start Work on Open Source Insulin

More than 370 million people suffer from diabetes worldwide, and these people rely on regular insulin injections to regulate the amount of sugar in their blood. Despite this major need for insulin, there is no generic version available on the market, and the high cost results in it having limited availability in poorer parts of the world. A group of citizen and academic researchers and biohackers, led by Counter Culture Labs, have launched a project to develop a simple method for producing insulin and release the process to the public domain. They have launched a crowdfunding campaign (that has already exceeded their goal) to fund the first stage of this research.  For stage 1, ” the team will insert an optimized DNA sequence for insulin into E. coli bacteria, induce the bacteria to express insulin precursors, and verify that human proinsulin has been produced.”

Netflix Launches Open Source Platform for Continuous Delivery

Netflix has launched Spinnaker, a new open source project for continuous delivery, in conjunction with Google, Microsoft, and Pivotal. It is designed to serve as a replacement to Asgard, the model Netflix previously used to manage Amazon Web Services. Spinnaker expands on Asgard by providing support for multiple platforms, including Google Cloud Platform, and Cloud Foundry, with support for Microsoft Azure reportedly in development. Spinnaker also provides cluster management and visibility into the footprint of applications on cloud infrastructure.

Check out the code on GitHub.

Linux Goes to the International Space Station

The United Space Alliance, a NASA contractor, has decided to migrate to Linux for operations on the International Space Station (ISS). The company’s Laptops and Network Integration team is in charge of the station’s OpsLAN: a network of laptops that provide the crew with vital capabilities for day-to-day operations. The team decided to migrate from Windows to Linux in order to improve stability and reliability, and provide better control over the operating system they deploy. The Linux Foundation provided the training needed for this team to develop a solution based on Linux through classes that introduced the team members to Linux and application development on the platform.

Read More from the Linux Foundation.

Microsoft Releases Distributed Machine Learning Toolkit

Following in the wake of Google’s release of TensorFlow, Microsoft has answered with their own open source, distributed machine learning toolkit. DMTK includes a flexible framework that supports a unified interface for data parallelization, big model storage, model scheduling, and automatic pipelining. It also includes distributed algorithms for word embedding and multi-sense word embedding as well as LightLDA: a fast and lightweight system for large-scale topic modeling. DMTK offers an API designed to reduce the barrier to entry of distributed machine learning with the goal being for researchers and developers to focus more on the data, model, and training the core machine logics. By open sourcing this software, Microsoft hopes to encourage academia and the industry to get involved in the platform’s development.

The code can be found on GitHub.

Other News

A New AI, Medical Robots, and More in This Week’s Wrap Up

Open Source Wrap Up: November 7 -13, 2015

Google Releases Open Source AI Engine: TensorFlow

Google has released TensorFlow, deep learning that is used in many of the company’s products, as open source. The software uses a library for numerical computation inside data flow graphs that pass dynamically sized, multi-dimensional arrays, called tensors, between nodes. The nodes perform processing asynchronously, in parallel once all tensors on their incoming edges become available. This design makes TensorFlow flexible and portable as it is relatively simple to reconstruct data flow graphs and use high-level programming languages to develop new computational systems. Google has used this to connect artificial intelligence research with product teams that build new products that use this software. By releasing it as open source, the company hopes to bring more people into this convergence effort and standardize the set of tools people rely on for this work.

To learn more, visit the TensorFlow website.

Linux Foundation Announces Launch of OpenHPC

The Linux Foundation has announced the intent to form the OpenHPC Collaborative Project which is aimed at promoting and developing open source frameworks for high performance computing environments. The organization includes educational institutions, governmental organizations, and major corporations. The members of this group will work together to create a stable environment for testing and validation, reduce costs, provide a robust and diverse open source software stack, and develop a flexible framework for configuration of high performance computing. They will do this through the development of tools like provisioning tools, resource management, I/O clients, development tools, and a variety of scientific libraries.

For more information, visit the OpenHPC community page.

Vanderbilt’s Medical Capsule Robot Software Goes Open Source

Researchers at the Vanderbilt University School of Engineering have released a set of hardware and software designs for medical robots that are designed to be swallowed (once the technology is small enough). Named Pillforge, the team had built three different robots for the colon, stomach, and to stop internal bleeding, and they realized they were reusing a lot of components. This led to the desire to release these building blocks to other researchers and product developers in order to reduce the barrier to entry for others and allow other researchers and developers to improve the designs. The robots run TinyOS for their operating system, and support a variety of modules including some for wireless communication, sensing, actuating,  and more.

To learn more, visit the Pillforge website.

Let’s Encrypt Public Beta Announced

Let’s Encrypt, a Linux Foundation Collaborative project to encrypt the web, will go into public beta on December 3, 2015. Let’s Encrypt is a project aimed an encrypting a greater number of places on the web by generating TLS/SSL certificates that can be installed on web servers with a simple application. On December 3, anyone who wants to request a certificate will be able to. The limited beta launched on September 12, 2015, and has issued over 11,000 certificates since; this experience has given the project enough confidence to open the system for public beta.

Other News

Open Source Eats Governance, and More. This Week in Open Source

Open Source Wrap Up: November 1 – 6, 2015

Linux Foundation Launches Open API Initiative

The Linux Foundation continues to expand their reach in the tech industry by launching the Open API Initiative (OAI) aimed at creating a vendor neutral, portable, open specification for providing metadata for RESTful APIs. This specification will be built on top of Swagger: a description system that makes APIs autodiscoverable and self-documenting. Open APIs have become a key component when creating new technologies, and are particularly important for sectors like the banking and health care industries. Any effort to improve standardization will certainly be valuable.

More information can be found at the project home page.

Open Source Mobile Voter Registration System Developed for Libya

If you have followed international news at all in recent years, you are certainly aware of the ongoing instability throughout much of the Middle East. Libya has been hit particularly hard by this instability with multiple factions currently fighting for power. As part of an effort to bring the factions together, the country is currently working towards holding a democratic election and the Libyan High National Elections Commission has hired Caktus, a US-based IT consultancy group, to develop an open source platform for voter registration. So far, more than 1.5 million Libyans have registered to vote using the simple SMS system. While the nation still has plenty of challenges to overcome, open source is helping the Elections Commission take steps in the right direction towards establishing sustainable governance.

Indian Government Makes Major Push Towards Open Source

The Indian government has expressed interest in open source software in the past, most notable through the development efforts on BOSS Linux. The Ministry of Communication and Information Technology has published a new policy on the use of open source for government organizations. This mandatory policy states that all government organizations should consider open source solutions to be the preferred option over all closed source, cloud applications for e-governance systems.

The new policy has three major goals:

  • To provide a policy framework for rapid and effective adoption of OSS.
  • To ensure strategic control in e-governance applications and systems from a long-term perspective.
  • To reduce the total cost of ownership of projects.

The policy can be read in its entirety here.

Docker Releases Production-Ready Version 1.9

Progress on Docker has continued ahead full steam with version 1.9 released this week. It includes a huge amount of changes and new features.

Most notably, this release includes:

  • Multi-host Networking – Allows the creation of virtual networks in Docker Engine that span multiple hosts to provide better control over network topology and container-to-container communication.
  • Persistent Storage – New and improved plugins make third-party, persistent storage easier to setup.
  • Docker Swarm 1.0 – Swarm makes scaling Docker clusters easier and it has reached a production-ready state with this release.
  • Many improvements to Docker Engine, Compose, and Registry.

Other Links