Connecting sensors to Mozilla’s IoT Gateway

Here is a 1st post about Mozilla’s IoT effort, and specifically the gateway project which is illustrating “Web Of Things” concept to create a decentralized Internet of Things, using Web technologies.

Today we will focus on the gateway, as it is the core component of the whole framework. Version 0.4.0 was just released, so you can try it your own on Raspberry Pi 3.  The Raspberry Pi 3 is the reference platform, but it should be possible to port to other single board computers (like ARTIK, etc).

The post will explain how to get started and how to establish basic automation using I2C sensors and actuators on gateway’s device (without any cloud connectivity).

To get started, first install the gateway according to these straightforward instructions:

Prepare SD Card

You need to download the Raspbian based gateway-0.4.0.img.zip (1GB archive) and dump it to SD card (2.6GB min).

lsblk # Identify your sdcard adapter ie:
disk=/dev/disk/by-id/usb-Generic-TODO
url=https://github.com/mozilla-iot/gateway/releases/download/0.4.0/gateway-0.4.0.img.zip
wget -O- $url | funzip | sudo dd of=$disk bs=8M oflag=dsync

If you only want to use the gateway and not hack on it, you can skip this next part which enables a developer shell though SSH.  However, if you do want access to a developer shell, mount the 1st partition called “boot” (you may need to replug your SD card adapter) and add a file to enable SSH:

sudo touch /media/$USER/boot/ssh
sudo umount /media/$USER/*

First boot

Next, install the SD card in your Raspberry PI 3 (Older RPis could work too, particularly if you have a wifi adapter).

When it has completed the first boot, you can check that the Avahi daemon is registering “gateway.local” using mDNS (multicast DNS)

ping gateway.local
ssh pi@gateway.local # Raspbian default password for pi user is "raspberry"

Let’s also track local changes to /etc by installing etckeeper, and change the default password.

sudo apt-get install etckeeper
sudo passwd pi

Logging in

You should now be able to access the web server, which is running on port 8080 (earlier version used 80):

http://gateway.local:8080/

It will redirect you to a page to configure wifi:

URL: http://gateway.local:8080/
Welcome
Connect to a WiFi network?
FreeWifi_secure
FreeWifi
OpenBar
...
(skip)

We can skip it for now:

URL: http://gateway.local:8080/connecting
WiFi setup skipped
The gateway is now being started. Navigate to gateway.local in your web browser while connected to same network as the gateway to continue setup.
Skip

After a short delay, the user should be able to reconnect to the entry page:

http://gateway.local:8080/

The gateway can be registered on mozilla.org for remote management, but we can skip this for now.

Then administrator is now welcome to register new users:

URL: http://gateway.local:8080/signup/
Mozilla IoT
Welcome
Create your first user account:
user: user
email: user@localhost
password: password
password: password
Next

And we’re ready to use it:

URL: http://gateway.local:8080/things
Mozilla IoT
No devices yet. Click + to scan for available devices.
Things
Rules
Floorplan
Settings
Log out

Filling dashboard

You can start filling your dashboard with Virtual Resources,

First hit the “burger menu” icon, go to settings page, and then go to the addons page.

Here you can enable a “Virtual Things” adapter:

URL: http://gateway.local:8080/settings/addons/
virtual-things-adapter 0.1.4
Mozilla IoT Virtual Things Adapter
by Mozilla IoT

Once enabled It should be listed along ThingURLAdapter on the adapters page:

URL: http://gateway.local:8080/settings/adapters
VirtualThingsAdapter
virtual-things
ThingURLAdapter
thing-url-adapter

You can then go back to the 1st Things page (it’s the first entry in the menu):

We can start adding “things” by pressing the bottom menu.

URL: http://gateway.local:8080/things
Virtual On/Off Color Light
Color Light
Save

Then press “Done” at bottom.

From this point, you can decide to control a virtual lamp from the UI, and even establish some basic rules (second entry in menu) with more virtual resources.

Sensing Reality

Because IoT is not about virtual worlds, let’s see how to deal with the physical world using sensors and actuators.

For sensors, there are many way to connect them to computers using analog or digital inputs on different buses.  To make it easier for applications developers, this can be abstracted using W3C’s generic sensors API.

While working on IoT.js‘s modules, I made a “generic-sensors-lite” module that abstracted a couple of I2C drivers from the NPM repository.  To verify the concept, I have made an adapter for Mozilla’s IoT Gateway (which is running Node.js), so I published the generic-sensors-lite NPM module first.

Before using the mozilla-iot-generic-sensors-adapter, you need to enable the I2C bus on the gateway (version 0.4.0, master has I2C enabled by default).

sudo raspi-config
Raspberry Pi Software Configuration Tool (raspi-config)
5 Interfacing Options Configure connections to peripherals
P5 I2C Enable/Disable automatic loading of I2C kernel module
Would you like the ARM I2C interface to be enabled?
Yes
The ARM I2C interface is enabled
ls -l /dev/i2c-1
lsmod | grep i2c
i2c_dev 16384 0
i2c_bcm2835 16384 0

Of course you’ll need at least one real sensor attached to the I2C pin of the board.  Today only 2 modules are supported:

You can double check if addresses are present on I2C the bus:

sudo apt-get install i2c-tools
/usr/sbin/i2cdetect -y 1
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- 23 -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- 77

Install mozilla-iot-generic-sensors-adapter

Until sensors adapter is officially supported by the mozilla iot gateway, you’ll need to install it on the device (and rebuild dependencies on the target) using:

url=https://github.com/rzr/mozilla-iot-generic-sensors-adapter
dir=~/.mozilla-iot/addons/generic-sensors-adapter
git clone --depth 1 -b 0.0.1 $url $dir
cd $dir
npm install

Restart gateway (or reboot)
sudo systemctl restart mozilla-iot-gateway.service
tail -F /home/pi/.mozilla-iot/log/run-app.log

Then the sensors addon can be enabled by pressing the “enable” button on the addons page:

URL: http://gateway.local:8080/settings/addons
generic-sensors-adapter 0.0.1
Generic Sensors for Mozilla IoT Gateway

It will appear on the adapters page too:

URL: https://gateway.local/settings/adapters
VirtualThingsAdapter
virtual-things
ThingURLAdapter
thing-url-adapter
GenericSensorsAdapter
generic-sensors-adapter

Now we can add those sensors as new things (Save and done buttons):

URL: http://gateway.local:8080/things
Ambient Light Sensor
Unknown device type
Save
Temperature Sensor
Unknown device type
Save

Then they will appear as:

  • http://gateway.local:8080/things/0 (for Ambient Light Sensor)
  • http://gateway.local:8080/things/1 (for Temperature Sensor)

To get value updated in the UI, they need to turned on first (try again if you find a big, and file tickets I will forward to drivers authors).

A GPIO adapter can be also used for actuators, as shown in this demo video.

If you have other sensors, check if the community has shared a JS driver, and please let me know about integrating new sensors drivers in generic-sensors-lite

IoT.js landed in Raspbian

Following previous efforts to deploy iotjs on Raspberry Pi 0, I am happy to announce that IoT.js 1.0 landed in Debian, and was sync’d to Raspbian for ArmHF and Ubuntu as well.

While the package is targeting the next distro release, it can be easily installed on current versions by adding a couple of config files for “APT pinning”.

If you haven’t set up Raspbian 9, just dump the current Raspbian image to SDcard (for the record I used version 2018-03-13-raspbian-stretch-lite)

Boot your Pi.  To keep track of changes in /etc/, let’s install etckeeper:

sudo apt-get update
sudo apt-get install etckeeper

Upgrade current packages:

sudo apt-get upgrade
sudo apt-get dist-upgrade

Declare the current release as default source:

cat<<EOT | sudo tee /etc/apt/apt.conf.d/50raspi
APT::Default-Release "stretch";
EOT

Then add a repo file for the next release:

cat /etc/apt/sources.list | sed 's/stretch/buster/g' | sudo tee /etc/apt/sources.list.d/raspi-buster.list

Unless you want to test the upcoming release, it maybe be safer to avoid upgrading all packages yet.  In other words, we prefer that only iotjs should be available from this “not yet supported” repo.

cat<<EOT | sudo tee /etc/apt/preferences.d/raspi-buster.pref
Package: *
Pin: release n=buster
Pin-Priority: -10
EOT

cat<<EOT | sudo tee /etc/apt/preferences.d/iotjs.pref
Package: iotjs
Pin: release n=buster
Pin-Priority: 1
EOT

Now iotjs 1.0-1 should appear as available for installation:

sudo apt-get update ; apt-cache search iotjs
iotjs - Javascript Framework for Internet of Things

apt-cache policy iotjs
iotjs:
  Installed: (none)
  Candidate: 1.0-1
  Version table:
     1.0-1 1
        -10 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages

Let’s install it:

sudo apt-get install iotjs
man iotjs

Even if version 1.0 is limited in compared to the development branch, you can start by using the http module which is enabled by default (not https).

To illustrate this, when I investigated “air quality monitoring” for a TizenRT+IoT.js demo I found out that OpenWeatherMap is collecting and publishing “Carbon Monoxide” Data, so let’s try their REST API.

Create a file, example.js for example, that contains:

var http = require('http');

var location = '48,-1';
var datetime = 'current';

//TODO: replace with your openweathermap.org personal key
var api_key = 'fb3924bbb699b17137ab177df77c220c';

var options = {
  hostname: 'api.openweathermap.org',
  port: 80,
  path: '/pollution/v1/co/' + location + '/' + datetime + '.json?appid=' + api_key,
};

// workaround bug
options.headers = {
  host: options.hostname
}

http.request(options, function (res) {
  receive(res, function (data) {
    console.log(data);
  });
}).end();

function receive(incoming, callback) {
  var data = '';

  incoming.on('data', function (chunk) {
    data += chunk;
  });

  incoming.on('end', function () {
    callback ? callback(data) : '';
  });
}

And just run it:

iotjs example.js
{"time":"2018-03-27T02:24:33Z","location":{"latitude":47.3509,"longitude":-0.9081},"data":[{"precision":-4.999999987376214e-07,"pressure":1000,"value":1.5543508880000445e-07}
(...)

You can then use this to do things such as update a map or raise an alert on anything useful, or try to rebuild master branch.

How to Build A Simple Connected Light with IoT.JS and Python Django

Many web developers I meet are interested in working with embedded systems and IoT, but they always seem to have reservations on just how to make the whole system (i.e. a server, a ‘thing,’ and a client) work! The amount of information online is extensive, but it’s often hard to know where to start! This blog post will provide a very simple example of how to get a basic LED light to work in a local network, with a web client that provides a way to identify the light with no prior knowledge from the user and no required installation on the client device.

To do this we are going to use a very popular Python Django web framework and the  Samsungs IoT.JS framework. This post will provide an overview, basic code snippets, and links to more information on the GitHub repo. I’ll also provide exact links to the hardware I used for anyone who wants to tinker.

The Problem

There’s an IoT device sitting on the local network, silently exposing functionality to the public. To save money on the bill of materials, there are no screens or buttons, and configuration is all done via a web UI. How does a developer get access to the system and use it? There are several ways to handle this discovery with physical interactions from the user, such as bluetooth, RFID, QR Codes, or even a URL printed on the device. There are also discovery protocols like Bonjour, although even that will not work on many WiFi LAN networks because UDP is commonly blocked at the WiFi hub (as is with our Samsung WiFi).

In this example our device will be registered on the local WiFi network and have a local IP address. What we want is for a developer to be able to find this IP address and get access to the UI of the device. OK, lets start…

The Hardware

For this guide, I’m using the following items:

 

In this example, the Raspberry Pi Zero was placed in the 240v AC power in ‘cavity’ as seen in the picture below. Since our new light will not use 240V AC, it provides a convenient place for the Raspberry Pi to sit.

 

The LED light strip replaces the LED matrix of the original light. In this example, I’m only using the plastic water resistant housing, all the electronics and control wires have been removed. The fixture was mounted on a piece of plywood to demonstrate the device.

For instructions on how to physically connect your LED light to the raspberry pi go here. The code to control the light is on my personal github repo, the file which controls the server is done in the file server_html.js. You can see line  63 calling the objects method: lightcontrol.showRainbowLight(). This method is exported in the lightconrol.js file which controls the light. For now I will leave details for a future blog, this is all about how we access and control the light and server. How we control the hardware pins of the LED and make that work is for blog 2 in the series.

The Server

The basic functions of the server are to register and update the IP of our IoT Light and to route the user to the correct local IP address.

Register And Update The Light

The server holds the details of the light, and again, there are a number of protocols and standards to pick from! To keep things simple, we created a REST endpoint that allows a light to register; it does this with PUT, POST and DELETE. The key to the server is to use the MAC address as a unique ID and hold the local IP address. In this example, the light will only work locally for the developer, and it’s a valid use case for certain applications to only be accessible to someone within the local network. The basic sequence diagrams looks like this:

To ease the writing of REST endpoints, I used the very popular Django Rest Framework. Hence, the light needs to register itself and update its local IP address. It uses the MAC address as a unique value, however you are free to invent your own solution here. I created a very rich object model in my server in an attempt to future proof the DB object as best as possible, but the controller for registering the light is surprisingly simple and compact, and most of the lines are documentation.

def iot_machines_register(request):                     # TODO Add some type of authentication for iot devices
    """
    :param request: JSON {
                    device_id:      mac_adress e.g. 98:83:89:3a:96:a5
                    device_name:    "any text string"
                    local_ip:       IPV4 or IPV6 e.g. 192.168.0.1
                    }

    :return: HttpResponseForbidden, HttpResponse, HttpResponseRedirect

    Register an IoT Machine based on it's MAC address.
    If this is a POST we check it's unique. Verify the JSON package. And register the machine

    If this is a PUT we check it exists. verify the JSON package. And update the machine.

    TODO - Authentication & Authorization!

    """

    if request.method == 'POST':
        data = JSONParser().parse(request)
        print("We got the following data in the request: {}".format(data))
        serializer = IoTMachineSerializer(data=data)
        if serializer.is_valid():
            serializer.save()
            return JsonResponse(serializer.data, status=201)
        return JsonResponse(serializer.errors, status=400)

The function checks the method with request.method == ‘POST’ and creates a serializer with serializer=IoTMachineSerializer(data=data) to parse the date from the POST message. Validation is done in the model, which makes things far simpler to code. If validation succeeds, the ORM saves the new object in the SQL database with serializer.save() and returns a HTTP 201 with return JsonResponse(serializer.data, status=201). If it fails, the model generates the correct error codes and the framework responds with the appropriate error message with return JsonResponse( serializer.errors, status=400). Details of this are on the GitHub repo.

Routing The User To The Local IP

The clever part of this system must allow the user to find the IP of the light. After doing this, all communications will be handled locally between the device on the local WiFi network and the IoT Light.

The sequence diagram above shows the flow of a user hitting the server that contains the local IP and sending a redirect. After this happens, the server is no longer part of the data flow for this user. All the clever control and data flow is between the local user and the IoT Device.

def iot_machine_find(request, mac_address):
    """
    Find an IoT Machine based on it's MAC address.
    If it has a public local IP address then redirect to this IP.
    If our embedded system does not have a valid IP address then don't redirect and show the user a
    friendly page saying the machine is not active at the moment.

    """
    print("Trying to find an IoT Machine and redirect locally with mac_address of: {}".format(mac_address))

    # Make sure we can find the machine via it's mac address
    try:
        iot_device = IoTMachine.objects.get(device_id=mac_address)
    except IoTMachine.DoesNotExist:
        return HttpResponse(status=404)

    # Make sure the IP address we pass back is still valid
    try:
        ipaddress.IPv4Address(iot_device.local_ip)
    except ValueError:
        # TODO Paint a nice screen and show the user the device exists but routing is broken
        print("There was an issue with the stored IP address for the device: {} - handle it an carry on".format(iot_device.mac_address))
        return Http404

    # Happy path - direct the user to the device
    if request.method == 'GET':

        scheme = 'http'                 # TODO make this a dynamic variable from settings or DB table
        path = 'machine'                # TODO make this a dynamic variable from settings or DB table
        remote_url = "{0}://{1}/{2}".format(scheme,iot_device.local_ip,path)
        print("We found the device returning the IOT Local address: {}".format(remote_url))
        return HttpResponseRedirect(remote_url)

    # If we get this far - we don't support other methods at the minute so reply with a forbidden.
    return HttpResponseForbidden

The first and second try/except clauses simply check that the MAC address exists, and if it does, that there is a valid IP address to send back. The happy path checks for the GET and PUT method, and a GET returns the local IP address in a redirect with return HttpResponseRedirect(remote_url). The URL is hard coded in this example, however production systems would dynamically take these values from either configuration files or from data the IoT device provides.

All other HTTP methods will be caught with the return HttpResponseForbidden method. Again, much of the heavy lifting is done with the Django ORM and REST framework.

It’s important to note how flat the structure is; Pythonic code tends to move away from multiple layers of abstraction, opting for structures that are as flat as possible. This only shows the logic the server is exercising, and the framework comes, as they say, with ‘batteries included’. To get to the Django administration interface as a registered admin user, you just login and hit the admin path. It will then be possible to view the active DB, registered IoT machines, and even manipulate any values of those machines. No additional code is required for this.

By selecting a single IoT device, the framework will pull all relevant data from the SQL DB and format and display it in the Django admin form. All of this is generated automatically:

I haven’t gone into detail about how the Django ORM works in this article. If you want to learn more, it’s best to visit the Python Django experts.

How Did The Client Find The Server?

At this point you might be thinking: wait a minute, not only do I still need to know about the public server, but I also need to know what the light ID is! Remember the light as no physical buttons or screen! In this example we used a QR code that sits next to our light. The user scans the code with the phone which routes them to the server using it’s own MAC address as the ID. You can try this yourself with your Samsung browser on your phone – scan this QR code below.

You will be routed to my test server at www.noisyatom.tech/iot/machine/b8:27:3b:01:d8:3f, which will forward you to the UI the light exposes. (Un)fortunately, it only works if you are here in the same WiFi network. :-) Your browser will try to connect you to http://192.168.110.99/light.

Incidentally you can try this from your Samsung browser by selecting the ‘Scan QR code’ from the top right selection menu, which should show you a screen like below:

Further Exploration

There is a lot of information here, and I’ve not really explained how the model is created or how to tell the server what parameters of the model are important! I will follow up on this in future blog posts where I’ll look at the details of the code running on the light. The light acts like both a server and client: it’s a server to the device that wants access to functionality, and a client of the central server that holds details about the device. Once the light is activated, it goes through a very nice looking rainbow dance; you could imagine this being used as a mood light or as a device that interacts with other systems. Finally, I will revisit the server side code later and explain how we make the model persist to a DB and have only the server check and verify components of that model with the REST interface in a future blog.

Check out the light in action!

The Business Obligations of Open Source Software

This article is a part of the Essential Guide to Open Source for Business.

The previous two articles in this series covered the technical and business benefits OSS offers. However, this only paints half the picture. While OSS might be free to use, modify, and distribute, it doesn’t come without effort and risk; if a company isn’t prepared to handle them, they can cause significant headaches down the road. This article will provide an overview of the obligations and risks a company that uses OSS in their products or services must be aware of.

Licensing and Compliance

While OSS has no initial licensing costs, this doesn’t always mean you’re free to use OSS however you want. Open source licenses can impose a range of obligations that must be satisfied whenever code that includes OSS is distributed in a product or service. This can include things like disclosure requirements, specific notifications, and licensing modifications under the same open source license.

At the very least, all businesses need to maintain an accurate inventory of all OSS that’s used in their products or services, and must have adequate processes to ensure compliance with all licenses both during and after product distribution. Typically, a single person is assigned the responsibility of being the open source compliance manager to oversee the execution of this, and it requires coordination between legal, engineering, human resources, and the compliance officer; larger companies often need to establish an Open Source Review Board to facilitate this coordination.

The most critical coordination is between product, engineering, and management teams: they must identify all OSS used in product code and to assist with any licensing or compliance questions. This often means employee roles need to be expanded to cover the various elements of OSS compliance that directly relate to each position. This coordination must also expand outside the company to include software providers as well because any OSS that’s included in the software they provide must also be properly accounted for and complied with. While establishing a framework for this coordination, it’s common to come across areas that need better tooling which creates more need for deployment and training.

Open source licensing can create substantial obligations, especially in larger organizations, and these obligations require more coordination between various teams. This article only briefly touches on the subject of open source compliance, and if you’re interested in this subject more there is a lot of detailed information on the web.

Training and Development

Training is a critical component of open source engineering because you’ll need to train developers, legal, software procurement, quality assurance, systems administrators, and others on the obligations of open source compliance. Additionally, you might need training for developers that have little to no experience working with an open source community on how open source development works.

Maintenance and Support

Proprietary software typically includes some level of paid support from the vendor company, which may include an SLA, pre-disclosure of vulnerabilities, or implementation support.  This is generally contractually enforceable, and it’s reasonable to demand responsiveness from the vendor in line with your support contract.  Similar support for OSS projects may be offered by companies participating in the ecosystem, but it is rare to find unpaid guaranteed support from the development community itself.  As most of the participants are either working on their own time or on behalf of an employer, it is neither reasonable nor fair to expect the same level of service as provided by a paid vendor.  As such, it may be the case where businesses that incorporate unpaid OSS into their products are responsible for installing updates and implementing security fixes and new modules themselves.  These tasks are easy to push aside in favor of more pressing issues, but they must be addressed nonetheless because a lack of maintenance can rapidly overcome the benefits of incorporating externally developed code.

To reiterate, OSS does not include any sort of support contract unless it is specifically negotiated with a third-party company that offers these services.  The most common OSS licenses specifically include a disclaimer of warranty.  If a business requires 24/7 support for services that use OSS, this will either need to be negotiated with an appropriate third-party company or the team will need to be hired in-house. Support services are a critical, but often overlooked, investment when using OSS.

Application Dependence

OSS can be a great avenue to avoid proprietary software dependence, but that doesn’t mean it’s a simple process to migrate away from OSS that no longer meets the requirements of a company. Whenever OSS is used for business-critical applications, the business becomes dependent on the community and project.  Changes that go against the interests of the business could result in the company needing to migrate to an alternative. The effort required to make this switch can vary widely depending on the availability of expertise and the maturity of the projects being migrated between. OSS community support can mitigate this problem, but this also varies substantially from project to project.  In general, it is considered a best practice to be actively engaged with and support select OSS communities to reduce the risk of unexpected changes to critical components.

Strategic Dependence

Open source leadership can bring substantial benefits to companies that properly leverage it, but the independent nature of open source communities can complicate strategic decisions. For starters, open source community choices may preclude downstream requirements, particularly if a company does a poor job of communicating with the upstream community.  This can introduce added effort to maintain downstream products. Additionally, if the community doesn’t have a formal technical roadmap, planning can become much more complicated. Finally, all software whether open or proprietary fits onto a spectrum of maturity, and this can often reflect the needs of a community.  For example, software developed for home use or for academic research may require some level of investment to meet product security requirements.

Do the Benefits of Open Source Outweigh the Obligations?

This article should have provided you with a general understanding of what obligations and risks are associated with the use of OSS in a product or service. There is no one-size-fits-all solution for whether a company should use OSS or simply buy software from a proprietary vendor, so it’s always important for any business that’s considering the use of OSS to consider their own specific requirements.  However, the overwhelming trend in the industry has been in favor of OSS components wherever possible, particularly in consumer electronics.

Here are some questions all businesses should ask themselves when considering the use of OSS in products.

  • How do the platform costs of OSS compare to similar offerings?
  • What commercial support offerings are available?
  • What license obligations does the code have? Does our company have the resources to fill them?
  • Who are the key individuals and organizations in the open source community? How long have they been around?
  • Do the open source community goals and roadmap meet our risk profile?
  • Does our company have adequate resources to support the use of OSS in our products?

 

Let’s Talk Artificial Intelligence

There has been a lot of buzz around Artificial Intelligence as Bixby, Alexa, and Siri become household names. Today, self driving cars share the roads with human drivers, hog farms keep an “artificial eye” on the vitals of individual animals in the pens, and soon enough we’ll be able to monitor heart health by staring into your eyes as a substitute to being hooked up to an EKG machine.

I wanted to know how this is happening and learn about the brain behind all of these advancements, so I recently set out to understand and learn the landscape of various Linux AI projects including NuPIC, Caffe and TensorFlow.  This post will cover some of the important things I learned about each of these platforms.

NuPIC

NuPIC stands for Numenta Platform for Intelligent Computing, and I found the theory behind it to be fascinating. It’s an implementation of Hierarchical Temporal Memory (HTM), a theory of intelligence that’s inspired by and based on the neuroscience of the neocortex, and it’s well suited for finding anomalies in live data streams. The algorithms and framework are implemented in Python,  JavaScript, C++, Shell, and Java.

I picked nupic and nupic-core (implementation of core NuPIC algorithms in C++) to play with on x86_64 (Ubuntu 17.10) and odroid-xu4 (Ubuntu 16.04) platforms. I had good luck compiling nupic and nupic-core from sources on both platforms. I was able to run unit_tests on x86_64, however, on odroid_xu4 I couldn’t run it due to an older version of pip. NuPIC requires pip version 9.0.1, so I’ll have to come back another time to build pip 9.01 from sources.

Here is a short summary of the steps to install NuPIC and run tests.

git clone https://github.com/numenta/nupic.core.git

cd nupic.core/
export NUPIC_CORE=$PWD
mkdir -p $NUPIC_CORE/build/scripts
cd ../nupic.core/build/scripts/
cmake $NUPIC_CORE -DCMAKE_BUILD_TYPE=Release \
   -DCMAKE_INSTALL_PREFIX=../release \
-DPY_EXTENSIONS_DIR=$NUPIC_CORE/bindings/py/src/nupic/bindings

# While still in $NUPIC_CORE/build/scripts
make -j3

# While still in $NUPIC_CORE/build/scripts
make install

# Validate install running cpp_region_test and unit_tests
cd $NUPIC_CORE/build/release/bin
./cpp_region_test
./unit_tests

#Note:
#This step worked on x86_64 as it has pip 9.0.1 and failed on
# odroid-xu4 due to older pip 8.1.1 ./unit_tests
git clone https://github.com/numenta/nupic
cd nupic

#Follow the detailed instructions in README.md to compile and install

# Install from local source code
sudo pip install -e .

sudo apt-get install python-pytest

Note:
This step worked on x86_64 as it has pip 9.0.1 and failed on
odroid-xu4 due to older pip 8.1.1 py.test tests/unit

Caffe

Caffe is Deep learning framework from Berkeley AI Research (BAIR) and The Berkeley Vision and Learning Center (BVLC). Tutorials and installation guides can be found in the project’s README.

I compiled Caffe on the odroid-xu4 successfully and was able to run tests. I had to install several dependencies for “make all” to complete successfully, and at the end I was able to run “make runtest” which ran 1110 tests from 152 test cases. The experience is relatively painless!

Here is a short summary of the install steps for installing and running tests.

https://github.com/BVLC/caffe.git
cd caffe
# Created Makefile.config from Makefile.config.example
# Uncomment CPU_ONLY=1 for Caffe cpu version

# Install dependenices
sudo apt-get install protobuf-compiler libboost-all-dev
sudo apt-get install libgoogle-glog-dev
sudo apt-get install libopenblas-dev
sudo apt-get install libopencv-highgui-dev
sudo apt-get install libhdf5-dev
export CPATH="/usr/include/hdf5/serial/"
sudo apt-get install libleveldb-dev
sudo apt-get install liblmdb-dev
sudo apt-get install libatlas-base-dev
sudo apt-get install libgflags-dev

make all
make test
make runtest
[==========] 1110 tests from 152 test cases ran. (197792 ms total)
[  PASSED  ] 1110 tests.

The Caffe command can be found under .build_release/tools/caffe.

caffe: command line brew
usage: caffe <command> <args>

commands:
  train           train or finetune a model
  test            score a model
  device_query    show GPU diagnostic information
  time            benchmark model execution time

  Flags from tools/caffe.cpp:
    -gpu (Optional; run in GPU mode on given device IDs separated by ','.Use
      '-gpu all' to run on all available GPUs. The effective training batch
      size is multiplied by the number of devices.) type: string default: ""
    -iterations (The number of iterations to run.) type: int32 default: 50
    -level (Optional; network level.) type: int32 default: 0
    -model (The model definition protocol buffer text file.) type: string
      default: ""
    -phase (Optional; network phase (TRAIN or TEST). Only used for 'time'.)
      type: string default: ""
    -sighup_effect (Optional; action to take when a SIGHUP signal is received:
      snapshot, stop or none.) type: string default: "snapshot"
    -sigint_effect (Optional; action to take when a SIGINT signal is received:
      snapshot, stop or none.) type: string default: "stop"
    -snapshot (Optional; the snapshot solver state to resume training.)
      type: string default: ""
    -solver (The solver definition protocol buffer text file.) type: string
      default: ""
    -stage (Optional; network stages (not to be confused with phase), separated
      by ','.) type: string default: ""
    -weights (Optional; the pretrained weights to initialize finetuning,
      separated by ','. Cannot be set simultaneously with snapshot.)
      type: string default: ""

TensorFlow

Now, on to TensorFlow: a machine learning framework. It provides a C API defined in c_api.h, which is suitable for building bindings for other languages; I played with TensorFlow and the C API, and I compiled TensorFlow and Bazel, a pre-requisite, from sources. Compiling from sources took a long time on the odroid-xu4 and ran into the same pip version issue I mentioned earlier. On the x86_64 platform, I managed to write a hello_tf.c using the C API and watch it run.

The Tensorflow website has a great guide that explains how to prepare your Linux environment. There are two choices for installing Bazel, you can either install Bazel binaries, or compile from sources. Here are my setup notes for compiling Bazel from sources:

sudo apt-get install build-essential openjdk-8-jdk python zip
sudo apt-get install python-numpy python-dev python-pip python-wheel

mkdir bazel; cd bazel

wget https://github.com/bazelbuild/bazel/releases/download/0.10.0/bazel-0.10.0-dist.zip

unzip bazel-0.10.0-dist.zip

./compile.sh

# The above failed with "The system is out of resources."
# message. Workaround reference:
# https://github.com/bazelbuild/bazel/issues/1308

vi scripts/bootstrap/compile.sh

# find the line:
"run "${JAVAC}" -classpath "${classpath}" 
 -sourcepath "${sourcepath}""
add -J-Xms256m -J-Xmx384m
as shown in 
"run "${JAVAC}" -J-Xms256m -J-Xmx384m -classpath
 "${classpath}" -sourcepath "${sourcepath}""

# Run compile again: Compile should work now.
./compile.sh

sudo pip install six numpy wheel

cd tensorflow
export PATH=$PATH:../linux_ai/bazel/output
./configure

# The bazel build command builds a script named build_pip_package. Running this script as follows will build a .whl file within the /tmp/tensorflow_pkg directory:

bazel build --jobs 1 --local_resources 2048,0.5,1.0 --verbose_failures --config=opt //tensorflow/tools/pip_package:build_pip_package

bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
sudo pip install /tmp/tensorflow_pkg/tensorflow-1.6.0rc0-cp27-cp27mu-linux_armv7l.whl

You can validate your TesnforFlow installation by running python in a different directory and executing the following code:

python

Python 2.7.14 (default, Sep 23 2017, 22:06:14) 
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow!')
>>> sess = tf.Session()
>>> print(sess.run(hello))

#If you see "Hello, TensorFlow!" as a result, you are good to start
#developing TensorFlow programs.

For added fun, I also installed the TensorFlow C API, here is the sample program:

#include <stdio.h>
#include <tensorflow/c/c_api.h>

int main()
{
	printf("Hello from TensorFlow C library version %s\n", TF_Version());
	return 0;
}
gcc -I/usr/local/include -L/usr/local/lib hello_tf.c -ltensorflow
./hello_tf 
Hello from TensorFlow C library version 1.5.0

Summary

My experience has given me a few tips and takeaways I think are worth sharing:

  • Most AI frameworks depend on new tool chain versions, so it’s important to ensure you have latest versions of things like pip and python.
  • Compiling TensorFlow can be very tedious.
  • Caffe installed easily on odroid-xu4.
  • Unlike TesnsorFlow Java and TensorFlow Go, TensorFlow for C appears to be covered by the TensorFlow API stability guarantees.

This blog should have provided you with a quick summary of the select AI projects I experimented with, and I hope some of my notes might be useful to you. My work in this exciting area continues, and I will share more new discoveries in the future. If you’re working on some interesting AI projects, we’d love to hear about them in the comments below.

An Introduction to IoT.js Architecture

IoT.js is a lightweight JavaScript platform for the Internet of Things. The platform keeps interoperable services at the forefront, and is designed to bring the success of Node.js to IoT devices like micro-controllers and other devices that are constrained to limited storage and only a few kilobytes of RAM. IoT.js is built on top of JerryScript: a lightweight JavaScript interpreter, and libtuv: an event driven (non-blocking I/O model) library. The project is open source under the Apache 2.0 license.

This article will introduce you to the architecture of IoT.js and the fundamentals of writing applications for it.

IoT.js Architecture

JerryScript – ECMAScript binding

JerryScript is the kernel for IoT.js on an ECMAScript binding, it’s an ultra lightweight JavaScript engine that was written from scratch at Samsung. The name “Jerry” comes from the popular character in Tom and Jerry, who’s small, smart, and fast! Since the engine is an interpreter only, it might be more precise to say JerryScript is a JavaScript interpreter.

Optimizations for memory footprint and performance have always been the top priorities of the project. The tiny engine has a base RAM footprint of less than 64KB, and the binary can accommodate less than 200KB of ROM. Amazingly, it implements the full ECMAScript 5.1 standard, and work has been ongoing recently to introduce new ES6 features such as promise, TypedArray, and more.

Like IoT.js, JerryScript was also released under the Apache 2.0 license. The community has experienced very rapid growth, especially in last couple years, and in 2016 the project was transferred to the JavaScript Foundation. A recent JavaScript Foundation press release mentioned how JerryScript was adopted in Fitbit’s latest product: Ionic.

JerryScript provides a good set of embedding APIs to compile and execute JavaScript programs, access JavaScript objects and their values, handle errors, manage the lifestyles of objects, and more. IoT.js uses these API’s to create the builtin module and native handler in IoT.js native core.

libtuv – I/O Event binding

Asynchronous I/O and threading in IoT.js are handled with libtuv: a library that focuses on asynchronous I/O and was primarily developed for use with Node.js. Samsung launched this open source project under the Apache 2.0 license, and it’s a multi-platform tiny event library that’s refactored from libuv to better serve IoT.js and on embedded systems.

Libtuv’s features include a loop, timer, poll, tcp & udp, fs event, thread, worker, and more. The platforms this library supports include i686-linux, arm-linux, arm-nuttx and arm-mbed.

IoT Binding

In the IoT.js community, there have been discussions about binding to an existing IoT platform or specification, such as IoTivity: The Open Connectivity Foundation‘s open source project. If this were to happen it would certainly add more dimension to supporting interoperability with other platforms.

IoT.js C Core

The IoT.js core layer is located above the binding layer, and it provides upper layer functionality to interact with the JavaScript engine, including running main event loops, managing I/O resources, etc. It also provides a set of builtin modules and native handlers.

Builtin modules are the basic and extended modules that are included in the IoT.js binary. Basically, these builtin modules are JavaScript objects implemented in C using the embedding API JerryScript provides, in either JavaScript or both languages. The native components of builtin modules are implemented as a native handle to access underlying systems via event handling, a C library, or system calls.

The life cycle of IoT.js is shown below:

 

IoT.js ECMAScript API and JavaScript Modules

Like Node.js, IoT.js is a module-based system. Each module in IoT.js has its own context and provides a set of API’s associated with the module’s functionality.

IoT.js offers basic API modules and extended API modules. The basic API modules are based on Node.js and follow same form for compatibility reasons. Basic API modules include File System, Net, HTTP, Process, etc. Pretty much all application code that calls these API’s can be run in a Node.js environment without any modification.

The extended modules, on the other hand, are more IoT.js specific, and they are currently mostly hardware related (e.g. GPIO, Bluetooth Low Energy (BLE), I2C, SPI, UART, PWM, etc.). Many contributors are interested in adding new extended API modules to support their own specific hardware, so to maintain consistent usability, the IoT.js community has set guidelines and rules for introducing extended API’s.

Enabling JavaScript on the Internet of Things

The overall architecture of IoT.js is very friendly to Node.js, as a result of the asynchronous I/O and threading library, and the subset of Node.js compatible modules; it has reflected the design philosophy of providing a lightweight version of Node.js along with an inter-operable service platform. IoT.js has opened a great opportunity for JavaScript developers to develop applications for the Internet of Things, so it’s definitely an IoT platform to watch!

Now Hiring: Open Source Cloud Engineers

The cloud is ripe with open source software as the collaborative development model has proven to be extremely effective at commoditizing much of the software stack. Today, foundations like the Cloud Native Computing Foundation are bringing together numerous companies to collaborate on vital, cutting edge technology, and open source software is doing what it has done in many other industries as it positions itself to be critical to the adoption of cloud technologies. More companies are launching open source initiatives to improve this technology domain and the Open Source Group recognizes this as a valuable place for us to increase our breadth of impact on Samsung’s innovation.

We’re Hiring Cloud Experts!

With this said, we’re pleased to announce we’re hiring multiple positions for experts in open source cloud technologies to help us in our mission to improve Samsung’s understanding and adoption of open source software. Here are the most important skills we’re looking for:

  • Programming experience in one or more languages, including C++, Go, Python, Shell, C#, Ruby, JavaScript, and PHP. Please note that the language competency depends on the project you’re expected to get involved in.
  • Expertise contributing to one or more of these projects: Kubernetes, OpenTracing, Jaeger, Envoy, Linkerd, gRPC, Fluentd, and related CNCF projects.
  • Knowledge of distributed tracing, monitoring, service management, and container orchestration.
  • Demonstrable experience participating in open source communities as a contributor, committer, or maintainer.
  • A history of providing technical leadership within open source communities and engaging in political influence and guidance.
  • A high degree of self-motivation and the ability to work alone, managing your own work and setting sensible priorities.
  • Background in pipeline operations, packaging and deployment of cloud applications, virtualization technologies and API’s,

However, don’t worry if you aren’t experienced with all of these skills; the most important skill you can have is the ability to expand your own technical knowledge to meet the rapidly changing open source software ecosystem. If you have knowledge of any of these items with a history of good open source contributions, we want you!

Why Work for Us?

You’ll want to work for us if you’re interested in:

  • Spending 100% of your development time on upstream open source projects,
  • Working from almost anywhere in the world (as long as you have an internet connection),
  • Using your preferred infrastructure for your development environment,
  • The freedom to work outside the pressure of product deadlines, and
  • Improving your visibility with open source communities via participation in project governance and events.

If you’re someone who’s passionate about open source software and deeply knowledgeable about the things mentioned in this post, we’d love to hear from you. If you’re interested, send us a message on twitter @SamsungOSG or an email to hiring[at]osg.samsung.com. If you want to learn more about our team, check out this blog post that explains why our team was created.

We look forward to meeting some great open source engineers!

How to Run IoT.js on the Raspberry PI 0

IoT.js is a lightweight JavaScript platform for building Internet of Things devices; this article will show you how to run it on a few dollars worth of hardware. The First version of it was released last year for various platforms including Linux, Tizen, and NuttX (the base of Tizen:RT). The Raspberry Pi 2 is one of the reference targets, but for demo purposes we also tried to build for the Raspberry Pi Zero, which is the most limited and cheapest device of the family. The main difference is the CPU architecture, which is ARMv6 (like the Pi 1), while the Pi 2 is ARMv7, and the Pi 3 is ARMv8 (aka ARM64).

IoT.js upstream uses a python helper script to crossbuild for supported devices, but instead of adding support to new device I tried to build on the device using native tools with cmake and the default compiler options; it simply worked! While working on this, I decided to package iotjs for debian to see how well it will support other architectures (MIPS, PPC, etc), we will see.

Unfortunately, Debian armel isn’t optimized for ARMv6 and FPU, both of which are present on the Pi 1 and Pi 0, so the Raspbian project had to rebuild Debian for the ARMv6+VFP2 ARM variant to support all Raspberry Pi SBC’s.

In this article, I’ll share hints for running IoT.js on Raspbian: the OS officially supported by the Raspberry foundation; the following instructions will work on any Pi device since a portability strategy was preferred over optimization. I’ll demonstrate three separate ways to do this: from packages, by building on the device, and by building in a virtual machine. By the way, an alternative to consider is to rebuild Tizen Yocto for  the Pi 0, but I’ll leave that as an exercise for the reader, you can accomplish this with a bitbake recipe, or you can ask for more hints in the comments section.

Install from Packages

iotjs landed in Debian’s sid, and until it is in testing branch (and subsequently Raspbian and Ubuntu), the fastest way is to download it is via precompiled packages from my personal Raspbian repo

url='https://dl.bintray.com/rzr/raspbian-9-armhf'
source="/etc/apt/sources.list.d/bintray-rzr-raspbian-9-armhf.list"
echo "deb $url raspbian main" | sudo tee "$source"
sudo apt-get update
apt-cache search iotjs
sudo apt-get install iotjs
/usr/bin/iotjs
Usage: iotjs [options] {script | script.js} [arguments]

Use it

Usage is pretty straightforward, start with a hello world source:

echo 'console.log("Hello IoT.js !");' > example.js
iotjs  example.js 
Hello IoT.js !

More details about the current environment can be used (this is for iotjs-1.0 with the default built-in modules):

echo 'console.log(JSON.stringify(process));' > example.js
iotjs  example.js 
{"env":{"HOME":"/home/user","IOTJS_PATH":"","IOTJS_ENV":""},"native_sources":{"assert":true,"buffer":true,"console":true,"constants":true,"dns":true,"events":true,"fs":true,"http":true,"http_client":true,"http_common":true,"http_incoming":true,"http_outgoing":true,"http_server":true,"iotjs":true,"module":true,"net":true,"stream":true,"stream_duplex":true,"stream_readable":true,"stream_writable":true,"testdriver":true,"timers":true,"util":true},"platform":"linux","arch":"arm","iotjs":{"board":"\"unknown\""},"argv":["iotjs","example.js"],"_events":{},"exitCode":0,"_exiting":false} null 2

From here, you can look to use other built-in modules like http, fs, net, timer, etc.

Need More Features?

More modules can be enabled in the master branch, so I also built snapshot packages that can be installed to enable more key features like GPIO, I2C, and more. For your convenience, the snapshot package can be installed to replace the latest release:

root@raspberrypi:/home/user$ apt-get remove iotjs iotjs-dev iotjs-dbgsym iotjs-snapshot
root@raspberrypi:/home/user$ aptitude install iotjs-snapshot
The following NEW packages will be installed:
  iotjs-snapshot{b}
(...)
The following packages have unmet dependencies:
 iotjs-snapshot : Depends: iotjs (= 0.0~1.0+373+gda75913-0~rzr1) but it is not going to be installed
The following actions will resolve these dependencies:
     Keep the following packages at their current version:
1)     iotjs-snapshot [Not Installed]                     
Accept this solution? [Y/n/q/?] n
The following actions will resolve these dependencies:

     Install the following packages:                 
1)     iotjs [0.0~1.0+373+gda75913-0~rzr1 (raspbian)]
Accept this solution? [Y/n/q/?] y
The following NEW packages will be installed:
  iotjs{a} iotjs-snapshot 
(...)
Do you want to continue? [Y/n/?] y
(...)
  iotjs-snapshot https://dl.bintray.com/rzr/raspbian-9-armhf/iotjs-snapshot_0.0~1.0+373+gda75913-0~rzr1_armhf.deb
  iotjs https://dl.bintray.com/rzr/raspbian-9-armhf/iotjs_0.0~1.0+373+gda75913-0~rzr1_armhf.deb

Do you want to ignore this warning and proceed anyway?
To continue, enter "yes"; to abort, enter "no": yes
Get: 1 https://dl.bintray.com/rzr/raspbian-9-armhf raspbian/main armhf iotjs armhf 0.0~1.0+373+gda75913-0~rzr1 [199 kB]
Get: 2 https://dl.bintray.com/rzr/raspbian-9-armhf raspbian/main armhf iotjs-snapshot armhf 0.0~1.0+373+gda75913-0~rzr1 [4344 B]
(...)

If you the run console.log(process) again, you’ll see more interesting modules to use, like gpio, i2c, uart and more, and external modules can be also used; check on the work in progress for sharing modules to the IoT.js community. Of course, this can be reverted to the latest release by simply installing the iotjs package because it has higher priority than the snapshot version.

root@raspberrypi:/home/user$ apt-get install iotjs
(...)
The following packages will be REMOVED:
  iotjs-snapshot
The following packages will be upgraded:
  iotjs
(...)
Do you want to continue? [Y/n] y
(...)

Build on the Device

It’s also possible to build the snapshot package from source with extra packaging patches, found in the community branch of IoT.js (which can be rebased on upstream anytime).

sudo apt-get install git time sudo
git clone https://github.com/tizenteam/iotjs
cd iotjs
./debian/rules
sudo debi

On the Pi 0, it took less than 30 minutes over NFS for this to finish. If you want to learn more you can follow similar instructions for building IoTivity on ARTIK;
it might be slower, but it will extend life span of your SD Cards.

Building IoTivity for ARM on ARTIK Devices

Build on a Virtual Machine

A faster alternative that’s somewhere between building on the device and setting up a cross build environment (which always has a risk of inconsistencies) is to rebuild IoT.js with QEMU, Docker, and binfmt.

First install docker (I used 17.05.0-ce and 1.13.1-0ubuntu6), then install the remaining tools:

sudo apt-get install qemu qemu-user-static binfmt-support time
sudo update-binfmts --enable qemu-arm

docker build 'http://github.com/tizenteam/iotjs.git'

It’s much faster this way, and took me less than five minutes. The files are inside the container, so they need to be copied back to host. I made a helper script for setup and to get the deb packages ready to be deployed on the device (sudo dpkg -i *.deb) :

curl -sL https://rawgit.com/tizenteam/iotjs/master/run.sh | bash -x -
./tmp/out/iotjs/iotjs-dbgsym_0.0-0_armhf.deb
./tmp/out/iotjs/iotjs-dev_0.0-0_armhf.deb
./tmp/out/iotjs/iotjs_0.0-0_armhf.deb

I used the Resin/rpi-raspbian docker image (thanks again Resin!). Finally, I want to also thank my coworker Stefan Schmidt for the idea after he setup a similar trick for EFL’s CI.

Further Reading

If you want to learn more, here are some additional resources to take your understanding further.

Raspberry Pi is a trademark of the Raspberry Pi Foundation\

Common EFL Focus Pitfalls

I started patching the focus subsystem of the EFL widget toolkit quite some time ago. During this time, people have started to assign me everything that somehow looks like an issue with focus, it sometimes only takes the existence of the word “focus” somewhere in a backtrace for this to happen. I’ve discovered that most people mix up these different types, so in this blog post I hope to provide some clarity about them.

How EFL Gets Focused

There are 3 different places focus happens in EFL:

  • ecore-evas – the window manager abstraction
  • evas – the canvas library
  • elementary – the widget toolkit.

First of all, I should point out what focus itself is, I think a good example is to consider your typical smartphone interaction. While interacting with your smartphone, your complete attention is given to its screen and all interactions are with the interface of the device; you will also likely cease interactions with the outside environment entirely. In the same way, each abstraction in EFL has its own interaction partners:

  • The focused canvas object gets the attention from the keyboard
  • The focused widget is highlighted visually, so the user can see where his attention should go
  • And the window manager in the end focuses an application, which is probably an EFL Application.

These differences are often the source of people’s confusion when it comes to focus in EFL. For example, loosing the toolkit focus on a window object does not mean that the window lost the input from the user; instead, it means that another widget got the toolkit focus and the window manager still has focus on this window.

For another example, consider a toolkit widget that’s built out of two objects: an image and a text field below the image. In this example, the widget receives the toolkit’s focus, and the focus of the canvas moves to the image. Then, the user presses some key bindings to change the name of the image and the canvas focus moves to the text field. In this case, the canvas focus moves, creating update events on the canvas focus. However, the widget’s focus stayed the same, and the user is meant to have their attention on that widget, meaning there was no change to it.

Some Tips for Understanding EFL Focus

The focus property can only be true on one single object in an entity, in practice this means:

  • One window focused per user window manager session
  • One object focused per user canvas
  • One widget focused per widget tree in the toolkit

Additionally:

  • Canvas focus is only used for injecting keyboard inputs from the input system of the display technology.
  • Widget focus is used for navigation, in the case of focus movement initiated by key bindings, this is the position from where the next upper/down/right/left element is calculated.

If you ever use change events for some kind of a focus property, here’s what you need to know about focus in EFL:

  • It’s window manager focus if the user has their attention on your application.
  • It’s canvas focus if you need to know where the keyboard inputs are coming from.
  • It’s a widget focus if the user has moved their attention to a subset of canvas objects that are bound together as a widget implementation.

If you have any questions about any of this content, head to the comments section!

How to Securely Encrypt A Linux Home Directory

These days our computer enables access to a lot of personal information that we don’t want random strangers to access, things like financial login information comes to mind. The problem is that it’s hard to make sure this information isn’t leaked somewhere in your home directory like the cache file for your web browser. Obviously, in the event your computer gets stolen you want your data at rest to be secure; for this reason you should encrypt your hard drive. Sometimes this is not a good solution as you may want to share your device with someone who you might not want to give your encryption password to. In this case, you can  encrypt only the home directory for your specific account.

Note: This article focuses on security for data at rest after that information is forever out of your reach, there are other threat models that may require different strategies.

Improving Upon Linux Encryption

I have found the home directory encryption feature of most Linux distributions to be lacking in some way. My main concern is that I can’t change my password as it’s used to directly unlock the encrypted directory. So, what I would like is the ability to have is a pass phrase that is used as a encryption key, but is itself encrypted with my password. This way, I can change my password by re-encrypting my pass phrase with a different password. I want it to be a pass phrase because that makes it possible to back it up in my global password manager which protects me in the case the file holding the key gets corrupted, and it allows me to share it with my IT admin who can then put it in cold storage.

Generate a Pass Phrase

So, how do we do this? First generate the key, for that I have slightly modified a Bitcoin wallet pass phrase generator that I use to generate the key as follows:

$ git clone https://github.com/Bluebugs/bitcoin-passphrase-generator.git
$ cd bitcoin-passphrase-generator
$ ./bitcoin-passphrase-generator.sh 32 2>/dev/null | openssl aes256 -md sha512 -out my-key

This will prompt you for the password to encrypt your pass phrase, and this will become your user password. Once this command has been called, it will generate a file that contains a pass phrase using 32 words out of your system dictionary. On Arch Linux, this dictionary contains 77,649 words, so one possibility in 77649^32. Anything more than 2^256, should be secure enough for some time. To check the content of your pass phrase and back it up, run the following:

$ openssl aes256 -md sha512 -d -in my-key

Once you have entered your password it will output your pass phrase: a line of words that you can now backup in your password manager.

Encrypt Your Home Directory

Now, let’s encrypt the home directory. I have chosen to use ext4 per directory encryption because it enables you to share the hard drive with other users in the future instead of requiring you to split it into different partitions. To initialize your home directory, first move your current one out of the way (This means stopping all processes that are running under your user ID and login under another ID). After this, load the key into the kernel:

$ openssl aes256 -md sha512 -d -in my-key | e4crypt add_key 2>1

This will ask you for the password you used to protect your pass phrase; once entered, it adds it to your current user kernel key ring. With this, you can now create you home directory, but first get the key handle with the following command:

$ keyctl show | grep ext4
12345678 --alsw-v    0   0   \_ logon: ext4:abcdef1234567890

The key handle is the text that follows ext4 at the end of the 2nd line above, this can now be used to create your new encrypted home directory :

$ mkdir /home/user
$ e4crypt set_policy abcdef1234567890 /home/user

After this, you can copy all of the data from your old user directory to the new directory and it will be properly encrypted.

Setup PAM to Unlock and Lock Your Home Directory

Now, we need to play with the PAM session handle to get it to load the key and unlock the home directory. I will assume that you have stored my-key at /home/keys/user/my-key. We’ll use pam_mount to run a custom script that will add the key in the system at login and remove it at log out. The script for mounting and unmounting can be found on GitHub.

The interesting bit that I’ve left out of the mounting script is the possibility to get the key from an external device or maybe even your phone. Also, you’ll notice the need to be part of the wheel group to cleanly unmount the directory as otherwise the file will still be in cache after log out, and it will be possible to navigate the directory and create files until they are removed. I think a clean way to handle this might be to have a script for all users that can flush the cache without asking for a sudo password; this will be an easy improvement to make. Finally, modify the pam system-login file as explained in the Arch Linux documentation with just one additional change to disable pam_keyinit.so.

In /etc/pam.d/system-login, paste the following:

#%PAM-1.0

auth required pam_tally.so onerr=succeed file=/var/log/faillog
 auth required pam_shells.so
 auth requisite pam_nologin.so
 auth optional pam_mount.so
 auth include system-auth

account required pam_access.so
 account required pam_nologin.so
 account include system-auth

password optional pam_mount.so
 password include system-auth

session optional pam_loginuid.so
 #session optional pam_keyinit.so force revoke
 session [success=1 default=ignore] pam_succeed_if.so service = systemd-user quiet
 session optional pam_mount.so
 session include system-auth
 session optional pam_motd.so motd=/etc/motd
 session optional pam_mail.so dir=/var/spool/mail standard quiet
 -session optional pam_systemd.so
 session required pam_env.so

Now, if you want to change your password, you need to decrypt the key, change the password, and re-encrypt it. The following command will do just that:

$ openssl aes256 -md sha512 -d -in my-old-key | openssl aes256 -md sha512 -out my-new-key

Et voila !