Web, Internet of Things and GDPR

From the 25th  of May 2018, the European General Data Protection Regulation, also known as GDPR, has come into force across European Union (EU) and the European Economic Area (EEA).  This new European Regulation on privacy has far-reaching consequences on how personal data is collected, stored and used in the  Internet of Things (IoT) world and across the web. For IoT and web developers, it has a significant impact on how we start to think about technologies we deploy and services we build. One promise of the open web platform is enhanced privacy in comparison to other technology stacks. Can some of the inherent privacy issues in the currently deployed IoT architectures be mitigated by the use of web technology? We think it can.

In this blog post we will look at The Web of Things (WoT) technologies we have been involved in. By walking through the options from local device discovery to a gateway framework that enables remote access, we hope to give you a picture of a loosely coupled IoT solution using Web of Things technologies. At each stage, GDPR compliance of these technologies will be discussed.

GDPR and the Internet of Things

The 88 pages long GDPR regulation document has 99 articles on the rights for all individuals and the obligations placed on organizations. These new requirements represent a significant change on how data will be handled across businesses and industries. When it comes to the Internet of Things, GDPR compliance is particularly challenging. Concerns about the risks that the Internet of Things has posed on data protection and personal privacy have been raised for a few years.  An study from ICO in 2016 stated that “Six in ten Internet of Things devices don’t properly tell customers how their personal information is being used”. The good news is that industry and governments are aware of the problem and taking actions to change this.

With the Internet of Things growing faster and smarter, a lot of new devices and new solutions will be introduced in the coming years. When looking into technical solutions and innovation, we need to have “GDPR awareness”, especially “data protection by design” that GDPR advocates, in mind.  “Data protection by design“, previously known as “privacy by design”, encourages business to implement technical and organizational measures, safeguard privacy and data protection principles within the full life cycle right from the start.

The Web of Things

Interoperability is one of the major challenges in the Internet of Things. The Web of Things addresses this challenge by providing  a universal application layer protocol for the Things to talk to each other regardless of the physical and transport layer protocols used. Rather than reinventing the wheel, the Web of Things reuses existing and well-known web standards and web technologies. The Web of Things addresses Things via URLs and following standard APIs. Using the web as the framework makes things discoverable and linkable, and provides web developers with an opportunity to let people interact via a wide range of interfaces: screens, voice, gesture, augmented reality… even without an Internet connection!

Web of Things has been referred to as the application layer of the Internet of Things on many occasions. Bearing in mind though, the scope of IoT applications is broader and includes systems that are not necessarily accessible through the web. We prefer to think  that the Web of Things is an option of an application layer added over the network layer of the traditional Internet of Things architecture.

The Web of Things is growing fast in its standardization and implementations. The W3C has launched the Web of Things Working Group to take a leading role at addressing the challenge of fragmentation of the IoT through standards. Early this year Mozilla announced the “Project Things” framework – an open framework of software and services that bridges the gap between devices via the web. The “Project Things” is a good reflection of Mozilla’s involvement in creating an open standard with the W3C around the Web of Things, and practical implementations that provide open interfaces to devices from different vendors.

For developers, the fast growth of the Web of Things has opened up a lot opportunities when we search for solutions for the Internet of Things.  Let’s walk through an example to see what this means.

Imagine that you are a homeowner and just bought a new washing machine. To have the new device integrated into your smart home and be able to control or access it, here are some optional solutions that the Web of Things can offer:

  • Discover, pair and connect new devices around you via Physical Web and Web Bluetooth.
  • Control devices via a Progressive Web Application
  • Communicate with your device via an end-to-end encrypted channel provided by Mozilla Web of Things Gateway, locally and remotely.

So what are these technologies about? How do they address privacy and security concerns? Let’s walk through them in a bit more detail.

Physical Web and Web Bluetooth

The Physical Web is a discovery service based on Bluetooth Low Energy (B.L.E.) technology. In the Physical Web concept, any object or location can be a source of content and addressed using a URL. The idea is that smart objects broadcast relevant URLs to nearby devices by way of a BLE beacon.

The Physical Web has been brought into Samsung’s web browser, Samsung Internet, as an extension called CloseBy. When the browser handles the URL information received on the phone,  “no personal information is included and our servers do not collect any information about users at all,” stated my colleague  Peter O’Shaughnessy in his article on “Bringing the real world to your browser with CloseBy”.

Web Bluetooth is another technology based on Bluetooth Low Energy. Alongside efforts like the Physical Web, it provides a set of APIs to connect and control B.L.E. devices directly from web. With this web API, developers are able to build one solution that could work across all platforms. Although the API is still under development, there are already a few very cool projects and applications around. My colleague Peter has produced  a Web Bluetooth Parrot Drone demo to give you a sense of controlling a physical device using a web browser. And I promise you that playing Jo’s Hedgehog Curling game is simply light and fun!

The Web Bluetooth Community Group aims to provide APIs that allow websites to communicate with devices in a secure and privacy-preserving way. Although some security features are already in place, for example, the site must be served over a secure connection (HTTPS) and discovering Bluetooth devices must be triggered by a user gesture , there are still lots security implications to consider as per the Web Bluetooth security model by Jeffrey Yasskin.

Samsung Internet had Web Bluetooth enabled for developers by default since v6.4 stable Release.  With Physical Web and Web Bluetooth available, it is possible to have device on-boarding “just a tap away”.

Progressive Web Application

Progressive Web Apps (PWAs) are websites that deliver native app-like user experiences. They address issues in native mobile applications and websites with new design concepts and new Web APIs. Some key features of PWAs include:

  • “Add to Home Screen” prompts
  • Offline functionality
  • Fast loading from cache
  • (Optionally) web push notifications

These features are achieved by deploying a collection of technologies. Service Workers are the core of Progressive Web Apps and work in the background to power the offline functionality, push notifications and other striking features. With the app shell concept, PWAs can achieve a fast loading time by caching the basic shell of an application UI, while still loading fresh content when possible. The native install banner installs the mobile website to the homescreen with Web App Manifest support.

In PWAs, you can only register service workers on pages served over HTTPS. Since service workers process all network requests from the web app in the background, this is essential to prevent a man-in-the-middle attack. PWAs can work offline as we mentioned earlier. From privacy perspective, this potentially offers us possibilities to:

  • Minimize Collecting, storing, and using user data as much as possible.
  • Know where the data resides.

Since the term “Progressive Web Apps”  was first coined by Google in 2015,  we have seen big brands such as FT, Forbes and Twitter switching to PWAs in the last few years. Our Samsung Internet Developer Advocacy team has been actively contributing to and promoting this new technology [1] [2] [3] [4] [5], and even designed a community-approved Progressive Web Apps logo! Want to have hands-on experiences of PWAs? Check our demos on Podle, Snapwat, Airhorn VR, PWAcman and Biscuit Tin-der.

Mozilla “Project Things”

Mozilla “Project Things” aims at “building a decentralized ‘Internet of Things’ that is focused on security, privacy, and interoperability”, as stated by the company. The structure of the framework is shown as below –

  • Things Cloud. It provides a collection of IoT cloud services including supports for setup, backup, updates, 3rd party applications and services integration, and remote encrypted tunneling.
  • Things Gateway. Generally speaking the Things Gateway is the IoT connectivity hub for your IoT network.
  • Things Controllers. Smart devices such as smart speakers, tablets/phones, AR headsets etc, to control the connected Things.
  • Things Device Framework. It consists of a collection of sensors and actuators, or “Things” in the context of the Web of Things.

The “Things Project” has introduced an add-on system, which is loosely modeled after the add-on system in Firefox, to allow for the addition of new features or devices such as an adapter to the Things Gateway. My colleague Phil Coval has posted a blog explaining how to get started and establish basic automation using I2C sensors and actuators on a gateway’s device.

The framework has provided security and privacy solutions such as:

  • Secure remote access is achieved using HTTPS via encrypted tunneling. Basically, the “Things Project” provides a TLS tunnelling service via its registration server to allow people to easily set up a secure subdomain during first time setup . An SSL certificate is generated via LetsEncrypt and a secure tunnel from a Mozilla cloud server to the gateway is set up using PageKite.
  • The Things Gateway provides a system for safely authorizing third-party applications using the de-facto authorization standard OAuth 2.0. When a third-party application needs to access or control another person’s Things, it always requires consent from the Things’ owner. The owner can decide the scope of the access token granted to the third-party application. Things’ owner also has options to delete or revoke the tokens that are assigned to the third-party application. Details on this have been discussed at our recent blog “An End-to-End Web IoT Demo Using Mozilla Gateway” and talk “The Complex IoT Equation.

Future work

GDPR challenges us all to re-prioritize digital privacy and to reconsider how and when we need to collect, manage and store people’s data. This is an opportunity for the the Web of Things. As the technology develops, some of the security and privacy issues have been or being addressed. Still, building the Web of Things has various challenges ahead. For developers, using the right technologies are a way forward to make the Internet of Things a better place. Join us on this exciting journey!

 

An End-to-End Web IoT Demo Using Mozilla Gateway

Imagine that you are on your way to a holiday home you have booked. The weather is changing and you might start wondering about temperature settings in the holiday home. A question might pop up in you mind: “Can I change the holiday home settings to my own preference before reaching there?”

Today we are going to show you an end-to-end demo we have created that allows a holiday maker to remotely control sensor devices, or Things in the context of Internet of Things, at a holiday home. The private smart holiday home is built with the exciting Mozilla Things Gateway – an open Gateway that anybody can now create with a Raspberry Pi to control Internet of Things devices.  For holiday makers, we provided a simple Node.js holiday application to access Things via Mozilla Things Gateway. Privacy is addressed by introducing concepts of Things ownership and Things usership, which is followed by the authorization work flow.

The Private Smart Holiday Home

The private smart holiday home is the home for Gateway and Things –

Things Gateway

One of the major challenges in the Internet of Things is interoperability. Getting different devices to work nicely with each other can be painful in a smart home environment. Mozilla Things Gateway addresses this challenge and provides a platform that bridges existing off-the-shelf smart home devices to the web by providing them with web URLs and a standardized data model and API [1]Implementations of the Things Gateway follows the proposed Web of Things standard and is open sourced under Mozilla Public License 2.0.

In this demo, we chose Raspberry Pi 3 as the physical board for the Gateway.  Raspberry Pi 3 is well-supported by the Gateway community and has been a brilliant initial choice for experimenting the platform. It is worth mentioning that the Mozilla Project Things is not tied only to the Raspberry Pi, but they are looking at supporting a wide range of hardwares.

The setup of the Gateway is pretty straightforward. We chose to use the tunneling service provided by Mozilla by creating a sub-domain of mozilla-iot.orgsosg.mozilla-iot.org. To try it yourself, we recommend going through the README file at Gateway github repository.  Also a great step-by-step guide has been created by Ben Francis on “How to build a private smart home with a Raspberry Pi and things Gateway”.

Things Add-ons

The Mozilla Things Gateway has introduced an Add-on system, which is loosely modeled after the add-on system in Firefox, to allow for the addition of new feature or device such as an adapter to the Things Gateway. The tutorial from James Hobin and Michael Stegeman on “Creating an Add-on for Project Things Gateway” is a good place to grab the concepts of Add-on, Adapter, Device and to start creating your own Add-ons. In our demo, we have introduced fan, lamp and thermostat Add-ons as shown below to support our own hardware devices.

Phil Coval has posted a blog explaining how to get started and how to establish basic automation using I2C sensors and actuators on gateway’s device. It is the base for our Add-ons.

Holiday Application

The holiday application is a small Node.js program that has functionalities of a client web server, OAuth client and browser User Interface.

The application consists of two parts. First is for the holiday maker to get authorization from the Gateway for accessing Things at the holiday home. Once authorized, it moves to the second part, Things access and control.



OAuth client implementation is based on simple-oauth2, a Node.js client library for OAuth2.0. The library is open sourced under Apache License, Version 2.0  and available at github.

The application code can be accessed here. The README file provides instructions for setting up the application.

Ownership and Usership of Things

So here we have it, the relationships among Things owner, Things user, third party application, Gateway and Things.

whole_picture2

  • The holiday home owner is the Things Gateway user and has full control of the Things Gateway and Things.
  • The holiday maker is a temporary user of Things and has no access to the Gateway.
  • The holiday home owner uses the Gateway to authorize the holiday maker  accesses to the Things with scopes via gateway.
  • Holiday application accesses the Things through the Gateway.

User Authorization

The Things Gateway provides a system for safely authorizing third-party applications using the de-facto authorization standard OAuth 2.0. The work flow for our demo use case is shown in the diagram below –

The third party application user, the Holiday App User in this case, requests authorization to access the Gateway’s Web Thing API. The Gateway presents the request list to the Gateway User, the holiday home owner, as below –

With the holiday user’s input, the Gateway responds with an authentication code. The holiday application then requests to exchange the authentication code to a JSON Web Token (JWT) . The token has a scope that indicates what accesses were actually granted by the holiday home owner. It is noted that the granted token scope can only be a subset of the request scope. With the JWT granted, the holiday application can access the Things that the user is granted to.

Demo Video

We also created a demo video which tailored together different parts we talked above, and is available at https://vimeo.com/271272094.

What’s Next?

The Mozilla Gateway is a work in progress and is not yet reached production use stage.  There are a lot exciting developments happening. Why not get involved?

Connecting sensors to Mozilla’s IoT Gateway

Here is a 1st post about Mozilla’s IoT effort, and specifically the gateway project which is illustrating “Web Of Things” concept to create a decentralized Internet of Things, using Web technologies.

Today we will focus on the gateway, as it is the core component of the whole framework. Version 0.4.0 was just released, so you can try it your own on Raspberry Pi 3.  The Raspberry Pi 3 is the reference platform, but it should be possible to port to other single board computers (like ARTIK, etc).

The post will explain how to get started and how to establish basic automation using I2C sensors and actuators on gateway’s device (without any cloud connectivity).

To get started, first install the gateway according to these straightforward instructions:

Prepare SD Card

You need to download the Raspbian based gateway-0.4.0.img.zip (1GB archive) and dump it to SD card (2.6GB min).

lsblk # Identify your sdcard adapter ie:
disk=/dev/disk/by-id/usb-Generic-TODO
url=https://github.com/mozilla-iot/gateway/releases/download/0.4.0/gateway-0.4.0.img.zip
wget -O- $url | funzip | sudo dd of=$disk bs=8M oflag=dsync

If you only want to use the gateway and not hack on it, you can skip this next part which enables a developer shell though SSH.  However, if you do want access to a developer shell, mount the 1st partition called “boot” (you may need to replug your SD card adapter) and add a file to enable SSH:

sudo touch /media/$USER/boot/ssh
sudo umount /media/$USER/*

First boot

Next, install the SD card in your Raspberry PI 3 (Older RPis could work too, particularly if you have a wifi adapter).

When it has completed the first boot, you can check that the Avahi daemon is registering “gateway.local” using mDNS (multicast DNS)

ping gateway.local
ssh pi@gateway.local # Raspbian default password for pi user is "raspberry"

Let’s also track local changes to /etc by installing etckeeper, and change the default password.

sudo apt-get install etckeeper
sudo passwd pi

Logging in

You should now be able to access the web server, which is running on port 8080 (earlier version used 80):

http://gateway.local:8080/

It will redirect you to a page to configure wifi:

URL: http://gateway.local:8080/
Welcome
Connect to a WiFi network?
FreeWifi_secure
FreeWifi
OpenBar
...
(skip)

We can skip it for now:

URL: http://gateway.local:8080/connecting
WiFi setup skipped
The gateway is now being started. Navigate to gateway.local in your web browser while connected to same network as the gateway to continue setup.
Skip

After a short delay, the user should be able to reconnect to the entry page:

http://gateway.local:8080/

The gateway can be registered on mozilla.org for remote management, but we can skip this for now.

Then administrator is now welcome to register new users:

URL: http://gateway.local:8080/signup/
Mozilla IoT
Welcome
Create your first user account:
user: user
email: user@localhost
password: password
password: password
Next

And we’re ready to use it:

URL: http://gateway.local:8080/things
Mozilla IoT
No devices yet. Click + to scan for available devices.
Things
Rules
Floorplan
Settings
Log out

Filling dashboard

You can start filling your dashboard with Virtual Resources,

First hit the “burger menu” icon, go to settings page, and then go to the addons page.

Here you can enable a “Virtual Things” adapter:

URL: http://gateway.local:8080/settings/addons/
virtual-things-adapter 0.1.4
Mozilla IoT Virtual Things Adapter
by Mozilla IoT

Once enabled It should be listed along ThingURLAdapter on the adapters page:

URL: http://gateway.local:8080/settings/adapters
VirtualThingsAdapter
virtual-things
ThingURLAdapter
thing-url-adapter

You can then go back to the 1st Things page (it’s the first entry in the menu):

We can start adding “things” by pressing the bottom menu.

URL: http://gateway.local:8080/things
Virtual On/Off Color Light
Color Light
Save

Then press “Done” at bottom.

From this point, you can decide to control a virtual lamp from the UI, and even establish some basic rules (second entry in menu) with more virtual resources.

Sensing Reality

Because IoT is not about virtual worlds, let’s see how to deal with the physical world using sensors and actuators.

For sensors, there are many way to connect them to computers using analog or digital inputs on different buses.  To make it easier for applications developers, this can be abstracted using W3C’s generic sensors API.

While working on IoT.js‘s modules, I made a “generic-sensors-lite” module that abstracted a couple of I2C drivers from the NPM repository.  To verify the concept, I have made an adapter for Mozilla’s IoT Gateway (which is running Node.js), so I published the generic-sensors-lite NPM module first.

Before using the mozilla-iot-generic-sensors-adapter, you need to enable the I2C bus on the gateway (version 0.4.0, master has I2C enabled by default).

sudo raspi-config
Raspberry Pi Software Configuration Tool (raspi-config)
5 Interfacing Options Configure connections to peripherals
P5 I2C Enable/Disable automatic loading of I2C kernel module
Would you like the ARM I2C interface to be enabled?
Yes
The ARM I2C interface is enabled
ls -l /dev/i2c-1
lsmod | grep i2c
i2c_dev 16384 0
i2c_bcm2835 16384 0

Of course you’ll need at least one real sensor attached to the I2C pin of the board.  Today only 2 modules are supported:

You can double check if addresses are present on I2C the bus:

sudo apt-get install i2c-tools
/usr/sbin/i2cdetect -y 1
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- 23 -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- 77

Install mozilla-iot-generic-sensors-adapter

Until sensors adapter is officially supported by the mozilla iot gateway, you’ll need to install it on the device (and rebuild dependencies on the target) using:

url=https://github.com/rzr/mozilla-iot-generic-sensors-adapter
dir=~/.mozilla-iot/addons/generic-sensors-adapter
git clone --depth 1 -b 0.0.1 $url $dir
cd $dir
npm install

Restart gateway (or reboot)
sudo systemctl restart mozilla-iot-gateway.service
tail -F /home/pi/.mozilla-iot/log/run-app.log

Then the sensors addon can be enabled by pressing the “enable” button on the addons page:

URL: http://gateway.local:8080/settings/addons
generic-sensors-adapter 0.0.1
Generic Sensors for Mozilla IoT Gateway

It will appear on the adapters page too:

URL: https://gateway.local/settings/adapters
VirtualThingsAdapter
virtual-things
ThingURLAdapter
thing-url-adapter
GenericSensorsAdapter
generic-sensors-adapter

Now we can add those sensors as new things (Save and done buttons):

URL: http://gateway.local:8080/things
Ambient Light Sensor
Unknown device type
Save
Temperature Sensor
Unknown device type
Save

Then they will appear as:

  • http://gateway.local:8080/things/0 (for Ambient Light Sensor)
  • http://gateway.local:8080/things/1 (for Temperature Sensor)

To get value updated in the UI, they need to turned on first (try again if you find a big, and file tickets I will forward to drivers authors).

A GPIO adapter can be also used for actuators, as shown in this demo video.

If you have other sensors, check if the community has shared a JS driver, and please let me know about integrating new sensors drivers in generic-sensors-lite

IoT.js landed in Raspbian

Following previous efforts to deploy iotjs on Raspberry Pi 0, I am happy to announce that IoT.js 1.0 landed in Debian, and was sync’d to Raspbian for ArmHF and Ubuntu as well.

While the package is targeting the next distro release, it can be easily installed on current versions by adding a couple of config files for “APT pinning”.

If you haven’t set up Raspbian 9, just dump the current Raspbian image to SDcard (for the record I used version 2018-03-13-raspbian-stretch-lite)

Boot your Pi.  To keep track of changes in /etc/, let’s install etckeeper:

sudo apt-get update
sudo apt-get install etckeeper

Upgrade current packages:

sudo apt-get upgrade
sudo apt-get dist-upgrade

Declare the current release as default source:

cat<<EOT | sudo tee /etc/apt/apt.conf.d/50raspi
APT::Default-Release "stretch";
EOT

Then add a repo file for the next release:

cat /etc/apt/sources.list | sed 's/stretch/buster/g' | sudo tee /etc/apt/sources.list.d/raspi-buster.list

Unless you want to test the upcoming release, it maybe be safer to avoid upgrading all packages yet.  In other words, we prefer that only iotjs should be available from this “not yet supported” repo.

cat<<EOT | sudo tee /etc/apt/preferences.d/raspi-buster.pref
Package: *
Pin: release n=buster
Pin-Priority: -10
EOT

cat<<EOT | sudo tee /etc/apt/preferences.d/iotjs.pref
Package: iotjs
Pin: release n=buster
Pin-Priority: 1
EOT

Now iotjs 1.0-1 should appear as available for installation:

sudo apt-get update ; apt-cache search iotjs
iotjs - Javascript Framework for Internet of Things

apt-cache policy iotjs
iotjs:
  Installed: (none)
  Candidate: 1.0-1
  Version table:
     1.0-1 1
        -10 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages

Let’s install it:

sudo apt-get install iotjs
man iotjs

Even if version 1.0 is limited in compared to the development branch, you can start by using the http module which is enabled by default (not https).

To illustrate this, when I investigated “air quality monitoring” for a TizenRT+IoT.js demo I found out that OpenWeatherMap is collecting and publishing “Carbon Monoxide” Data, so let’s try their REST API.

Create a file, example.js for example, that contains:

var http = require('http');

var location = '48,-1';
var datetime = 'current';

//TODO: replace with your openweathermap.org personal key
var api_key = 'fb3924bbb699b17137ab177df77c220c';

var options = {
  hostname: 'api.openweathermap.org',
  port: 80,
  path: '/pollution/v1/co/' + location + '/' + datetime + '.json?appid=' + api_key,
};

// workaround bug
options.headers = {
  host: options.hostname
}

http.request(options, function (res) {
  receive(res, function (data) {
    console.log(data);
  });
}).end();

function receive(incoming, callback) {
  var data = '';

  incoming.on('data', function (chunk) {
    data += chunk;
  });

  incoming.on('end', function () {
    callback ? callback(data) : '';
  });
}

And just run it:

iotjs example.js
{"time":"2018-03-27T02:24:33Z","location":{"latitude":47.3509,"longitude":-0.9081},"data":[{"precision":-4.999999987376214e-07,"pressure":1000,"value":1.5543508880000445e-07}
(...)

You can then use this to do things such as update a map or raise an alert on anything useful, or try to rebuild master branch.

How to Build A Simple Connected Light with IoT.JS and Python Django

Many web developers I meet are interested in working with embedded systems and IoT, but they always seem to have reservations on just how to make the whole system (i.e. a server, a ‘thing,’ and a client) work! The amount of information online is extensive, but it’s often hard to know where to start! This blog post will provide a very simple example of how to get a basic LED light to work in a local network, with a web client that provides a way to identify the light with no prior knowledge from the user and no required installation on the client device.

To do this we are going to use a very popular Python Django web framework and the  Samsungs IoT.JS framework. This post will provide an overview, basic code snippets, and links to more information on the GitHub repo. I’ll also provide exact links to the hardware I used for anyone who wants to tinker.

The Problem

There’s an IoT device sitting on the local network, silently exposing functionality to the public. To save money on the bill of materials, there are no screens or buttons, and configuration is all done via a web UI. How does a developer get access to the system and use it? There are several ways to handle this discovery with physical interactions from the user, such as bluetooth, RFID, QR Codes, or even a URL printed on the device. There are also discovery protocols like Bonjour, although even that will not work on many WiFi LAN networks because UDP is commonly blocked at the WiFi hub (as is with our Samsung WiFi).

In this example our device will be registered on the local WiFi network and have a local IP address. What we want is for a developer to be able to find this IP address and get access to the UI of the device. OK, lets start…

The Hardware

For this guide, I’m using the following items:

How to Build A Simple Connected Light with IoT.JS and Python Django - default-light-1-small

In this example, the Raspberry Pi Zero was placed in the 240v AC power in ‘cavity’ as seen in the picture below. Since our new light will not use 240V AC, it provides a convenient place for the Raspberry Pi to sit.

How to Build A Simple Connected Light with IoT.JS and Python Django - light-cavity-small

The LED light strip replaces the LED matrix of the original light. In this example, I’m only using the plastic water resistant housing, all the electronics and control wires have been removed. The fixture was mounted on a piece of plywood to demonstrate the device.

How to Build A Simple Connected Light with IoT.JS and Python Django - light-internall-small

For instructions on how to physically connect your LED light to the raspberry pi go here. The code to control the light is on my personal github repo, the file which controls the server is done in the file server_html.js. You can see line  63 calling the objects method: lightcontrol.showRainbowLight(). This method is exported in the lightconrol.js file which controls the light. For now I will leave details for a future blog, this is all about how we access and control the light and server. How we control the hardware pins of the LED and make that work is for blog 2 in the series.

The Server

The basic functions of the server are to register and update the IP of our IoT Light and to route the user to the correct local IP address.

Register And Update The Light

The server holds the details of the light, and again, there are a number of protocols and standards to pick from! To keep things simple, we created a REST endpoint that allows a light to register; it does this with PUT, POST and DELETE. The key to the server is to use the MAC address as a unique ID and hold the local IP address. In this example, the light will only work locally for the developer, and it’s a valid use case for certain applications to only be accessible to someone within the local network. The basic sequence diagrams looks like this:

How to Build A Simple Connected Light with IoT.JS and Python Django - LightRegistration

To ease the writing of REST endpoints, I used the very popular Django Rest Framework. Hence, the light needs to register itself and update its local IP address. It uses the MAC address as a unique value, however you are free to invent your own solution here. I created a very rich object model in my server in an attempt to future proof the DB object as best as possible, but the controller for registering the light is surprisingly simple and compact, and most of the lines are documentation.

def iot_machines_register(request):                     # TODO Add some type of authentication for iot devices
    """
    :param request: JSON {
                    device_id:      mac_adress e.g. 98:83:89:3a:96:a5
                    device_name:    "any text string"
                    local_ip:       IPV4 or IPV6 e.g. 192.168.0.1
                    }

    :return: HttpResponseForbidden, HttpResponse, HttpResponseRedirect

    Register an IoT Machine based on it's MAC address.
    If this is a POST we check it's unique. Verify the JSON package. And register the machine

    If this is a PUT we check it exists. verify the JSON package. And update the machine.

    TODO - Authentication & Authorization!

    """

    if request.method == 'POST':
        data = JSONParser().parse(request)
        print("We got the following data in the request: {}".format(data))
        serializer = IoTMachineSerializer(data=data)
        if serializer.is_valid():
            serializer.save()
            return JsonResponse(serializer.data, status=201)
        return JsonResponse(serializer.errors, status=400)

The function checks the method with request.method == ‘POST’ and creates a serializer with serializer=IoTMachineSerializer(data=data) to parse the date from the POST message. Validation is done in the model, which makes things far simpler to code. If validation succeeds, the ORM saves the new object in the SQL database with serializer.save() and returns a HTTP 201 with return JsonResponse(serializer.data, status=201). If it fails, the model generates the correct error codes and the framework responds with the appropriate error message with return JsonResponse( serializer.errors, status=400). Details of this are on the GitHub repo.

Routing The User To The Local IP

The clever part of this system must allow the user to find the IP of the light. After doing this, all communications will be handled locally between the device on the local WiFi network and the IoT Light.

How to Build A Simple Connected Light with IoT.JS and Python Django - LightRouting

The sequence diagram above shows the flow of a user hitting the server that contains the local IP and sending a redirect. After this happens, the server is no longer part of the data flow for this user. All the clever control and data flow is between the local user and the IoT Device.

def iot_machine_find(request, mac_address):
    """
    Find an IoT Machine based on it's MAC address.
    If it has a public local IP address then redirect to this IP.
    If our embedded system does not have a valid IP address then don't redirect and show the user a
    friendly page saying the machine is not active at the moment.

    """
    print("Trying to find an IoT Machine and redirect locally with mac_address of: {}".format(mac_address))

    # Make sure we can find the machine via it's mac address
    try:
        iot_device = IoTMachine.objects.get(device_id=mac_address)
    except IoTMachine.DoesNotExist:
        return HttpResponse(status=404)

    # Make sure the IP address we pass back is still valid
    try:
        ipaddress.IPv4Address(iot_device.local_ip)
    except ValueError:
        # TODO Paint a nice screen and show the user the device exists but routing is broken
        print("There was an issue with the stored IP address for the device: {} - handle it an carry on".format(iot_device.mac_address))
        return Http404

    # Happy path - direct the user to the device
    if request.method == 'GET':

        scheme = 'http'                 # TODO make this a dynamic variable from settings or DB table
        path = 'machine'                # TODO make this a dynamic variable from settings or DB table
        remote_url = "{0}://{1}/{2}".format(scheme,iot_device.local_ip,path)
        print("We found the device returning the IOT Local address: {}".format(remote_url))
        return HttpResponseRedirect(remote_url)

    # If we get this far - we don't support other methods at the minute so reply with a forbidden.
    return HttpResponseForbidden

The first and second try/except clauses simply check that the MAC address exists, and if it does, that there is a valid IP address to send back. The happy path checks for the GET and PUT method, and a GET returns the local IP address in a redirect with return HttpResponseRedirect(remote_url). The URL is hard coded in this example, however production systems would dynamically take these values from either configuration files or from data the IoT device provides.

All other HTTP methods will be caught with the return HttpResponseForbidden method. Again, much of the heavy lifting is done with the Django ORM and REST framework.

It’s important to note how flat the structure is; Pythonic code tends to move away from multiple layers of abstraction, opting for structures that are as flat as possible. This only shows the logic the server is exercising, and the framework comes, as they say, with ‘batteries included’. To get to the Django administration interface as a registered admin user, you just login and hit the admin path. It will then be possible to view the active DB, registered IoT machines, and even manipulate any values of those machines. No additional code is required for this.

How to Build A Simple Connected Light with IoT.JS and Python Django - django-screen1-small

By selecting a single IoT device, the framework will pull all relevant data from the SQL DB and format and display it in the Django admin form. All of this is generated automatically:

How to Build A Simple Connected Light with IoT.JS and Python Django - django-screen2-small

I haven’t gone into detail about how the Django ORM works in this article. If you want to learn more, it’s best to visit the Python Django experts.

How Did The Client Find The Server?

At this point you might be thinking: wait a minute, not only do I still need to know about the public server, but I also need to know what the light ID is! Remember the light as no physical buttons or screen! In this example we used a QR code that sits next to our light. The user scans the code with the phone which routes them to the server using it’s own MAC address as the ID. You can try this yourself with your Samsung browser on your phone – scan this QR code below.

How to Build A Simple Connected Light with IoT.JS and Python Django - qrcode_b827eb01d83f-small

You will be routed to my test server at www.noisyatom.tech/iot/machine/b8:27:3b:01:d8:3f, which will forward you to the UI the light exposes. (Un)fortunately, it only works if you are here in the same WiFi network. :-) Your browser will try to connect you to http://192.168.110.99/light.

Incidentally you can try this from your Samsung browser by selecting the ‘Scan QR code’ from the top right selection menu, which should show you a screen like below:

How to Build A Simple Connected Light with IoT.JS and Python Django - Screenshot_qr-small

Further Exploration

There is a lot of information here, and I’ve not really explained how the model is created or how to tell the server what parameters of the model are important! I will follow up on this in future blog posts where I’ll look at the details of the code running on the light. The light acts like both a server and client: it’s a server to the device that wants access to functionality, and a client of the central server that holds details about the device. Once the light is activated, it goes through a very nice looking rainbow dance; you could imagine this being used as a mood light or as a device that interacts with other systems. Finally, I will revisit the server side code later and explain how we make the model persist to a DB and have only the server check and verify components of that model with the REST interface in a future blog.

Check out the light in action!

How to Build A Simple Connected Light with IoT.JS and Python Django - light-rainbow-small

 

The Business Obligations of Open Source Software

This article is a part of a series on using open source in business:

The previous two articles in this series covered the technical and business benefits OSS offers. However, this only paints half the picture. While OSS might be free to use, modify, and distribute, it doesn’t come without effort and risk; if a company isn’t prepared to handle them, they can cause significant headaches down the road. This article will provide an overview of the obligations and risks a company that uses OSS in their products or services must be aware of.

Licensing and Compliance

While OSS has no initial licensing costs, this doesn’t always mean you’re free to use OSS however you want. Open source licenses can impose a range of obligations that must be satisfied whenever code that includes OSS is distributed in a product or service. This can include things like disclosure requirements, specific notifications, and licensing modifications under the same open source license.

At the very least, all businesses need to maintain an accurate inventory of all OSS that’s used in their products or services, and must have adequate processes to ensure compliance with all licenses both during and after product distribution. Typically, a single person is assigned the responsibility of being the open source compliance manager to oversee the execution of this, and it requires coordination between legal, engineering, human resources, and the compliance officer; larger companies often need to establish an Open Source Review Board to facilitate this coordination.

The most critical coordination is between product, engineering, and management teams: they must identify all OSS used in product code and to assist with any licensing or compliance questions. This often means employee roles need to be expanded to cover the various elements of OSS compliance that directly relate to each position. This coordination must also expand outside the company to include software providers as well because any OSS that’s included in the software they provide must also be properly accounted for and complied with. While establishing a framework for this coordination, it’s common to come across areas that need better tooling which creates more need for deployment and training.

Open source licensing can create substantial obligations, especially in larger organizations, and these obligations require more coordination between various teams. This article only briefly touches on the subject of open source compliance, and if you’re interested in this subject more there is a lot of detailed information on the web.

Training and Development

Training is a critical component of open source engineering because you’ll need to train developers, legal, software procurement, quality assurance, systems administrators, and others on the obligations of open source compliance. Additionally, you might need training for developers that have little to no experience working with an open source community on how open source development works.

Maintenance and Support

Proprietary software typically includes some level of paid support from the vendor company, which may include an SLA, pre-disclosure of vulnerabilities, or implementation support.  This is generally contractually enforceable, and it’s reasonable to demand responsiveness from the vendor in line with your support contract.  Similar support for OSS projects may be offered by companies participating in the ecosystem, but it is rare to find unpaid guaranteed support from the development community itself.  As most of the participants are either working on their own time or on behalf of an employer, it is neither reasonable nor fair to expect the same level of service as provided by a paid vendor.  As such, it may be the case where businesses that incorporate unpaid OSS into their products are responsible for installing updates and implementing security fixes and new modules themselves.  These tasks are easy to push aside in favor of more pressing issues, but they must be addressed nonetheless because a lack of maintenance can rapidly overcome the benefits of incorporating externally developed code.

To reiterate, OSS does not include any sort of support contract unless it is specifically negotiated with a third-party company that offers these services.  The most common OSS licenses specifically include a disclaimer of warranty.  If a business requires 24/7 support for services that use OSS, this will either need to be negotiated with an appropriate third-party company or the team will need to be hired in-house. Support services are a critical, but often overlooked, investment when using OSS.

Application Dependence

OSS can be a great avenue to avoid proprietary software dependence, but that doesn’t mean it’s a simple process to migrate away from OSS that no longer meets the requirements of a company. Whenever OSS is used for business-critical applications, the business becomes dependent on the community and project.  Changes that go against the interests of the business could result in the company needing to migrate to an alternative. The effort required to make this switch can vary widely depending on the availability of expertise and the maturity of the projects being migrated between. OSS community support can mitigate this problem, but this also varies substantially from project to project.  In general, it is considered a best practice to be actively engaged with and support select OSS communities to reduce the risk of unexpected changes to critical components.

Strategic Dependence

Open source leadership can bring substantial benefits to companies that properly leverage it, but the independent nature of open source communities can complicate strategic decisions. For starters, open source community choices may preclude downstream requirements, particularly if a company does a poor job of communicating with the upstream community.  This can introduce added effort to maintain downstream products. Additionally, if the community doesn’t have a formal technical roadmap, planning can become much more complicated. Finally, all software whether open or proprietary fits onto a spectrum of maturity, and this can often reflect the needs of a community.  For example, software developed for home use or for academic research may require some level of investment to meet product security requirements.

Do the Benefits of Open Source Outweigh the Obligations?

This article should have provided you with a general understanding of what obligations and risks are associated with the use of OSS in a product or service. There is no one-size-fits-all solution for whether a company should use OSS or simply buy software from a proprietary vendor, so it’s always important for any business that’s considering the use of OSS to consider their own specific requirements.  However, the overwhelming trend in the industry has been in favor of OSS components wherever possible, particularly in consumer electronics.

Here are some questions all businesses should ask themselves when considering the use of OSS in products.

  • How do the platform costs of OSS compare to similar offerings?
  • What commercial support offerings are available?
  • What license obligations does the code have? Does our company have the resources to fill them?
  • Who are the key individuals and organizations in the open source community? How long have they been around?
  • Do the open source community goals and roadmap meet our risk profile?
  • Does our company have adequate resources to support the use of OSS in our products?

 

Let’s Talk Artificial Intelligence

There has been a lot of buzz around Artificial Intelligence as Bixby, Alexa, and Siri become household names. Today, self driving cars share the roads with human drivers, hog farms keep an “artificial eye” on the vitals of individual animals in the pens, and soon enough we’ll be able to monitor heart health by staring into your eyes as a substitute to being hooked up to an EKG machine.

I wanted to know how this is happening and learn about the brain behind all of these advancements, so I recently set out to understand and learn the landscape of various Linux AI projects including NuPIC, Caffe and TensorFlow.  This post will cover some of the important things I learned about each of these platforms.

NuPIC

NuPIC stands for Numenta Platform for Intelligent Computing, and I found the theory behind it to be fascinating. It’s an implementation of Hierarchical Temporal Memory (HTM), a theory of intelligence that’s inspired by and based on the neuroscience of the neocortex, and it’s well suited for finding anomalies in live data streams. The algorithms and framework are implemented in Python,  JavaScript, C++, Shell, and Java.

I picked nupic and nupic-core (implementation of core NuPIC algorithms in C++) to play with on x86_64 (Ubuntu 17.10) and odroid-xu4 (Ubuntu 16.04) platforms. I had good luck compiling nupic and nupic-core from sources on both platforms. I was able to run unit_tests on x86_64, however, on odroid_xu4 I couldn’t run it due to an older version of pip. NuPIC requires pip version 9.0.1, so I’ll have to come back another time to build pip 9.01 from sources.

Here is a short summary of the steps to install NuPIC and run tests.

git clone https://github.com/numenta/nupic.core.git

cd nupic.core/
export NUPIC_CORE=$PWD
mkdir -p $NUPIC_CORE/build/scripts
cd ../nupic.core/build/scripts/
cmake $NUPIC_CORE -DCMAKE_BUILD_TYPE=Release \
   -DCMAKE_INSTALL_PREFIX=../release \
-DPY_EXTENSIONS_DIR=$NUPIC_CORE/bindings/py/src/nupic/bindings

# While still in $NUPIC_CORE/build/scripts
make -j3

# While still in $NUPIC_CORE/build/scripts
make install

# Validate install running cpp_region_test and unit_tests
cd $NUPIC_CORE/build/release/bin
./cpp_region_test
./unit_tests

#Note:
#This step worked on x86_64 as it has pip 9.0.1 and failed on
# odroid-xu4 due to older pip 8.1.1 ./unit_tests
git clone https://github.com/numenta/nupic
cd nupic

#Follow the detailed instructions in README.md to compile and install

# Install from local source code
sudo pip install -e .

sudo apt-get install python-pytest

Note:
This step worked on x86_64 as it has pip 9.0.1 and failed on
odroid-xu4 due to older pip 8.1.1 py.test tests/unit

Caffe

Caffe is Deep learning framework from Berkeley AI Research (BAIR) and The Berkeley Vision and Learning Center (BVLC). Tutorials and installation guides can be found in the project’s README.

I compiled Caffe on the odroid-xu4 successfully and was able to run tests. I had to install several dependencies for “make all” to complete successfully, and at the end I was able to run “make runtest” which ran 1110 tests from 152 test cases. The experience is relatively painless!

Here is a short summary of the install steps for installing and running tests.

https://github.com/BVLC/caffe.git
cd caffe
# Created Makefile.config from Makefile.config.example
# Uncomment CPU_ONLY=1 for Caffe cpu version

# Install dependenices
sudo apt-get install protobuf-compiler libboost-all-dev
sudo apt-get install libgoogle-glog-dev
sudo apt-get install libopenblas-dev
sudo apt-get install libopencv-highgui-dev
sudo apt-get install libhdf5-dev
export CPATH="/usr/include/hdf5/serial/"
sudo apt-get install libleveldb-dev
sudo apt-get install liblmdb-dev
sudo apt-get install libatlas-base-dev
sudo apt-get install libgflags-dev

make all
make test
make runtest
[==========] 1110 tests from 152 test cases ran. (197792 ms total)
[  PASSED  ] 1110 tests.

The Caffe command can be found under .build_release/tools/caffe.

caffe: command line brew
usage: caffe <command> <args>

commands:
  train           train or finetune a model
  test            score a model
  device_query    show GPU diagnostic information
  time            benchmark model execution time

  Flags from tools/caffe.cpp:
    -gpu (Optional; run in GPU mode on given device IDs separated by ','.Use
      '-gpu all' to run on all available GPUs. The effective training batch
      size is multiplied by the number of devices.) type: string default: ""
    -iterations (The number of iterations to run.) type: int32 default: 50
    -level (Optional; network level.) type: int32 default: 0
    -model (The model definition protocol buffer text file.) type: string
      default: ""
    -phase (Optional; network phase (TRAIN or TEST). Only used for 'time'.)
      type: string default: ""
    -sighup_effect (Optional; action to take when a SIGHUP signal is received:
      snapshot, stop or none.) type: string default: "snapshot"
    -sigint_effect (Optional; action to take when a SIGINT signal is received:
      snapshot, stop or none.) type: string default: "stop"
    -snapshot (Optional; the snapshot solver state to resume training.)
      type: string default: ""
    -solver (The solver definition protocol buffer text file.) type: string
      default: ""
    -stage (Optional; network stages (not to be confused with phase), separated
      by ','.) type: string default: ""
    -weights (Optional; the pretrained weights to initialize finetuning,
      separated by ','. Cannot be set simultaneously with snapshot.)
      type: string default: ""

TensorFlow

Now, on to TensorFlow: a machine learning framework. It provides a C API defined in c_api.h, which is suitable for building bindings for other languages; I played with TensorFlow and the C API, and I compiled TensorFlow and Bazel, a pre-requisite, from sources. Compiling from sources took a long time on the odroid-xu4 and ran into the same pip version issue I mentioned earlier. On the x86_64 platform, I managed to write a hello_tf.c using the C API and watch it run.

The Tensorflow website has a great guide that explains how to prepare your Linux environment. There are two choices for installing Bazel, you can either install Bazel binaries, or compile from sources. Here are my setup notes for compiling Bazel from sources:

sudo apt-get install build-essential openjdk-8-jdk python zip
sudo apt-get install python-numpy python-dev python-pip python-wheel

mkdir bazel; cd bazel

wget https://github.com/bazelbuild/bazel/releases/download/0.10.0/bazel-0.10.0-dist.zip

unzip bazel-0.10.0-dist.zip

./compile.sh

# The above failed with "The system is out of resources."
# message. Workaround reference:
# https://github.com/bazelbuild/bazel/issues/1308

vi scripts/bootstrap/compile.sh

# find the line:
"run "${JAVAC}" -classpath "${classpath}" 
 -sourcepath "${sourcepath}""
add -J-Xms256m -J-Xmx384m
as shown in 
"run "${JAVAC}" -J-Xms256m -J-Xmx384m -classpath
 "${classpath}" -sourcepath "${sourcepath}""

# Run compile again: Compile should work now.
./compile.sh

sudo pip install six numpy wheel

cd tensorflow
export PATH=$PATH:../linux_ai/bazel/output
./configure

# The bazel build command builds a script named build_pip_package. Running this script as follows will build a .whl file within the /tmp/tensorflow_pkg directory:

bazel build --jobs 1 --local_resources 2048,0.5,1.0 --verbose_failures --config=opt //tensorflow/tools/pip_package:build_pip_package

bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
sudo pip install /tmp/tensorflow_pkg/tensorflow-1.6.0rc0-cp27-cp27mu-linux_armv7l.whl

You can validate your TesnforFlow installation by running python in a different directory and executing the following code:

python

Python 2.7.14 (default, Sep 23 2017, 22:06:14) 
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow!')
>>> sess = tf.Session()
>>> print(sess.run(hello))

#If you see "Hello, TensorFlow!" as a result, you are good to start
#developing TensorFlow programs.

For added fun, I also installed the TensorFlow C API, here is the sample program:

#include <stdio.h>
#include <tensorflow/c/c_api.h>

int main()
{
	printf("Hello from TensorFlow C library version %s\n", TF_Version());
	return 0;
}
gcc -I/usr/local/include -L/usr/local/lib hello_tf.c -ltensorflow
./hello_tf 
Hello from TensorFlow C library version 1.5.0

Summary

My experience has given me a few tips and takeaways I think are worth sharing:

  • Most AI frameworks depend on new tool chain versions, so it’s important to ensure you have latest versions of things like pip and python.
  • Compiling TensorFlow can be very tedious.
  • Caffe installed easily on odroid-xu4.
  • Unlike TesnsorFlow Java and TensorFlow Go, TensorFlow for C appears to be covered by the TensorFlow API stability guarantees.

This blog should have provided you with a quick summary of the select AI projects I experimented with, and I hope some of my notes might be useful to you. My work in this exciting area continues, and I will share more new discoveries in the future. If you’re working on some interesting AI projects, we’d love to hear about them in the comments below.

An Introduction to IoT.js Architecture

IoT.js is a lightweight JavaScript platform for the Internet of Things. The platform keeps interoperable services at the forefront, and is designed to bring the success of Node.js to IoT devices like micro-controllers and other devices that are constrained to limited storage and only a few kilobytes of RAM. IoT.js is built on top of JerryScript: a lightweight JavaScript interpreter, and libtuv: an event driven (non-blocking I/O model) library. The project is open source under the Apache 2.0 license.

This article will introduce you to the architecture of IoT.js and the fundamentals of writing applications for it.

IoT.js Architecture

An Introduction to IoT.js Architecture - iotjs_arch

JerryScript – ECMAScript binding

JerryScript is the kernel for IoT.js on an ECMAScript binding, it’s an ultra lightweight JavaScript engine that was written from scratch at Samsung. The name “Jerry” comes from the popular character in Tom and Jerry, who’s small, smart, and fast! Since the engine is an interpreter only, it might be more precise to say JerryScript is a JavaScript interpreter.

Optimizations for memory footprint and performance have always been the top priorities of the project. The tiny engine has a base RAM footprint of less than 64KB, and the binary can accommodate less than 200KB of ROM. Amazingly, it implements the full ECMAScript 5.1 standard, and work has been ongoing recently to introduce new ES6 features such as promise, TypedArray, and more.

Like IoT.js, JerryScript was also released under the Apache 2.0 license. The community has experienced very rapid growth, especially in last couple years, and in 2016 the project was transferred to the JavaScript Foundation. A recent JavaScript Foundation press release mentioned how JerryScript was adopted in Fitbit’s latest product: Ionic.

JerryScript provides a good set of embedding APIs to compile and execute JavaScript programs, access JavaScript objects and their values, handle errors, manage the lifestyles of objects, and more. IoT.js uses these API’s to create the builtin module and native handler in IoT.js native core.

libtuv – I/O Event binding

Asynchronous I/O and threading in IoT.js are handled with libtuv: a library that focuses on asynchronous I/O and was primarily developed for use with Node.js. Samsung launched this open source project under the Apache 2.0 license, and it’s a multi-platform tiny event library that’s refactored from libuv to better serve IoT.js and on embedded systems.

Libtuv’s features include a loop, timer, poll, tcp & udp, fs event, thread, worker, and more. The platforms this library supports include i686-linux, arm-linux, arm-nuttx and arm-mbed.

IoT Binding

In the IoT.js community, there have been discussions about binding to an existing IoT platform or specification, such as IoTivity: The Open Connectivity Foundation‘s open source project. If this were to happen it would certainly add more dimension to supporting interoperability with other platforms.

IoT.js C Core

The IoT.js core layer is located above the binding layer, and it provides upper layer functionality to interact with the JavaScript engine, including running main event loops, managing I/O resources, etc. It also provides a set of builtin modules and native handlers.

Builtin modules are the basic and extended modules that are included in the IoT.js binary. Basically, these builtin modules are JavaScript objects implemented in C using the embedding API JerryScript provides, in either JavaScript or both languages. The native components of builtin modules are implemented as a native handle to access underlying systems via event handling, a C library, or system calls.

The life cycle of IoT.js is shown below:
 
An Introduction to IoT.js Architecture - iotjs_lifecycle-1

IoT.js ECMAScript API and JavaScript Modules

Like Node.js, IoT.js is a module-based system. Each module in IoT.js has its own context and provides a set of API’s associated with the module’s functionality.

IoT.js offers basic API modules and extended API modules. The basic API modules are based on Node.js and follow same form for compatibility reasons. Basic API modules include File System, Net, HTTP, Process, etc. Pretty much all application code that calls these API’s can be run in a Node.js environment without any modification.

The extended modules, on the other hand, are more IoT.js specific, and they are currently mostly hardware related (e.g. GPIO, Bluetooth Low Energy (BLE), I2C, SPI, UART, PWM, etc.). Many contributors are interested in adding new extended API modules to support their own specific hardware, so to maintain consistent usability, the IoT.js community has set guidelines and rules for introducing extended API’s.

Enabling JavaScript on the Internet of Things

The overall architecture of IoT.js is very friendly to Node.js, as a result of the asynchronous I/O and threading library, and the subset of Node.js compatible modules; it has reflected the design philosophy of providing a lightweight version of Node.js along with an inter-operable service platform. IoT.js has opened a great opportunity for JavaScript developers to develop applications for the Internet of Things, so it’s definitely an IoT platform to watch!

Now Hiring: Open Source Cloud Engineers

The cloud is ripe with open source software as the collaborative development model has proven to be extremely effective at commoditizing much of the software stack. Today, foundations like the Cloud Native Computing Foundation are bringing together numerous companies to collaborate on vital, cutting edge technology, and open source software is doing what it has done in many other industries as it positions itself to be critical to the adoption of cloud technologies. More companies are launching open source initiatives to improve this technology domain and the Open Source Group recognizes this as a valuable place for us to increase our breadth of impact on Samsung’s innovation.

We’re Hiring Cloud Experts!

With this said, we’re pleased to announce we’re hiring multiple positions for experts in open source cloud technologies to help us in our mission to improve Samsung’s understanding and adoption of open source software. Here are the most important skills we’re looking for:

  • Programming experience in one or more languages, including C++, Go, Python, Shell, C#, Ruby, JavaScript, and PHP. Please note that the language competency depends on the project you’re expected to get involved in.
  • Expertise contributing to one or more of these projects: Kubernetes, OpenTracing, Jaeger, Envoy, Linkerd, gRPC, Fluentd, and related CNCF projects.
  • Knowledge of distributed tracing, monitoring, service management, and container orchestration.
  • Demonstrable experience participating in open source communities as a contributor, committer, or maintainer.
  • A history of providing technical leadership within open source communities and engaging in political influence and guidance.
  • A high degree of self-motivation and the ability to work alone, managing your own work and setting sensible priorities.
  • Background in pipeline operations, packaging and deployment of cloud applications, virtualization technologies and API’s,

However, don’t worry if you aren’t experienced with all of these skills; the most important skill you can have is the ability to expand your own technical knowledge to meet the rapidly changing open source software ecosystem. If you have knowledge of any of these items with a history of good open source contributions, we want you!

Why Work for Us?

You’ll want to work for us if you’re interested in:

  • Spending 100% of your development time on upstream open source projects,
  • Working from almost anywhere in the world (as long as you have an internet connection),
  • Using your preferred infrastructure for your development environment,
  • The freedom to work outside the pressure of product deadlines, and
  • Improving your visibility with open source communities via participation in project governance and events.

If you’re someone who’s passionate about open source software and deeply knowledgeable about the things mentioned in this post, we’d love to hear from you. If you’re interested, send us a message on twitter @SamsungOSG or an email to hiring[at]osg.samsung.com. If you want to learn more about our team, check out this blog post that explains why our team was created.

We look forward to meeting some great open source engineers!

How to Run IoT.js on the Raspberry PI 0

IoT.js is a lightweight JavaScript platform for building Internet of Things devices; this article will show you how to run it on a few dollars worth of hardware. The First version of it was released last year for various platforms including Linux, Tizen, and NuttX (the base of Tizen:RT). The Raspberry Pi 2 is one of the reference targets, but for demo purposes we also tried to build for the Raspberry Pi Zero, which is the most limited and cheapest device of the family. The main difference is the CPU architecture, which is ARMv6 (like the Pi 1), while the Pi 2 is ARMv7, and the Pi 3 is ARMv8 (aka ARM64).

IoT.js upstream uses a python helper script to crossbuild for supported devices, but instead of adding support to new device I tried to build on the device using native tools with cmake and the default compiler options; it simply worked! While working on this, I decided to package iotjs for debian to see how well it will support other architectures (MIPS, PPC, etc), we will see.

Unfortunately, Debian armel isn’t optimized for ARMv6 and FPU, both of which are present on the Pi 1 and Pi 0, so the Raspbian project had to rebuild Debian for the ARMv6+VFP2 ARM variant to support all Raspberry Pi SBC’s.

In this article, I’ll share hints for running IoT.js on Raspbian: the OS officially supported by the Raspberry foundation; the following instructions will work on any Pi device since a portability strategy was preferred over optimization. I’ll demonstrate three separate ways to do this: from packages, by building on the device, and by building in a virtual machine. By the way, an alternative to consider is to rebuild Tizen Yocto for  the Pi 0, but I’ll leave that as an exercise for the reader, you can accomplish this with a bitbake recipe, or you can ask for more hints in the comments section.

How to Run IoT.js on the Raspberry PI 0 - tizen-pizero

Install from Packages

iotjs landed in Debian’s sid, and until it is in testing branch (and subsequently Raspbian and Ubuntu), the fastest way is to download it is via precompiled packages from my personal Raspbian repo

url='https://dl.bintray.com/rzr/raspbian-9-armhf'
source="/etc/apt/sources.list.d/bintray-rzr-raspbian-9-armhf.list"
echo "deb $url raspbian main" | sudo tee "$source"
sudo apt-get update
apt-cache search iotjs
sudo apt-get install iotjs
/usr/bin/iotjs
Usage: iotjs [options] {script | script.js} [arguments]

Use it

Usage is pretty straightforward, start with a hello world source:

echo 'console.log("Hello IoT.js !");' > example.js
iotjs  example.js 
Hello IoT.js !

More details about the current environment can be used (this is for iotjs-1.0 with the default built-in modules):

echo 'console.log(JSON.stringify(process));' > example.js
iotjs  example.js 
{"env":{"HOME":"/home/user","IOTJS_PATH":"","IOTJS_ENV":""},"native_sources":{"assert":true,"buffer":true,"console":true,"constants":true,"dns":true,"events":true,"fs":true,"http":true,"http_client":true,"http_common":true,"http_incoming":true,"http_outgoing":true,"http_server":true,"iotjs":true,"module":true,"net":true,"stream":true,"stream_duplex":true,"stream_readable":true,"stream_writable":true,"testdriver":true,"timers":true,"util":true},"platform":"linux","arch":"arm","iotjs":{"board":"\"unknown\""},"argv":["iotjs","example.js"],"_events":{},"exitCode":0,"_exiting":false} null 2

From here, you can look to use other built-in modules like http, fs, net, timer, etc.

Need More Features?

More modules can be enabled in the master branch, so I also built snapshot packages that can be installed to enable more key features like GPIO, I2C, and more. For your convenience, the snapshot package can be installed to replace the latest release:

root@raspberrypi:/home/user$ apt-get remove iotjs iotjs-dev iotjs-dbgsym iotjs-snapshot
root@raspberrypi:/home/user$ aptitude install iotjs-snapshot
The following NEW packages will be installed:
  iotjs-snapshot{b}
(...)
The following packages have unmet dependencies:
 iotjs-snapshot : Depends: iotjs (= 0.0~1.0+373+gda75913-0~rzr1) but it is not going to be installed
The following actions will resolve these dependencies:
     Keep the following packages at their current version:
1)     iotjs-snapshot [Not Installed]                     
Accept this solution? [Y/n/q/?] n
The following actions will resolve these dependencies:

     Install the following packages:                 
1)     iotjs [0.0~1.0+373+gda75913-0~rzr1 (raspbian)]
Accept this solution? [Y/n/q/?] y
The following NEW packages will be installed:
  iotjs{a} iotjs-snapshot 
(...)
Do you want to continue? [Y/n/?] y
(...)
  iotjs-snapshot https://dl.bintray.com/rzr/raspbian-9-armhf/iotjs-snapshot_0.0~1.0+373+gda75913-0~rzr1_armhf.deb
  iotjs https://dl.bintray.com/rzr/raspbian-9-armhf/iotjs_0.0~1.0+373+gda75913-0~rzr1_armhf.deb

Do you want to ignore this warning and proceed anyway?
To continue, enter "yes"; to abort, enter "no": yes
Get: 1 https://dl.bintray.com/rzr/raspbian-9-armhf raspbian/main armhf iotjs armhf 0.0~1.0+373+gda75913-0~rzr1 [199 kB]
Get: 2 https://dl.bintray.com/rzr/raspbian-9-armhf raspbian/main armhf iotjs-snapshot armhf 0.0~1.0+373+gda75913-0~rzr1 [4344 B]
(...)

If you the run console.log(process) again, you’ll see more interesting modules to use, like gpio, i2c, uart and more, and external modules can be also used; check on the work in progress for sharing modules to the IoT.js community. Of course, this can be reverted to the latest release by simply installing the iotjs package because it has higher priority than the snapshot version.

root@raspberrypi:/home/user$ apt-get install iotjs
(...)
The following packages will be REMOVED:
  iotjs-snapshot
The following packages will be upgraded:
  iotjs
(...)
Do you want to continue? [Y/n] y
(...)

Build on the Device

It’s also possible to build the snapshot package from source with extra packaging patches, found in the community branch of IoT.js (which can be rebased on upstream anytime).

sudo apt-get install git time sudo
git clone https://github.com/tizenteam/iotjs
cd iotjs
./debian/rules
sudo debi

On the Pi 0, it took less than 30 minutes over NFS for this to finish. If you want to learn more you can follow similar instructions for building IoTivity on ARTIK;
it might be slower, but it will extend life span of your SD Cards.

Build on a Virtual Machine

A faster alternative that’s somewhere between building on the device and setting up a cross build environment (which always has a risk of inconsistencies) is to rebuild IoT.js with QEMU, Docker, and binfmt.

First install docker (I used 17.05.0-ce and 1.13.1-0ubuntu6), then install the remaining tools:

sudo apt-get install qemu qemu-user-static binfmt-support time
sudo update-binfmts --enable qemu-arm

docker build 'http://github.com/tizenteam/iotjs.git'

It’s much faster this way, and took me less than five minutes. The files are inside the container, so they need to be copied back to host. I made a helper script for setup and to get the deb packages ready to be deployed on the device (sudo dpkg -i *.deb) :

curl -sL https://rawgit.com/tizenteam/iotjs/master/run.sh | bash -x -
./tmp/out/iotjs/iotjs-dbgsym_0.0-0_armhf.deb
./tmp/out/iotjs/iotjs-dev_0.0-0_armhf.deb
./tmp/out/iotjs/iotjs_0.0-0_armhf.deb

I used the Resin/rpi-raspbian docker image (thanks again Resin!). Finally, I want to also thank my coworker Stefan Schmidt for the idea after he setup a similar trick for EFL’s CI.

Further Reading

If you want to learn more, here are some additional resources to take your understanding further.

Raspberry Pi is a trademark of the Raspberry Pi Foundation\