More Webthings on IoT.js, MCUs, Tizen…

Connect webthings devices to gateway

So called “Webthings” (understand “servers”) are implementing WebThing API (this relates to W3C Web of Things (WoT) Thing Description).
Various languages can be used for implementation, today “Things Framework” page is listing NodeJS, Python, Java, Rust and ESP8266 (+ESP32).

Today’s challenge is to try IoT.js (the Internet of Things framework using JavaScript), as alternate runtime to NodeJS (on v8) and thus gain performance.

The consequence for application developers, is without adding complexity they can now also target more constrained devices using JavaScript high level language.

Make Webthings powered by IoTjs

My latest effort was to port webthing-node to IoT.js, this has been done by removing a couple of features (Actions, Events, Websockets, mDNS), then rewrite some JS parts that used latest ECMA features not supported by Jerryscript the underlying JavaScript interpreter, Also to preserve code structure I also reimplemented parts of express.js on IoT.js’ HTTP module.

This article will explain how to replicate this from scratch using Edison board running community Debian port., but it should be easy to adapt to other configurations. For GNU/Linux based OS this should flawlessly work too (as there is nothing specific here only GPIO pin for demo) while platforms supported by IoT.js should be possible too (Tizen:RT supported by ARTIK05x or NuttX as supported by STM32F4).

Setup Edison

Since IoT.js landed in Debian, I wanted to test it on Edison board running community maintained Debian port: jubilinux v9.

Note if you’re using (outdated) Poky/Yocto reference OS, it’s easy to rebuild on device too (just install libc6-staticdev).

Anyway here is the procedure to flash distro to Edison:

cd jubilinux-stretch
# Unplug edison
time sudo bash flashall.sh
# Plug USB cables to edison
Using U-Boot target: edison-blankcdc
Now waiting for dfu device 8087:0a99
Please plug and reboot the board

Flashing IFWI
(...)

(While I am on this, here is my bottle in the ocean, do you know How to unbrick a unflashable Intel Edison ?)

Then log in and configure system:

# Connect to terminal
picocom  -b 115200 /dev/ttyUSB0
U-Boot 2014.04 (Feb 09 2015 - 15:40:31)
(...)
Jubilinux Stretch (Debian 9) jubilinux ttyMFD2
rootlinux login: 
# Log in as root:edison and change password
root@jubilinux:~# passwd
root@jubilinux:~# cat /etc/os-release 
PRETTY_NAME="Jubilinux Stretch (Debian 9)"
NAME="Jubilinux Stretch"
VERSION_ID="9+0.4"
VERSION="9+0.4 (stretch jubilinux)"
root@jubilinux:~# cat /proc/version 
Linux version 3.10.98-jubilinux-edison (robin@robin-i7) (gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3) ) #3 SMP PREEMPT Sun Aug 13 04:22:45 EDT 2017

Setup Wifi, the quick and dirty way:

ssid='Private' # TODO: Update with your network credentials
password='password' # TODO:
sed -e "s|.*wpa-ssid.*|wpa-ssid \"$ssid\"|g" -b -i /etc/network/interfaces
sed -e "s|.*wpa-psk.*|wpa-psk \"$password\"|g" -b -i /etc/network/interfaces

Optionally, we can change device’s hostname to custom one (iotjs or any name), it will be easier to find later:

sed -e 's|jubilinux|iotjs|g' -i /etc/hosts /etc/hostname

System should ready to use after rebooting:

reboot
apt-get update ; apt-get install screen ; screen

Then you should be ready to install precompiled IoT.js 1.0 package using apt-pining as explained before:

IoT.js landed in Raspbian

This will work, but for demo purpose, you can skip this and rebuild latest snapshot with GPIO module (not supported in default profile of version 1.0).

Rebuild IoT.js snapshot package

Based on Debian iotjs’s packaging file, you can rebuild a snapshot package from my development branch very easily:

apt-get update ; apt-get install git make time
branch=sandbox/rzr/devel/master
git clone https://github.com/tizenteam/iotjs -b "$branch" --recursive --depth 1
cd iotjs
time ./debian/rules && sudo debi

It took only 15 min to build on device, if short of resources you can eventually build deb packages out of device like explained for Raspberry Pi 0:

How to Run IoT.js on the Raspberry PI 0

WebThing for IoT.js

Now we have an environment ready to try my development branch of “webthing-node” ported to IoT.js (until ready for iotjs-modules):

cd /usr/local/src/
git clone https://github.com/tizenteam/webthing-node -b sandbox/rzr/devel/iotjs/master --recursive --depth 1 

Then just start “webthing” server:

cd webthing-node
iotjs example/simplest-thing.js

In other shell check if thing is alive:

curl -H "Content-Type: application/json"  http://localhost:8888/

{"name":"ActuatorExample","href":"/","type":"onOffSwitch","properties":{"on":{"type":"boolean","description":"Whether the output is changed","href":"/properties/on"}},"links":[{"rel":"properties","href":"/properties"}],"description":"An actuator example that just log"}

Then we can control our resource’s property (which actually just a LED on GPIO, but it could be a relay or any other actuator)

curl -X PUT -H "Content-Type: application/json" --data '{"on": true }' http://localhost:8888/properties/on
gpio: writing: true
curl http://localhost:8888/properties/on
{ "on": true }

Install webthing service

Systemd will help us to start the webthing server on boot listing on default http port:

unit="webthing-iotjs"
dir="/usr/local/src/$unit"
exedir="$dir/bin"
exe="$exedir/$unit.sh"
servicedir="$dir/lib/systemd/"
service="$servicedir/$unit.service"

mkdir -p "$exedir"
cat<"$exe" && chmod a+rx "$exe"
#!/bin/sh
set -x
set -e
PATH=$PATH:/usr/local/bin
cd /usr/local/src
cd webthing-node
# Update port and GPIO if needed
iotjs example/simplest-thing.js 80 45
EOF


mkdir -p "$servicedir" && cat<$service
[Unit]
Description=$unit
After=network.target

[Service]
ExecStart=$exe
Restart=always

[Install]
WantedBy=multi-user.target
EOF

Enable service:

killall iotjs
systemctl enable $service
systemctl start $unit
systemctl status $unit
reboot

Connect to gateway

Set up “Mozilla IoT gateway” as explained earlier, you can add some “virtual resources” to check it’s working too:

Connecting sensors to Mozilla’s IoT Gateway

Connecting our “ActuatorExample webthing” to gateway will require you to fill the explicit URL of Edison device.

Using your browser, open gateway location at mozilla-iot.org or http://gateway.local :

On “/things” page click on “+” button to add a thing, then “Add by URL…” wait for form:


Enter web thing URL:
iotjs.local

press “Submit” it will display:


ActuatorExample
On/Off Switch from iotjs.local:80

then press “Save” and “Done”, it should appear on Dashboard, you can click on “Actuator Example”‘s icon to turn it off or on.

Let’s verify is working by connecting a LED (Red) to Intel Edison’s minibreakout board (pinout) which is delivering 1.8V :

  • GPIO45: on J20 (bottom row), use the 4th pin (from right to left)
  • GND: on J19 (just above J20), use the 3th pin (from right to left)

Note, for later the reason why it was not scanned automagically is because I removed mDNS feature to ease the port, but a native or pure js module could be reintroduced later in, to behave like NodeJS webthing.

RGBLamp on Microcontrollers

Another platform to consider is ESP8266 (or ESP32), it’s a Wifi enabled microcontroller.
ESPxx are officially supported by Mozilla with a partial implementation of WebThings currently using native Ardiuno APIs. You can even try out my native implementation of RGBLamp.

If curious, you can also get (or make!) OpenSourceHardware light controller from Tizen community’s friend Leon.

Then it would worth comparing with a JavaScript version since JerryScript is supporting ESP8266.
Note that If IoT.js should be downsized for ESP8266, maybe ESP32 could be considered too (on FreeRTOS ?).

Other devices supported by IoT.js can be considered too, such as ARTIK05x (on TizenRT)

Standalone Tizen WebApp

Finally as a bonus chapter, while hacking on Tizen TM1, I made a standalone app that can login to Mozilla gateway and browse resources.

First time it should retrieve OAuth token, and then the browser is able to list existing resources.

If you don’t have a Tizen TM1 device, you can try using SDK or even your desktop browser (with CORS feature configured):

rm -rf tmp/chromium
mkdir -p tmp/chromium
chromium-browser --disable-web-security  --user-data-dir="tmp/chromium" https://rzr.github.io/webthings-webapp/

Feel free to contribute, for debugging purposes this URL can also be used
https://tizenteam.github.io/webthings-webapp/

More details on this experiment at https://github.com/rzr/webthings-webapp.

Further Reading

IoT.js landed in Raspbian

Following previous efforts to deploy iotjs on Raspberry Pi 0, I am happy to announce that IoT.js 1.0 landed in Debian, and was sync’d to Raspbian for ArmHF and Ubuntu as well.

While the package is targeting the next distro release, it can be easily installed on current versions by adding a couple of config files for “APT pinning”.

If you haven’t set up Raspbian 9, just dump the current Raspbian image to SDcard (for the record I used version 2018-03-13-raspbian-stretch-lite)

Boot your Pi.  To keep track of changes in /etc/, let’s install etckeeper:

sudo apt-get update
sudo apt-get install etckeeper

Upgrade current packages:

sudo apt-get upgrade
sudo apt-get dist-upgrade

Declare the current release as default source:

cat<<EOT | sudo tee /etc/apt/apt.conf.d/50raspi
APT::Default-Release "stretch";
EOT

Then add a repo file for the next release:

cat /etc/apt/sources.list | sed 's/stretch/buster/g' | sudo tee /etc/apt/sources.list.d/raspi-buster.list

Unless you want to test the upcoming release, it maybe be safer to avoid upgrading all packages yet.  In other words, we prefer that only iotjs should be available from this “not yet supported” repo.

cat<<EOT | sudo tee /etc/apt/preferences.d/raspi-buster.pref
Package: *
Pin: release n=buster
Pin-Priority: -10
EOT

cat<<EOT | sudo tee /etc/apt/preferences.d/iotjs.pref
Package: iotjs
Pin: release n=buster
Pin-Priority: 1
EOT

Now iotjs 1.0-1 should appear as available for installation:

sudo apt-get update ; apt-cache search iotjs
iotjs - Javascript Framework for Internet of Things

apt-cache policy iotjs
iotjs:
  Installed: (none)
  Candidate: 1.0-1
  Version table:
     1.0-1 1
        -10 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages

Let’s install it:

sudo apt-get install iotjs
man iotjs

Even if version 1.0 is limited in compared to the development branch, you can start by using the http module which is enabled by default (not https).

To illustrate this, when I investigated “air quality monitoring” for a TizenRT+IoT.js demo I found out that OpenWeatherMap is collecting and publishing “Carbon Monoxide” Data, so let’s try their REST API.

Create a file, example.js for example, that contains:

var http = require('http');

var location = '48,-1';
var datetime = 'current';

//TODO: replace with your openweathermap.org personal key
var api_key = 'fb3924bbb699b17137ab177df77c220c';

var options = {
  hostname: 'api.openweathermap.org',
  port: 80,
  path: '/pollution/v1/co/' + location + '/' + datetime + '.json?appid=' + api_key,
};

// workaround bug
options.headers = {
  host: options.hostname
}

http.request(options, function (res) {
  receive(res, function (data) {
    console.log(data);
  });
}).end();

function receive(incoming, callback) {
  var data = '';

  incoming.on('data', function (chunk) {
    data += chunk;
  });

  incoming.on('end', function () {
    callback ? callback(data) : '';
  });
}

And just run it:

iotjs example.js
{"time":"2018-03-27T02:24:33Z","location":{"latitude":47.3509,"longitude":-0.9081},"data":[{"precision":-4.999999987376214e-07,"pressure":1000,"value":1.5543508880000445e-07}
(...)

You can then use this to do things such as update a map or raise an alert on anything useful, or try to rebuild master branch.

An Introduction to IoT.js Architecture

IoT.js is a lightweight JavaScript platform for the Internet of Things. The platform keeps interoperable services at the forefront, and is designed to bring the success of Node.js to IoT devices like micro-controllers and other devices that are constrained to limited storage and only a few kilobytes of RAM. IoT.js is built on top of JerryScript: a lightweight JavaScript interpreter, and libtuv: an event driven (non-blocking I/O model) library. The project is open source under the Apache 2.0 license.

This article will introduce you to the architecture of IoT.js and the fundamentals of writing applications for it.

IoT.js Architecture

An Introduction to IoT.js Architecture - iotjs_arch

JerryScript – ECMAScript binding

JerryScript is the kernel for IoT.js on an ECMAScript binding, it’s an ultra lightweight JavaScript engine that was written from scratch at Samsung. The name “Jerry” comes from the popular character in Tom and Jerry, who’s small, smart, and fast! Since the engine is an interpreter only, it might be more precise to say JerryScript is a JavaScript interpreter.

Optimizations for memory footprint and performance have always been the top priorities of the project. The tiny engine has a base RAM footprint of less than 64KB, and the binary can accommodate less than 200KB of ROM. Amazingly, it implements the full ECMAScript 5.1 standard, and work has been ongoing recently to introduce new ES6 features such as promise, TypedArray, and more.

Like IoT.js, JerryScript was also released under the Apache 2.0 license. The community has experienced very rapid growth, especially in last couple years, and in 2016 the project was transferred to the JavaScript Foundation. A recent JavaScript Foundation press release mentioned how JerryScript was adopted in Fitbit’s latest product: Ionic.

JerryScript provides a good set of embedding APIs to compile and execute JavaScript programs, access JavaScript objects and their values, handle errors, manage the lifestyles of objects, and more. IoT.js uses these API’s to create the builtin module and native handler in IoT.js native core.

libtuv – I/O Event binding

Asynchronous I/O and threading in IoT.js are handled with libtuv: a library that focuses on asynchronous I/O and was primarily developed for use with Node.js. Samsung launched this open source project under the Apache 2.0 license, and it’s a multi-platform tiny event library that’s refactored from libuv to better serve IoT.js and on embedded systems.

Libtuv’s features include a loop, timer, poll, tcp & udp, fs event, thread, worker, and more. The platforms this library supports include i686-linux, arm-linux, arm-nuttx and arm-mbed.

IoT Binding

In the IoT.js community, there have been discussions about binding to an existing IoT platform or specification, such as IoTivity: The Open Connectivity Foundation‘s open source project. If this were to happen it would certainly add more dimension to supporting interoperability with other platforms.

IoT.js C Core

The IoT.js core layer is located above the binding layer, and it provides upper layer functionality to interact with the JavaScript engine, including running main event loops, managing I/O resources, etc. It also provides a set of builtin modules and native handlers.

Builtin modules are the basic and extended modules that are included in the IoT.js binary. Basically, these builtin modules are JavaScript objects implemented in C using the embedding API JerryScript provides, in either JavaScript or both languages. The native components of builtin modules are implemented as a native handle to access underlying systems via event handling, a C library, or system calls.

The life cycle of IoT.js is shown below:
 
An Introduction to IoT.js Architecture - iotjs_lifecycle-1

IoT.js ECMAScript API and JavaScript Modules

Like Node.js, IoT.js is a module-based system. Each module in IoT.js has its own context and provides a set of API’s associated with the module’s functionality.

IoT.js offers basic API modules and extended API modules. The basic API modules are based on Node.js and follow same form for compatibility reasons. Basic API modules include File System, Net, HTTP, Process, etc. Pretty much all application code that calls these API’s can be run in a Node.js environment without any modification.

The extended modules, on the other hand, are more IoT.js specific, and they are currently mostly hardware related (e.g. GPIO, Bluetooth Low Energy (BLE), I2C, SPI, UART, PWM, etc.). Many contributors are interested in adding new extended API modules to support their own specific hardware, so to maintain consistent usability, the IoT.js community has set guidelines and rules for introducing extended API’s.

Enabling JavaScript on the Internet of Things

The overall architecture of IoT.js is very friendly to Node.js, as a result of the asynchronous I/O and threading library, and the subset of Node.js compatible modules; it has reflected the design philosophy of providing a lightweight version of Node.js along with an inter-operable service platform. IoT.js has opened a great opportunity for JavaScript developers to develop applications for the Internet of Things, so it’s definitely an IoT platform to watch!

How to Run IoT.js on the Raspberry PI 0

IoT.js is a lightweight JavaScript platform for building Internet of Things devices; this article will show you how to run it on a few dollars worth of hardware. The First version of it was released last year for various platforms including Linux, Tizen, and NuttX (the base of Tizen:RT). The Raspberry Pi 2 is one of the reference targets, but for demo purposes we also tried to build for the Raspberry Pi Zero, which is the most limited and cheapest device of the family. The main difference is the CPU architecture, which is ARMv6 (like the Pi 1), while the Pi 2 is ARMv7, and the Pi 3 is ARMv8 (aka ARM64).

IoT.js upstream uses a python helper script to crossbuild for supported devices, but instead of adding support to new device I tried to build on the device using native tools with cmake and the default compiler options; it simply worked! While working on this, I decided to package iotjs for debian to see how well it will support other architectures (MIPS, PPC, etc), we will see.

Unfortunately, Debian armel isn’t optimized for ARMv6 and FPU, both of which are present on the Pi 1 and Pi 0, so the Raspbian project had to rebuild Debian for the ARMv6+VFP2 ARM variant to support all Raspberry Pi SBC’s.

In this article, I’ll share hints for running IoT.js on Raspbian: the OS officially supported by the Raspberry foundation; the following instructions will work on any Pi device since a portability strategy was preferred over optimization. I’ll demonstrate three separate ways to do this: from packages, by building on the device, and by building in a virtual machine. By the way, an alternative to consider is to rebuild Tizen Yocto for  the Pi 0, but I’ll leave that as an exercise for the reader, you can accomplish this with a bitbake recipe, or you can ask for more hints in the comments section.

How to Run IoT.js on the Raspberry PI 0 - tizen-pizero

Install from Packages

iotjs landed in Debian’s sid, and until it is in testing branch (and subsequently Raspbian and Ubuntu), the fastest way is to download it is via precompiled packages from my personal Raspbian repo

url='https://dl.bintray.com/rzr/raspbian-9-armhf'
source="/etc/apt/sources.list.d/bintray-rzr-raspbian-9-armhf.list"
echo "deb $url raspbian main" | sudo tee "$source"
sudo apt-get update
apt-cache search iotjs
sudo apt-get install iotjs
/usr/bin/iotjs
Usage: iotjs [options] {script | script.js} [arguments]

Use it

Usage is pretty straightforward, start with a hello world source:

echo 'console.log("Hello IoT.js !");' > example.js
iotjs  example.js 
Hello IoT.js !

More details about the current environment can be used (this is for iotjs-1.0 with the default built-in modules):

echo 'console.log(JSON.stringify(process));' > example.js
iotjs  example.js 
{"env":{"HOME":"/home/user","IOTJS_PATH":"","IOTJS_ENV":""},"native_sources":{"assert":true,"buffer":true,"console":true,"constants":true,"dns":true,"events":true,"fs":true,"http":true,"http_client":true,"http_common":true,"http_incoming":true,"http_outgoing":true,"http_server":true,"iotjs":true,"module":true,"net":true,"stream":true,"stream_duplex":true,"stream_readable":true,"stream_writable":true,"testdriver":true,"timers":true,"util":true},"platform":"linux","arch":"arm","iotjs":{"board":"\"unknown\""},"argv":["iotjs","example.js"],"_events":{},"exitCode":0,"_exiting":false} null 2

From here, you can look to use other built-in modules like http, fs, net, timer, etc.

Need More Features?

More modules can be enabled in the master branch, so I also built snapshot packages that can be installed to enable more key features like GPIO, I2C, and more. For your convenience, the snapshot package can be installed to replace the latest release:

root@raspberrypi:/home/user$ apt-get remove iotjs iotjs-dev iotjs-dbgsym iotjs-snapshot
root@raspberrypi:/home/user$ aptitude install iotjs-snapshot
The following NEW packages will be installed:
  iotjs-snapshot{b}
(...)
The following packages have unmet dependencies:
 iotjs-snapshot : Depends: iotjs (= 0.0~1.0+373+gda75913-0~rzr1) but it is not going to be installed
The following actions will resolve these dependencies:
     Keep the following packages at their current version:
1)     iotjs-snapshot [Not Installed]                     
Accept this solution? [Y/n/q/?] n
The following actions will resolve these dependencies:

     Install the following packages:                 
1)     iotjs [0.0~1.0+373+gda75913-0~rzr1 (raspbian)]
Accept this solution? [Y/n/q/?] y
The following NEW packages will be installed:
  iotjs{a} iotjs-snapshot 
(...)
Do you want to continue? [Y/n/?] y
(...)
  iotjs-snapshot https://dl.bintray.com/rzr/raspbian-9-armhf/iotjs-snapshot_0.0~1.0+373+gda75913-0~rzr1_armhf.deb
  iotjs https://dl.bintray.com/rzr/raspbian-9-armhf/iotjs_0.0~1.0+373+gda75913-0~rzr1_armhf.deb

Do you want to ignore this warning and proceed anyway?
To continue, enter "yes"; to abort, enter "no": yes
Get: 1 https://dl.bintray.com/rzr/raspbian-9-armhf raspbian/main armhf iotjs armhf 0.0~1.0+373+gda75913-0~rzr1 [199 kB]
Get: 2 https://dl.bintray.com/rzr/raspbian-9-armhf raspbian/main armhf iotjs-snapshot armhf 0.0~1.0+373+gda75913-0~rzr1 [4344 B]
(...)

If you the run console.log(process) again, you’ll see more interesting modules to use, like gpio, i2c, uart and more, and external modules can be also used; check on the work in progress for sharing modules to the IoT.js community. Of course, this can be reverted to the latest release by simply installing the iotjs package because it has higher priority than the snapshot version.

root@raspberrypi:/home/user$ apt-get install iotjs
(...)
The following packages will be REMOVED:
  iotjs-snapshot
The following packages will be upgraded:
  iotjs
(...)
Do you want to continue? [Y/n] y
(...)

Build on the Device

It’s also possible to build the snapshot package from source with extra packaging patches, found in the community branch of IoT.js (which can be rebased on upstream anytime).

sudo apt-get install git time sudo
git clone https://github.com/tizenteam/iotjs
cd iotjs
./debian/rules
sudo debi

On the Pi 0, it took less than 30 minutes over NFS for this to finish. If you want to learn more you can follow similar instructions for building IoTivity on ARTIK;
it might be slower, but it will extend life span of your SD Cards.

Build on a Virtual Machine

A faster alternative that’s somewhere between building on the device and setting up a cross build environment (which always has a risk of inconsistencies) is to rebuild IoT.js with QEMU, Docker, and binfmt.

First install docker (I used 17.05.0-ce and 1.13.1-0ubuntu6), then install the remaining tools:

sudo apt-get install qemu qemu-user-static binfmt-support time
sudo update-binfmts --enable qemu-arm

docker build 'http://github.com/tizenteam/iotjs.git'

It’s much faster this way, and took me less than five minutes. The files are inside the container, so they need to be copied back to host. I made a helper script for setup and to get the deb packages ready to be deployed on the device (sudo dpkg -i *.deb) :

curl -sL https://rawgit.com/tizenteam/iotjs/master/run.sh | bash -x -
./tmp/out/iotjs/iotjs-dbgsym_0.0-0_armhf.deb
./tmp/out/iotjs/iotjs-dev_0.0-0_armhf.deb
./tmp/out/iotjs/iotjs_0.0-0_armhf.deb

I used the Resin/rpi-raspbian docker image (thanks again Resin!). Finally, I want to also thank my coworker Stefan Schmidt for the idea after he setup a similar trick for EFL’s CI.

Further Reading

If you want to learn more, here are some additional resources to take your understanding further.

Raspberry Pi is a trademark of the Raspberry Pi Foundation\