IoT.js landed in Raspbian

Following previous efforts to deploy iotjs on Raspberry Pi 0, I am happy to announce that IoT.js 1.0 landed in Debian, and was sync’d to Raspbian for ArmHF and Ubuntu as well.

While the package is targeting the next distro release, it can be easily installed on current versions by adding a couple of config files for “APT pinning”.

If you haven’t set up Raspbian 9, just dump the current Raspbian image to SDcard (for the record I used version 2018-03-13-raspbian-stretch-lite)

Boot your Pi.  To keep track of changes in /etc/, let’s install etckeeper:

sudo apt-get update
sudo apt-get install etckeeper

Upgrade current packages:

sudo apt-get upgrade
sudo apt-get dist-upgrade

Declare the current release as default source:

cat<<EOT | sudo tee /etc/apt/apt.conf.d/50raspi
APT::Default-Release "stretch";
EOT

Then add a repo file for the next release:

cat /etc/apt/sources.list | sed 's/stretch/buster/g' | sudo tee /etc/apt/sources.list.d/raspi-buster.list

Unless you want to test the upcoming release, it maybe be safer to avoid upgrading all packages yet.  In other words, we prefer that only iotjs should be available from this “not yet supported” repo.

cat<<EOT | sudo tee /etc/apt/preferences.d/raspi-buster.pref
Package: *
Pin: release n=buster
Pin-Priority: -10
EOT

cat<<EOT | sudo tee /etc/apt/preferences.d/iotjs.pref
Package: iotjs
Pin: release n=buster
Pin-Priority: 1
EOT

Now iotjs 1.0-1 should appear as available for installation:

sudo apt-get update ; apt-cache search iotjs
iotjs - Javascript Framework for Internet of Things

apt-cache policy iotjs
iotjs:
  Installed: (none)
  Candidate: 1.0-1
  Version table:
     1.0-1 1
        -10 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages

Let’s install it:

sudo apt-get install iotjs
man iotjs

Even if version 1.0 is limited in compared to the development branch, you can start by using the http module which is enabled by default (not https).

To illustrate this, when I investigated “air quality monitoring” for a TizenRT+IoT.js demo I found out that OpenWeatherMap is collecting and publishing “Carbon Monoxide” Data, so let’s try their REST API.

Create a file, example.js for example, that contains:

var http = require('http');

var location = '48,-1';
var datetime = 'current';

//TODO: replace with your openweathermap.org personal key
var api_key = 'fb3924bbb699b17137ab177df77c220c';

var options = {
  hostname: 'api.openweathermap.org',
  port: 80,
  path: '/pollution/v1/co/' + location + '/' + datetime + '.json?appid=' + api_key,
};

// workaround bug
options.headers = {
  host: options.hostname
}

http.request(options, function (res) {
  receive(res, function (data) {
    console.log(data);
  });
}).end();

function receive(incoming, callback) {
  var data = '';

  incoming.on('data', function (chunk) {
    data += chunk;
  });

  incoming.on('end', function () {
    callback ? callback(data) : '';
  });
}

And just run it:

iotjs example.js
{"time":"2018-03-27T02:24:33Z","location":{"latitude":47.3509,"longitude":-0.9081},"data":[{"precision":-4.999999987376214e-07,"pressure":1000,"value":1.5543508880000445e-07}
(...)

You can then use this to do things such as update a map or raise an alert on anything useful, or try to rebuild master branch.

Beware of Ubuntu 16.10 Upgrade Woes

I wanted to share a word of caution for anybody planning to update their development and test systems to Ubuntu 16.10: I can’t build kernels anymore. Ubuntu recommends a special patch to the kernel Makefile. This patch will work only on Ubuntu kernel sources and not the upstream Linux kernel trees.

Linux kernel builds fail with the following message

CHK include/config/kernel.release
Cannot use CONFIG_CC_STACKPROTECTOR_STRONG: -fstack-protector-strong not supported by compiler
Makefile:1058: recipe for target 'prepare-compiler-check' failed
make: *** [prepare-compiler-check] Error 1

The message about CONFIG_CC_STACKPROTECTOR_STRONG is misleading because this Kernel config option is enabled in most distro kernels; disabling it won’t solve the kernel build failure problem. It fails because the position independent executable option is set as default in gcc version 6.2.0 20161005 (Ubuntu 6.2.0-5ubuntu12). As a result, Linux Kernel Makefile needs to update to build the kernel with “-fno-pie” option.

The Ubuntu 16.10 release notes say

We have modified GCC to by-default compile programs with position independent executable support to improve the security benefits provided by Address Space Layout Randomization.

This may cause difficulty when trying to compile Linux kernels that still need this patch applied. Other programs may experience other problems; some debugging guidelines are at https://wiki.ubuntu.com/SecurityTeam/PIE

This clearly states a patch needs to applied to the Linux Kernel makefile. This patch forces no-pie for distro compilers that enable pie by default. So at the moment, I am going to refrain from upgrading my development and test systems.

I am following up with the Ubuntu kernel team and upstream on this issue. There is a patch in the works upstream to address the GCC change to enable position independent executable option by default. In the meantime, the following change worked for me on Linux 4.8.4 and Linux 4.9-rc1. I was able to build Linux 4.8.4 successfully and it is running nicely.

diff --git a/Makefile b/Makefile
index 82a36ab..0a01ad1 100644
--- a/Makefile
+++ b/Makefile
@@ -651,6 +651,11 @@ ifneq ($(CONFIG_FRAME_WARN),0)
KBUILD_CFLAGS += $(call cc-option,-Wframe-larger-than=${CONFIG_FRAME_WARN})
endif

+# force no-pie for distro compilers that enable pie by default
+KBUILD_CFLAGS += $(call cc-option, -fno-pie)
+KBUILD_CFLAGS += $(call cc-option, -no-pie)
+KBUILD_AFLAGS += $(call cc-option, -fno-pie)
+
# This selects the stack protector compiler flag. Testing it is delayed
# until after .config has been reprocessed, in the prepare-compiler-check
# target.

How to Recover Deleted Files From Ubuntu Guest Sessions on Encrypted Volumes

Ubuntu guest sessions are a convenient way to let someone use your system with limited access to the network, file system, and other system functions without the ability to save data. The /usr/lib/lightdm/lightdm-guest-session binary handles guest sessions, and several services which are deemed unnecessary for a guest user are disabled. Refer to /usr/share/lightdm/guest-session/setup.sh for details on what is enabled.

When guest session starts, you will see a warning that any data saved will be lost as shown below.

How to Recover Deleted Files From Ubuntu Guest Sessions on Encrypted Volumes - guest_session

A temporary home directory is created under /tmp which will be deleted when guest session ends via logout, or a reboot. When the guest session is active, you will see a directory guest-xxxx as in the e.g /tmp/guest-6vqi30.

So what do you do if a guest session ends unexpectedly?! I recently logged out of a guest session by mistake and had to scramble to recover my son’s school essay which was ready to be submitted. I searched for help and every single article said I was out of luck and there is no way to recover files after a guest session ends. I found one article that gave me some hope by recommending testdisk; I installed testdisk and started my analysis. However, adding to my troubles, I have an encrypted root partition and testdisk failed to find the lost file even when I asked it to analyze the decrypted device file. Then, I came across scalpel, a tool that can restore deleted files. Since I kept the laptop running without rebooting it, I decided to give scalpel a try to see if I can find anything on tmpfs.

sudo apt-get install scalpel

Scalpel recovers files using a header/footer database. This means you can search for specific file types such as audio (wav, ra), LibreOffice (odt, odp),  PGP (pgd, pgp, txt), graphics (jpg, png), and so on. The scalpel configuration file  /etc/scalpel/scalpel.conf is used to control the types and sizes of files that are carved. For each file type, the configuration file describes the file’s extension, header and footer case sensitivity, size, and the header/footer; the footer is optional.

#               case    size    header                  footer
#extension   sensitive  
#
#---------------------------------------------------------------------
# EXAMPLE WITH NO SUFFIX
#---------------------------------------------------------------------
#
# Here is an example of how to use the no extension option. Any files 
# beginning with the string "FOREMOST" are carved and no file extensions
# are used. No footer is defined and the max carve size is 1000 bytes.
#
#      NONE     y      1000     FOREMOST
#

The scalpel default configuration file didn’t include LibreOffice, but I found an article that specified the right information. I added the following to the end of /etc/scalpel/scalpel.conf

#---------------------------------------------------------------------
# OPENOFFICE FILES
#---------------------------------------------------------------------
    odt y   20000000    PK????????????????????????????mimetypeapplication/vnd.oasis.opendocument.textPK META-INF/manifest.xmlPK????????????????????
    ods y   10000000    PK????????????????????????????mimetypeapplication/vnd.oasis.opendocument.spreadsheetPK  META-INF/manifest.xmlPK????????????????????
    odp y   10000000    PK????????????????????????????mimetypeapplication/vnd.oasis.opendocument.presentationPK META-INF/manifest.xmlPK????????????????????
#    odg y   10000000    PK????????????????????????????mimetypeapplication/vnd.oasis.opendocument.graphicsPK META-INF/manifest.xmlPK????????????????????
#    odc y   10000000    PK????????????????????????????mimetypeapplication/vnd.oasis.opendocument.chartPK    META-INF/manifest.xmlPK????????????????????
#    odf y   10000000    PK????????????????????????????mimetypeapplication/vnd.oasis.opendocument.formulaPK  META-INF/manifest.xmlPK????????????????????
#    odi y   10000000    PK????????????????????????????mimetypeapplication/vnd.oasis.opendocument.imagePK    META-INF/manifest.xmlPK????????????????????
#    odm y   10000000    PK????????????????????????????mimetypeapplication/vnd.oasis.opendocument.text-masterPK  META-INF/manifest.xmlPK????????????????????
#    sxw y   10000000    PK????????????????????????????mimetypeapplication/vnd.sun.xml.writerPK  META-INF/manifest.xmlPK????????????????????
#---------------------------------------------------------------------

The disk is encrypted, so we have to find the right device file to ask scalpel to carve.

sudo fdisk -l

Find the device named /dev/mapper/ubuntu–vg-root, this is the decrypted root device. Now we can run scalpel on this device. First create a recovery directory, I’m going to call it recover. I used the -b option to tell scalpel to carve files even if the defined footers aren’t discovered within maximum carve size for the file type.

sudo scalpel /dev/mapper/ubuntu--vg-root -b -o recover

When scalpel completed, I found several .odt files in the recover directory. At this point, it’s a matter of opening each to find the file that contains the complete essay. We had a happy ending with the full essay restored!

Here’s some lessons I learned about recovery from this experience:

  1. Guest accounts are best for special cases when data doesn’t need to saved.
  2. If you end up using guest account and have data to save, save it to permanent storage before logging out or rebooting.
  3. Customize guest sessions on the system to store files permanently.
  4. If you end up in a similar situation as me, there’s no need to despair as it isn’t the end of the world. Don’t reboot the system and make sure it stays powered on.
  5. Even if you have an encrypted disk it’s no problem, scalpel can recover deleted files from tmpfs.

The Future of Open Source, Ubuntu Joins Open Compute Project & New Leadership for the OSI

Open Source Weekly Wrap Up: May 9 -15 2015

2015 Future of Open Source Study

For the last 15 years, North Bridge and Black Duck Software have joined efforts every year to conduct a survey about the use of open source in modern business. A record 1,300 people responded to the survey this year, up from 822 in 2013. Respondents included software engineers, CIOs, CEOs, educators, analysts, and more. There is little that has changed in open source trends, and this survey demonstrates the continued pervasive expansion of open source.

Here are some notable findings from this year’s survey:

Enterprise Use/Participation

  • 78% of companies run on open source, and less than 3% don’t use FOSS at all.
  • 64% of respondents said their company currently participates in open source projects, up 14% YOY.
  • 88% expect their companies to increase open source contributions, up from 56% last year.
  • 66% consider OSS to be the default option before proprietary.

The Open Source Advantage

  • 50% said open source involvement helps their company recruit top talent.
  • 55% said open source delivers better security
  • 89% said open source greatly impacts the speed of innovation
  • 78% said open source improves margins

What Needs Improvements

  • 55% have no policies or procedures for open source compliance.
  • 27% have a formal policy for employee contributions to OSS.
  • 16% have an automated code approval process.

Read More from North Bridge

Snappy Ubuntu Joins the Open Compute Project

Canonical, the lead company behind the Ubuntu Linux distribution, has announced that Ubuntu Snappy will be coming to Open Compute Project (OCP) network switches. Ubuntu will bring an assortment of benefits to OCP, including the fastest, smallest, and most reliable Ubuntu distribution ever, transactional updates for each app and kernel separately with rollback functionality, a new, simpler packaging system, and an assortment of NOS and NFV applications. Penguin Computing will be the first company to offer switches running Snappy, and their initial offerings will include top of rack 10GE and aggregation 40GE models.

Read more from Ubuntu Insights.

Samsung Launches ARTIK, a New Platform for IoT

Samsung has announced the release of ARTIK, an open source platform designed for building IoT devices. The goal of ARTIK is to reduce the fragmentation of the IoT market by providing an end-to-end solution for making interconnected applications for everything from low-powered wearables to high-end smart homes. The group behind the product worked with the team from Arduino to fully integrated the Arduino IDE into ARTIK, making it easier for developers to get started. Samsung is currently taking applications until May 31st to receive an alpha developer kit, so if you are interested in building IoT devices, be sure to check this out.

Read More at the Artik Blog.

The Open Source Initiative Elects a New President

Simon Phipps was elected as the President of the Open Source Initiative (OSI) in 2012, and his term ends in April 2016. He recently decided not to pursue a new term as president of the board, and the board has elected Allison Randal to replace him starting next year. Allison has a long history of working for open source foundations including the Perl Foundation, Parrot Foundation, and Python Software Foundation. She appears to be an ideal candidate to run the organization.

Read more at the OSI blog.