Psion, documents over FTP

In the early 2000s we were into adventure motorcycling. To be able to share our adventures, we implemented a way to publish blogs while on the go. The device that enabled this was the Psion Series 5, a handheld computer that was very much a device ahead of its time.

The Psion had a reasonably sized keyboard and a good native word processing app. And battery life good for weeks of usage. Writing while underway was easy. The Psion could use a mobile phone as a modem over an infrared connection, and with that we could upload the documents to a server over FTP.

Server-side, a cron job would grab the new documents, converting them to HTML and adding them to our CMS.

In the early days of GPRS, getting this to work while roaming was quite tricky. But the system served us well for years.

If we wanted to include photos to the stories, we’d have to find an Internet cafe.

• To the Alps is a post from these times. Lots more in the motorcycling category

SMS and MMS

For an even more mobile setup, I implemented an SMS-based blogging system. We had an old phone connected to a computer back in the office, and I could write to my blog by simply sending a text. These would automatically end up as a new paragraph in the latest post. If I started the text with NEWPOST, an empty blog post would be created with the rest of that message’s text as the title.

• In the Caucasus is a good example of a post from this era

As I got into neogeography, I could also send a NEWPOSITION message. This would update my position on the map, connecting weather metadata to the posts.

As camera phones became available, we wanted to do pictures too. For the Death Monkey rally where we rode minimotorcycles from Helsinki to Gibraltar, we implemented an MMS-based system. With that the entries could include both text and pictures. But for that you needed a gateway, which was really only realistic for an event with sponsors.

• Mystery of the Missing Monkey is typical. Some more in Internet Archive

Photos over email

A much easier setup than MMS was to slightly come back to the old Psion setup, but instead of word documents, sending email with picture attachments. This was something that the new breed of (pre-iPhone) smartphones were capable of. And by now the roaming question was mostly sorted.

And so my blog included a new “moblog” section. This is where I could share my daily activities as poor-quality pictures. Sort of how people would use Instagram a few years later.

• Internet Archive has some of my old moblogs but nowadays, I post similar stuff on Pixelfed

Pause

Then there was sort of a long pause in mobile blogging advancements. Modern smartphones, data roaming, and WiFi hotspots had become ubiquitous.

In the meanwhile the blog also got migrated to a Jekyll-based system hosted on AWS. That means the old Midgard-based integrations were off the table.

And I traveled off-the-grid rarely enough that it didn’t make sense to develop a system.

But now that we’re sailing offshore, that has changed. Time for new systems and new ideas. Or maybe just a rehash of the old ones?

Starlink, Internet from Outer Space

Most cruising boats - ours included - now run the Starlink satellite broadband system. This enables full Internet, even in the middle of an ocean, even video calls! With this, we can use normal blogging tools. The usual one for us is GitJournal, which makes it easy to write Jekyll-style Markdown posts and push them to GitHub.

However, Starlink is a complicated, energy-hungry, and fragile system on an offshore boat. The policies might change at any time preventing our way of using it, and also the dishy itself, or the way we power it may fail.

But despite what you’d think, even on a nerdy boat like ours, loss of Internet connectivity is not an emergency. And this is where the old-style mobile blogging mechanisms come handy.

• Any of the 2025 Atlantic crossing posts is a good example of this setup in action

Inreach, texting with the cloud

Our backup system to Starlink is the Garmin Inreach. This is a tiny battery-powered device that connects to the Iridium satellite constellation. It allows tracking as well as basic text messaging.

When we head offshore we always enable tracking on the Inreach. This allows both our blog and our friends ashore to follow our progress.

I also made a simple integration where text updates sent to Garmin MapShare get fetched and published on our blog. Right now this is just plain text-based entries, but one could easily implement a command system similar to what I had over SMS back in the day.

One benefit of the Inreach is that we can also take it with us when we go on land adventures. And it’d even enable rudimentary communications if we found ourselves in a liferaft.

• There are various InReach integration hacks that could be used for more sophisticated data transfer

Sailmail and email over HF radio

The other potential backup for Starlink failures would be to go seriously old-school. It is possible to get email access via a SSB radio and a Pactor (or Vara) modem.

Our boat is already equipped with an isolated aft stay that can be used as an antenna. And with the popularity of Starlink, many cruisers are offloading their old HF radios.

Licensing-wise this system could be used either as a marine HF radio (requiring a Long Range Certificate), or amateur radio. So that part is something I need to work on. Thankfully post-COVID, radio amateur license exams can be done online.

With this setup we could send and receive text-based email. The Airmail application used for this can even do some automatic templating for position reports. We’d then need a mailbox that can receive these mails, and some automation to fetch and publish.

• Sailmail and No Foreign Land support structured data via email to update position. Their formats could be useful inspiration

0 0

As customization increases, it becomes important to have a way to manage that and distribute it across the different computers. For years, I’ve used a dotfiles repository on GitHub together with GNU Stow for this.

However, this still means I have to install all the software and tools before I can have my environment up and running.

Using Docker

Docker is a tool for building and running software in a containerized fashion. Recently Tiago gave me the inspiration to use Docker not only for distributing production software, but also for actually running my development environment.

Taking ideas from his setup, I built upon my existing dotfiles and built a reusable developer shell container.

With this, I only need Docker installed on a machine, and then I’m two commands away from having my normal development environment:

$ docker volume create workstation
$ docker run -v ~/Projects:/projects -v workstation:/root -v ~/.ssh:/keys --name workstation --rm -it bergie/shell

Here’s how it looks in action:

Once I update my Docker setup (for example to install or upgrade some tool), I can get the latest version on a machine with:

$ docker pull bergie/shell

At least in theory this should give me a fully identical working environment regardless of the host machine. Linux VPS, a MacBook, or a Windows machine should all be able to run this. And soon, this should also work out of the box on Chromebooks.

Setting this up

The basics are pretty simple. I already had a repository for my dotfiles, so I only needed to write a Dockerfile to install and set up all my software.

To make things even easier, I configured Travis so that every time I push some change to the dotfiles repository, it will create and publish a new container image.

Further development ideas

So far this setup seems to work pretty well. However, here are some ideas for further improvements:

• ARM build: Sometimes I need to work on Raspberry Pis. It might be nice to cross-compile an ARM version of the same setup
• Key management: Currently I create new SSH keys for each host machine, and then upload them to the relevant places. With this setup I could use a USB stick, or maybe even a Yubikey to manage them
• Application authentication: Since the Docker image is public, it doesn’t come with any secrets built in. This means I still need to authenticate with tools like NPM and Travis. It might be interesting to manage these together with my SSH keys
• SSH host: with some tweaking it might be possible to run the same container on cloud services. Then I’d need a way to get my SSH public keys there and start an SSH server

If you have ideas on how to best implement the above, please get in touch.

0 0

The Atreus keyboard

Atreus is a 40% ergonomic mechanical keyboard designed by Phil Hagelberg. It is an open hardware design, but he also sells kits for easier construction. From the kit introduction:

The Atreus is a small mechanical keyboard that is based around the shape of the human hand. It combines the comfort of a split ergonomic keyboard with the crisp key action of mechanical switches, all while fitting into a tiny profile.

My use case was also quite travel-oriented. I wanted a small keyboard that would enable me to work with it also on the road. There are many other small-ish DIY keyboard designs like Planck and Gherkin available, but Atreus had the advantage of better ergonomics. I really liked the design of the Ergodox keyboard, and Atreus essentially is that made mobile:

I found the split halves and relatively large size (which are fantastic for stationary use at a desk) make me reluctant to use it on the lap, at a coffee shop, or on the couch, so that’s the primary use case I’ve targeted with the Atreus. It still has most of the other characteristics that make the Ergodox stand out, like mechanical Cherry switches, staggered columns instead of rows, heavy usage of the thumbs, and a hackable microcontroller with flexible firmware, but it’s dramatically smaller and lighter

I had the opportunity to try a kit-built Atreus in the Berlin Mechanical Keyboard meetup, and it felt nice. It was time to start the project.

Sourcing the parts

When building an Atreus the first decision is whether to go with the kit or hand-wire it yourself. Building from a kit is certainly easier, but since I’m a member of a hackerspace, doing a hand-wired build seemed like the way to go.

To build a custom keyboard, you need:

• Switches: in my case 37 Cherry MX blues and 5 Cherry MX blacks
• Diodes: one 1N4148 per switch
• Microcontroller: a Arduino Pro Micro on my keyboard
• Keycaps: started with recycled ones and later upgraded to DSA blanks
• Case: got a set of laset-cut steel plates

Even though Cherry — the maker of the most common mechanical key switches — is a German company, it is quite difficult to get switches in retail here. Luckily a fellow hackerspace member had just dismantled some old mechanical keyboards, and so I was able to get the switches I needed via barter.

The Cherry MX blues are tactile clicky switches that feel super-nice to type on, but are quite loud. For modifiers I went with Cherry MX blacks that are linear. This way there is quite a clear difference in feel between keys you typically hold down compared to the ones you just press.

The diodes and the microcontroller I ordered from Amazon for about 20€ total.

At first I used a set of old keycaps that I got with the switches, but once the keyboard was up and running I upgraded to a very nice set of blank DSA-profile keycaps that I ordered from AliExpress for 30€. That set came with enough keycaps that I’ll have myself covered if I ever build a second Atreus.

All put together, I think the parts ended up costing me around 100€ total.

Preparations

When I received all the parts, there were some preparation steps to be made. Since the key switches were 2nd hand, I had to start by dismantling them and removing old diodes that had been left inside some of them.

The keycaps I had gotten with the switches were super grimy, and so I ended up sending them to the washing machine. After that you could see that they were not new, but at least they were clean.

With the steel mounting plate there had been a slight misunderstading, and the plates I received were a few millimeters thicker than needed, so the switches wouldn’t “click” in place. While this could’ve been worked around with hot glue, we ended up filing the mounting holes down to the right thickness.

Wiring the keyboard

Once the mounting plate was in the right shape, I clicked the switches in and it was time to solder.

Hand-wiring keyboards is not that tricky. You have to attach a diode to each keyswitch, and then connect each row together via the diodes.

The two thumb keys are wired to be on the same column, but different rows.

Then each column is connected together via the other pin on the switches.

This is how the matrix looks like:

After these are done, connect a wire from each column, and each row to a I/O pin on the microcontroller.

If you haven’t done it earlier, this is a good stage to test all connections with a multimeter!

Firmware

After finishing the wiring, I downloaded the QMK firmware, changed the PIN mapping for how my Atreus is wired up, switched the layout to Colemak, and the keyboard was ready to go.

Don’t mind the key labels in the picture above. These are the second-hand keycaps I started with. Since then I’ve switched to blank ones.

USB-C

The default Atreus design has the USB cable connected directly to the microcontroller, meaning that you’ll have to open the case to change the cable. To mitigate that I wanted to add a USB breakout board to the project, and this being 2017, it felt right to go with USB-C.

I found some cheap USB-C breakout boards from AliExpress. Once they arrived, it was time to figure out how the spec works. Since USB-C is quite new, there are very few resources available on how to use it with microcontrollers. These tutorials were quite helpful:

• Using USB-C on hobbyist projects
• USB Type C in a Micro-B world

Here is how we ended up wiring the breakout board. After these you only have four wires to connect to the microcontroller: ground, power, and the positive and negative data pins.

This Atreus build log was useful for figuring out where to connect the USB wires on the Pro Micro. Once all was done, I had a custom, USB-C keyboard!

Next steps

Now I have the Atreus working nicely on my new standing desk. Learning Colemak is a bit painful, but the keyboard itself feels super nice!

However, I’d still like to CNC mill a proper wooden case for the keyboard. I may update this post once that happens.

I’m also considering to order an Atreus kit so I’d have a second, always packed for travel keyboard. The kit comes with a PCB, which might work better at airport security checks than the hand-wired build.

Another thing that is quite tempting is to make a custom firmware with MicroFlo. I have no complaints on how QMK works, but it’d be super-cool to use our visual programming tool to tweak the keyboard live.

Big thanks to Technomancy for the Atreus design, and to XenGi for all the help during the build!

0 0

But from April 2016 onwards I’ve been again working with an Android device. Some people have asked me about my setup, and so here is an update.

Why work on a tablet?

When I started on this path in 2013, using a tablet for “real work” was considered crazy. While every story on tablet productivity still brings out the people claiming it is not a real computer for real work, using tablets for real work is becoming more and more common.

A big contributor to this has been the plethora of work-oriented tablets and convertibles released since then. Microsoft’s popular Surface Pro line brought the PC to tablet form factor, and Apple’s iPad Pro devices gave the iPad a keyboard.

Here are couple of great posts talking about how it feels to work on an iPad:

• Rediscovering the iPad
• When traveling, my iPad is essential and my Mac is the add-on
• A Computer for Everything: One Year of iPad Pro
• Evolving iPad Desktop Usage
• My Tablet Has Stickers

With all the activity going on, one could claim using a tablet for work has been normalized. But why work on a tablet instead of a “real computer”? Here are some reasons, at least for me:

Free of legacy cruft

Desktop operating systems have become clunky. Window management. File management. Multiple ways to discover, install, and uninstall applications. Broken notification mechanisms.

With a tablet you can bypass pretty much all of that, and jump into a simpler, cleaner interface designed for the modern connected world.

I think this is also the reason driving some developers back to Linux and tiling window managers — cutting manual tweaking and staying focused.

Amazing endurance

Admittedly, laptop battery life has increased a lot since 2013. But with some manufacturers using this an excuse to ship thinner devices, tablets still win the endurance game.

With my current work tablet, I’m customarily getting 12 or more hours of usage. This means I can power through the typical long days of a startup founder without having to plug in. And when traveling, I really don’t have to care where power sockets are located on trains, airplanes, and conference centers.

Low power usage also means that I can really get a lot of more runtime by utilizing the mobile battery pack I originally bought to use with my phone. While I’ve never actually had to try this, back-of-the-envelope math claims I should be able to get a full workweek from the combo without plugging in.

Work and play

The other aspect of using a tablet is that it becomes a very nice content consumption device after I’m done working. Simply disconnect the keyboard and lean back, and the same device you used for writing software becomes a great e-reader, video player, or a gaming machine.

This combined with the battery life has meant that I’ve actually stopped carrying a Kindle with me. While an e-ink screen is still nicer to read, not needing an extra device has its benefits, especially for a frequent one-bag traveller.

The setup

I’m writing this on a Pixel C, a 10.2” Android tablet made by Google. I got the device last spring when there were developer discounts available at ramp-up to the Android 7 release, and have been using it full-time since.

Software

Surprisingly little has changed in my software use since 2013 — I still spend the most of the time writing software in either Flowhub or terminal. Here are the apps I use on daily basis:

• JuiceSSH for mosh access to my remote development servers
• Termux for local and offline development
• Flowhub for visual programming
• The Grid for updating my various websites
• Slack, Hangouts, and Inbox by Gmail for communications
• Google Drive and the associated applications for budgeting and planning
• Chrome for web

Looking back to the situation in early 2013, the biggest change is that Slack has pretty much killed work email.

Termux is a new app that has done a lot to improve the local development situation. By starting the app you get a very nice Linux chroot environment where a lot of software is only a quick apt install away.

Since much of my non-Flowhub work is done in tmux and vim, I get the exactly same working environment on both local chroot and cloud machines by simply installing my dotfiles on each of them.

Keyboard

When I’m on the road I’m using the Pixel C keyboard. This doubles as a screen protector, and provides a reasonable laptop-like typing environment. It attaches to the tablet with very strong magnets and allows a good amount of flexibility on the screen angles.

However, when stationary, no laptop keyboard compares to a real mechanical keyboard. When I’m in the office I use a Filco MiniLa Air, a bluetooth keyboard with quiet-ish Cherry MX brown switches.

This tenkeyless (60%) keyboard is extremely comfortable to type on. However, the sturdy metal case means that it is a little too big and heavy to carry on a daily basis.

In practice I’ve only taken the mechanical keyboard with me when there has been a longer trip where I know that I’ll be doing a lot of typing. To solve this, I’m actually looking to build a more compact custom mechanical keyboard so I could always have it with me. (Update: here is the keyboard I built).

Comparison with iOS

So, why work on Android instead of getting an iPad Pro? I’ve actually worked on both, and here are my reasons:

• Communications between apps: while iOS has extensions now, the ability to send data from an app to another is still a hit-or-miss. Android had intents from day one, meaning pretty much any app can talk to any other app
• Standard charging: all of my other devices charge with the same USB-C chargers and cables. iPads still use the proprietary Lightnight plug, requiring custom dongles for everything
• Standard accessories: this boils down to USB-C just like charging. With Android I can plug in a network adapter or even a mouse, and it’ll just work
• Ecosystem lock-in: we’re moving to a world where everything — from household electronics to cars — is either locked to the Apple ecosystem or following standards. I don’t want to be locked to a single vendor for everything digital
• Browser choice: with iOS you only get one web renderer, the rather dated Safari. On Android I can choose between Chrome, Firefox, or any other browser that has been ported to the platform

Of course, iOS has its own benefits. Apple has a stronger stance on privacy than Google. And there is more well-made tablet software available for iPads than Android. But when almost everything I use is available on the web, this doesn’t matter that much.

The future

As a software developer working on Android tablets, the weakest point of the platform is still that there are no browser developer tools available. This was a problem in 2013, and it is still a problem now.

From my conversations with some Chrome developers, it seems Google has very little interest in addressing this. However, there is a bright spot: the new breed of convertible Chromebooks being released now. And they run Android apps:

• How Chromebooks Are About to Totally Transform Laptop Design
• Detachable Chromebooks, Pixel C And The Future Of Chrome OS
• Samsung’s Chromebook Pro gives me hope in Chrome OS—thanks to Android’s help

Chrome OS is another clean, legacy free, modern computing interface. With these new devices you get the combination of a full desktop browser and the ability to run all Android tablet software.

The Samsung Chromebook Pro/Plus mentioned above is definitely interesting. A high-res 12” screen and a digital pen which I see as something very promising for visual programming purposes.

However, given that I already have a great mechanical keyboard, I’d love a device that shipped without an attached keyboard. We’ll see what kind of devices get out later this year.

0 0

One example is the new component API in NoFlo that has been emerging over the last year or so.

Most of the work described here was done by Vladimir Sibirov from The Grid team.

Introducing the Process API

NoFlo programs consist of graphs where different nodes are connected together. These nodes can themselves be graphs, or they can be components written in JavaScript.

A NoFlo component is simply a JavaScript module that provides a certain interface that allows NoFlo to run it. In the early days there was little convention on how to write components, but over time some conventions emerged, and with them helpers to build well-behaved components more easily.

Now with the upcoming NoFlo 0.8 release we’ve taken the best ideas from those helpers and rolled them back into the noflo.Component base class.

So, how does a component written using the Process API look like?

// Load the NoFlo interface
var noflo = require('noflo');
// Also load any other dependencies you have
var fs = require('fs');

// Implement the getComponent function that NoFlo's component loader
// uses to instantiate components to the program
exports.getComponent = function () {
  // Start by instantiating a component
  var c = new noflo.Component();

  // Provide some metadata, including icon for visual editors
  c.description = 'Reads a file from the filesystem';
  c.icon = 'file';

  // Declare the ports you want your component to have, including
  // their data types
  c.inPorts.add('in', {
    datatype: 'string'
  });
  c.outPorts.add('out', {
    datatype: 'string'
  });
  c.outPorts.add('error', {
    datatype: 'object'
  });

  // Implement the processing function that gets called when the
  // inport buffers have packets available
  c.process(function (input, output) {
    // Precondition: check that the "in" port has a data packet.
    // Not necessary for single-inport components but added here
    // for the sake of demonstration
    if (!input.hasData('in')) {
      return;
    }

    // Since the preconditions matched, we can read from the inport
    // buffer and start processing
    var filePath = input.getData('in');
    fs.readFile(filePath, 'utf-8', (err, contents) {
      // In case of errors we can just pass the error to the "error"
      // outport
      if (err) {
        output.done(err);
        return;
      }

      // Send the file contents to the "out" port
      output.send({
        out: contents
      });
      // Tell NoFlo we've finished processing
      output.done();
    });
  });

  // Finally return to component to the loader
  return c;
}

Most of this is still the same component API we’ve had for quite a while: instantiation, component metadata, port declarations. What is new is the process function and that is what we’ll focus on.

When is process called?

NoFlo components call their processing function whenever they’ve received packets to any of their regular inports.

In general any new information packets received by the component cause the process function to trigger. However, there are some exceptions:

• Non-triggering ports don’t cause the function to be called
• Ports that have been set to forward brackets don’t cause the function to be called on bracket IPs, only on data

Handling preconditions

When the processing function is called, the first job is to determine if the component has received enough data to act. These “firing rules” can be used for checking things like:

• When having multiple inports, do all of them contain data packets?
• If multiple input packets are to be processed together, are all of them available?
• If receiving a stream of packets is the complete stream available?
• Any input synchronization needs in general

The NoFlo component input handler provides methods for checking the contents of the input buffer. Each of these return a boolean if the conditions are matched:

• input.has('portname') whether an input buffer contains packets of any type
• input.hasData('portname') whether an input buffer contains data packets
• input.hasStream('portname') whether an input buffer contains at least one complete stream of packets

For convenience, has and hasData can be used to check multiple ports at the same time. For example:

// Fail precondition check unless both inports have a data packet
if (!input.hasData('in1', 'in2')) return;

For more complex checking it is also possible to pass a validation function to the has method. This function will get called for each information packet in the port(s) buffer:

// We want to process only when color is green
var validator = function (packet) {
  if (packet.data.color === 'green') {
    return true;
  }
  return false;
}
// Run all packets in in1 and in2 through the validator to
// check that our firing conditions are met
if (!input.has('in1', 'in2', validator)) return;

The firing rules should be checked in the beginning of the processing function before we start actually reading packets from the buffer. At that stage you can simply finish the run with a return.

Processing packets

Once your preconditions have been met, it is time to read packets from the buffers and start doing work with them.

For reading packets there are equivalent get functions to the has functions used above:

• input.get('portname') read the first packet from the port’s buffer
• input.getData('portname') read the first data packet, discarding preceding bracket IPs if any
• input.getStream('portname') read a whole stream of packets from the port’s buffer

For get and getStream you receive whole IP objects. For convenience, getData returns just the data payload of the data packet.

When you have read the packets you want to work with, the next step is to do whatever your component is supposed to do. Do some simple data processing, call some remote API function, or whatever. NoFlo doesn’t really care whether this is done synchronously or asynchronously.

Note: once you read packets from an inport, the component activates. After this it is necessary to finish the process by calling output.done() when you’re done.

Sending packets

While the component is active, it can send packets to any number of outports using the output.send method. This method accepts a map of port names and information packets.

output.send({
  out1: new noflo.IP('data', "some data"),
  out2: new noflo.IP('data', [1, 2, 3])
});

For data packets you can also just send the data as-is, and NoFlo will wrap it to an information packet.

Once you’ve finished processing, simply call output.done() to deactivate the component. There is also a convenience method that is a combination of send and done. This is useful for simple components:

c.process(function (input, output) {
  var data = input.getData('in');
  // We just add one to the number we received and send it out
  output.sendDone({
    out: data + 1
  });
});

In normal situations there packets are transmitted immediately. However, when working on individual packets that are part of a stream, NoFlo components keep an output buffer to ensure that packets from the stream are transmitted in original order.

Component lifecycle

In addition to making input processing easier, the other big aspect of the Process API is to help formalize NoFlo’s component and program lifecycle.

The component lifecycle is quite similar to the program lifecycle shown above. There are three states:

• Initialized: the component has been instantiated in a NoFlo graph
• Activated: the component has read some data from inport buffers and is processing it
• Deactivated: all processing has finished

Once all components in a NoFlo network have deactivated, the whole program is finished.

Components are only allowed to do work and send packets when they’re activated. They shouldn’t do any work before receiving input packets, and should not send anything after deactivating.

Generator components

Regular NoFlo components only send data associated with input packets they’ve received. One exception is generators, a class of components that can send packets whenever something happens.

Some examples of generators include:

• Network servers that listen to requests
• Components that wait for user input like mouse clicks or text entry
• Timer loops

The same rules of “only send when activated” apply also to generators. However, they can utilize the processing context to self-activate as needed:

exports.getComponent = function () {
 var c = new noflo.Component();
 c.inPorts.add('start', { datatype: 'bang' });
 c.inPorts.add('stop', { datatype: 'bang' });
 c.outPorts.add('out', { datatype: 'bang' });
 // Generators generally want to send data immediately and
 // not buffer
 c.autoOrdering = false;

 // Helper function for clearing a running timer loop
 var cleanup = function () {
   // Clear the timer
   clearInterval(c.timer.interval);
   // Then deactivate the long-running context
   c.timer.deactivate();
   c.timer = null;
 }

 // Receive the context together with input and output
 c.process(function (input, output, context) {
   if (input.hasData('start')) {
     // We've received a packet to the "start" port
     // Stop the previous interval and deactivate it, if any
     if (c.timer) {
       cleanup();
     }
     // Activate the context by reading the packet
     input.getData('start');
     // Set the activated context to component so it can
     // be deactivated from the outside
     c.timer = context
     // Start generating packets
     c.timer.interval = setInterval(function () {
       // Send a packet
       output.send({
         out: true
       });
     }, 100);
     // Since we keep the generator running we don't
     // call done here
   }

   if (input.hasData('stop')) {
     // We've received a packet to the "stop" port
     input.getData('stop');
     if (!c.timer) {
       // No timers running, we can just finish here
       output.done();
       return;
     }
     // Stop the interval and deactivate
     cleanup();
     // Also call done for this one
     output.done();
   }
 });

 // We also may need to clear the timer at network shutdown
 c.tearDown = function (callback) {
   if (c.timer) {
     // Stop the interval and deactivate
     cleanup();
   }
   c.emit('end');
   c.started = false;
   callback();
 }

 return c;
}

Time to prepare

NoFlo 0.7 included a preview version of the Process API. However, last week during the 33C3 conference we finished some tricky bits related to process lifecycle and automatic bracket forwarding that make it more useful for real-life NoFlo applications.

These improvements will land in NoFlo 0.8, due out soon.

So, if you’re maintaining a NoFlo application, now is a good time to give the git version a spin and look at porting your components to the new API. Make sure to report any issues you encounter!

We’re currently migrating all the hundred-plus NoFlo open source modules to latest build and testing process so that they can be easily updated to the new APIs when they land.

1 0

With Flow-Based Programming and the emerging Flowhub ecosystem, we can take this even further. Thanks to the FBP network protocol we can build and monitor graphs spanning multiple devices and flow-based environments.

Jon Nordby gave a Flow-Based Programming talk in FOSDEM Internet of Things track last weekend. His demo was running a FBP network comprising of three different environments that talk together. You can find the talk online.

Here are some screenshots of the different graphs.

MicroFlo running on an Arduino Microcontroller and monitoring a temperature sensor:

NoFlo running on Node.js and communicating with the Arduino over a serial port:

NoFlo running in browser and communicating with the Node.js process over WebSockets:

(click to see the full-size picture)

Taking this further

While this setup already works, as you can see the three graphs are still treated separately. The next obvious step will be to utilize the subgraph features of NoFlo UI and allow different nodes of a graph represent different runtime environments.

This way you could introspect the data passing through all the wires in a single UI window, and “zoom in” to see each individual part of the system.

The FBP ecosystem is growing all the time, with different runtimes popping up for different languages and use cases. While NoFlo’s JavaScript focus makes it part of the Universal Runtime, there are many valid scenarios where other runtimes would be useful, especially on mobile, embedded, and desktop.

Work to be done

Interoperability between them is an area we should focus on. The network protocol needs more scrutiny to ensure all scenarios are covered, and more of the FBP/dataflow systems need to integrate it.

Some steps are already being taken in this direction. After Jon’s session in FOSDEM we had a nice meetup discussing better integration between MicroFlo on microcontrollers, NoFlo on browser and server, and Lionel Landwerlin’s work on porting NoFlo to the GNOME desktop.

If you’re interested in collaborating, please get in touch!

Photo by Forrest Oliphant.

2 0

Once it launches for general use, Flowhub is the fruition of the project initiated by our successful NoFlo Development Environment Kickstarter from last August.

Flowhub will provide a development environment for your flow-based programs, allowing you to collaborate and build things inside the same interface whether you’re targeting the client-side or the server-side of the web world. And thanks to the efforts on creating a standard protocol for flow-based runtimes, Flowhub will be able to work with other environments as well. To get a glimpse of this potential, take a look at Jon Nordby’s work on Arduino programming with NoFlo UI and what Lionel Landwerlin is doing with NoFlo in the Linux desktop development world.

This is the main reason why we went with a more generic name for the new service. Flowhub aims to be the central hub for any flow-based development, not just the JavaScript-centric world where my NoFlo runtime is focused. Our Kickstarter sparked a lot of interest in flow-based programming in general, as evidenced by the various new FBP runtimes, and the upcoming book on the subject.

While NoFlo is and will remain our main focus, we should embrace the different ideas and different projects out there. One of the key points of FBP is everything can be modeled as a black box with input and output ports. With this level of abstraction, and a standard communications protocol, the different systems will be able to work together. Flowhub can be the central point for enabling that.

Collaboration as a service

NoFlo is open source, and so is the development environment we’re building for it. For developing things on your own, this is pretty much all you’ll need, especially if you’re willing to set up things like the NoFlo environment for Node.js yourself.

The role of Flowhub as a service is analogous to what GitHub provides for traditional software development. Anybody can serve git repositories and issue trackers on their own, but having a third party to take care of that gives a much simpler, smoother experience. And even more importantly, having a central point where multiple projects reside enables much better collaboration and discovery between teams and projects.

This is the essence of what Flowhub will provide:

• Hosted version of the NoFlo UI, able to communicate with multiple different FBP runtimes whether running on client or server
• Integration with GitHub’s version control and issue tracking capabilities
• Peer-to-peer environment for team collaboration based on WebRTC
• Hosted environment for running server-side NoFlo programs and their tests

As promised in our Kickstarter, the service will open to the public in the early summer of 2014. Our Kickstarter backers will gain an early access and have already service plans provided for them. With our Flowhub pre-order campaign, those who missed the Kickstarter opportunity have still a way to get in with early adopter pricing.

New technologies and a new UI

Like NoFlo itself, Flowhub is also an exploration of various new technologies. From the layout technologies and the use of Web Components, and the flow-based payment processing infrastructure handling both Bitcoin and credit card transactions that we built in NoFlo — there are many things being dogfooded on the service.

At this stage of the game there are sure to be some rough edges, but by the time Flowhub opens to the public there should’ve been enough time at both our end, and with web browsers to mature to the point where these things provide a smooth experience.

The user interface we showed in September, and user-tested in the NodeCopter NoFlo event has since seen quite a lot of improvement based on what we’ve learned. The new version has been rebuilt ground-up with Polymer and NoFlo, and is a lot more efficient and touch-friendly.

It is still not perfect, but having used it on various devices from small tablets through laptops to huge touch-screen PCs, I feel we’re definitely on the right track. Having a user interface where you can see the connections and data flows of your software in real time, and can rewire any part when needed is incredibly powerful. And once component editing is working fully with the runtimes, I can’t really see myself wanting to go back to text-only development.

Here is a sneak peek (click to see a bigger version):

To see more of what we’re building, visit flowhub.io, play with the demo we have there, and make sure to watch the intro video. Preorder today to help fund the hosted and open source version of Flowhub!

2 0

Since then we’ve been busy hacking on the user interface and other parts of NoFlo to deliver the flow-based programming experience the community deserves. This post gives an overview on some of that.

Managing your graphs

Here is how the NoFlo Development Environment looks like today, in this case editing the graph that runs a battery level instrument connected to an AR.Drone:

All the basics of graph editing are there. You can drag and drop to make connections between ports of the different nodes, and rearrange them. When dragging we emphasise the ports that are compatible with whatever you’re connecting. Ports that are not available or are of a wrong type fade out:

When you click a node you can see what component is running it, as well as edit possible Initial Information Packets to be sent to it when the graph is started:

Clicking a connection gives you a card showing the data passing through that connection in real time:

Adding new nodes can be done by dragging a component out from the library:

As is usual with modern software, there is no save button anywhere. Instead, we constantly store the changes to your browser’s LocalStorage ensuring that you never lose content. Eventually we’ll give you tools to commit the changes up to version control too.

Keyboard control

While the NoFlo Development Environment should work very nicely with tablets and other touchscreen devices, it is likely that many developers will still be using devices with a physical keyboard. For this, we’re adding a bunch of keyboard shortcuts for faster graph editing.

The UI also features a search box where you can query things inside your graphs and the available components:

In addition to textual queries, the search widget allows written commands, a feature that was inspired by Mozilla Ubiquity:

On compatible browsers you can even edit your graph by talking, though at times the results may be more funny than useful:

Talking to different FBP runtimes

NoFlo works on both Node.js and web browsers. Since a big part of the development environment is being able to inspect and modify live NoFlo graphs, we needed a way for the UI to talk to both environments. To make this easier, we defined a network protocol for FBP runtimes.

This protocol allows transmitting graph and component information across the network, as well as controlling the execution of FBP graphs and seeing what happens with them. With client-side graphs we isolate the runtime to an iframe, and with server-side we talk over WebSockets.

The runtime protocol allows us to start and stop network execution. For client-side graphs you even get a nice preview:

Here is a simple server-side graph that just reads a file and outputs its contents. You can see how we even get the output of console.log back to the UI:

Error handling is also provided. You get notifications on any errors that happen with your graphs, as well as things like lost connection to a server-side NoFlo runtime:

An interesting side effect of having a network protocol is that it doesn’t really matter to the UI whether it talks to NoFlo or to any other FBP system, as long as they are able to talk the same protocol. Various flow-based programming environments for other languages have expressed interest in our development environment, and by providing the protocol they can do this easily.

The first FBP environment to get there was Jon Nordby’s MicroFlo, a flow-based programming system for microcontrollers like Arduino. Imagine controlling hardware in a flow-based manner! Jon already programmed his fridge this way.

GoFlow, the flow-based library for the Go programming language is also planning to add support, as is the cape environment for Python.

Relevant repositories

NoFlo is open source, and so is the development environment we’re building. If you want to stay up to date on our progress, these are the GitHub repositories to follow:

• noflo/noflo-ui - the client-side development environment
• meemoo/dataflow - our graph editing widget
• noflo/noflo-ui-server - Node.js server for serving the UI and running server-side NoFlo processes

Trying it out

If you just want to play with some client-side graphs, the easiest way to use the NoFlo Development Environment is to open it in your browser. Try tweaking the clock demo for instance.

This is a snapshot of the UI that we update every now and then. For the cutting edge version, just install from git and build locally. However, we’re actively changing things all the time at this point, so the UI might be incomplete or some things might not work at times.

What happens next?

Right now our main areas of focus are:

• Better touch interaction (ticket)
• Persistent cards for keeping inspectors you need open all the time (ticket)
• Managing whole NoFlo projects instead of single, isolated graphs (ticket)

The last one is quite important, as that will open the way for implementing highly useful features like component editing in the UI, subgraph creation, and integration with version control services.

On technical level, big part right now is move to graphs and Web Components. The NoFlo user interface is based on the graph editing tools we inherited from Forrest’s Meemoo project, but we’re moving more and more of the UI logic itself into graphs managed and run by NoFlo. It is kind of cool to start getting to the stage where we can use the tool to build itself!

2 0

I ran into the concept in 2011 while trying to figure out a better way to create software. I read his canonical book on the subject, and decided to try and implement it in JavaScript. Thus, NoFlo got started.

It has been an honor to have Paul as a mentor in the process. In late June 2013 we flew to Toronto to meet him. What resulted was quite a thorough code review of NoFlo’s FBP implementation, and a video interview. The interview with Paul is finally online. Enjoy!

If you want to meet Paul in person, the next opportunity will be in the Toronto Flow-Based Programming meetup on September 11th. On the same week we also have my NoFlo talk in Zurich on the 10th, and Forrest’s talk in Berlin on the 14th.

In other news, the NoFlo Kickstarter campaign just hit 96%. Please help us get it funded so we can bring better flow-based tools to designers and programmers everywhere!

3 0

Thanks to the amazing advances in adoption of mobile platforms, this dream has more or less become true, especially in the more widespread Apple and Android ecosystems. All these devices know where they are, and developers are coming up with different smart applications to utilize this information.

The free software world has been at risk of getting left behind. GeoClue, the location framework designed for these environments was in a state of flux for a long time with very little happening to it. But now we have GeoClue2, a rewritten implementation of the original idea.

LWN has a good write-up:

Zeeshan Ali spoke about GNOME’s geo-awareness, which is undergoing a rewrite. Geo-awareness consists of four major pieces, he said. The first is geolocation, or the “where am I?” question. The second is the opposite; the user wants to find a different location: a particular address, a nearby restaurant or gas station, or other points of interest. The third issue is routing, finding the best way to get between locations. Finally, there is the user interface topic: locations, points of interest, and routes all need to be presented to the user on a map.

GeoClue2 can determine location from four different sources: coordinates from GPS devices (the most accurate), the location of nearby WiFi access points (which is accurate to just a few hundred meters), the location of 3G cellular towers (which are accurate only to a few kilometers), and IP addresses (which are accurate only down to the city level).

A major shortcoming that the new service addresses is privacy:

GeoClue2 also offers better privacy controls; the previous version of the library would provide the current location to any application; with GeoClue2, GNOME will require the user to confirm location requests from each application.

Kudos to Zeeshan and the others involved for keeping the location-aware dream alive!

I haven’t personally been involved much in the free desktop world lately. This is mainly because I’ve been busy trying to change the worlds of web publishing and software development, but also because I don’t really have a desktop at the moment. Instead, I do my work with an Android tablet and a web browser with an attached keyboard. But despite that, I hope this will be picked up not only by GNOME, but by the other Linux ecosystems as well.

0 0

Thanks to the amazing advances in adoption of mobile platforms, this dream has more or less become true, especially in the more widespread Apple and Android ecosystems. All these devices know where they are, and developers are coming up with different smart applications to utilize this information.

The free software world has been at risk of getting left behind. GeoClue, the location framework designed for these environments was in a state of flux for a long time with very little happening to it. But now we have GeoClue2, a rewritten implementation of the original idea.

LWN has a good write-up:

Zeeshan Ali spoke about GNOME’s geo-awareness, which is undergoing a rewrite. Geo-awareness consists of four major pieces, he said. The first is geolocation, or the “where am I?” question. The second is the opposite; the user wants to find a different location: a particular address, a nearby restaurant or gas station, or other points of interest. The third issue is routing, finding the best way to get between locations. Finally, there is the user interface topic: locations, points of interest, and routes all need to be presented to the user on a map.

GeoClue2 can determine location from four different sources: coordinates from GPS devices (the most accurate), the location of nearby WiFi access points (which is accurate to just a few hundred meters), the location of 3G cellular towers (which are accurate only to a few kilometers), and IP addresses (which are accurate only down to the city level).

A major shortcoming that the new service addresses is privacy:

GeoClue2 also offers better privacy controls; the previous version of the library would provide the current location to any application; with GeoClue2, GNOME will require the user to confirm location requests from each application.

Kudos to Zeeshan and the others involved for keeping the location-aware dream alive!

I haven’t personally been involved much in the free desktop world lately. This is mainly because I’ve been busy trying to change the worlds of web publishing and software development, but also because I don’t really have a desktop at the moment. Instead, I do my work with an Android tablet and a web browser with an attached keyboard. But despite that, I hope this will be picked up not only by GNOME, but by the other Linux ecosystems as well.

2 0

You’ve seen short pieces from this in the NoFlo Kickstarter video, but most of the material is new. This was shot when NoFlo turned two, and so I’m obviously spending some time talking about where things came from.

The video should work well to support the hacker’s perspective of our funding campaign. Please support us and spread the word!

It is pretty amazing to think that since my last post on Friday, we’ve almost doubled the amount of contributions. Thanks to all the nearly 500 backers who have already helped us to change the world of programming!

2 0

This is our effort to bring visual and collaborative flow-based tools into the world of mainstream software development. Similar tools are already in use in many specialized industries from movie special effects to hardware simulation, but we programmers still have to construct these complex maps of our software’s control flow inside our brains based on their textual representation. With your support, we could change that!

We’ve already reached a third of our goal on the first day. Clearly people see the need for these tools. Exciting times!

The stories from TechCrunch and GigaOm tell the story well for non-programmers. But based on the questions I’ve received today, I thought it would be good to clarify various points from a more programmer-centric point of view.

Hasn’t visual programming been tried before?

The story of visual programming tools started with the GRaIL system of the 60s, and has progressed to tools like LabView and Pure Data. So far none of these tools has reached mainstream acceptance outside of their (sometimes fanatic) industry niches.

This is partly because these tools were built originally with a particular problem domain in mind, and partly because of the user experience. Execution matters, as we’ve seen so many times in the tech industry. After all, there were tablets a lot before the iPads.

With our team I have the confidence that we have the necessary skills and vision to build something that is actually a pleasure to use, and that makes it truly easier to work with the control flow of your software than it is with the text editors.

This is not a rehash of UML. UML is a diagram mapping out object-oriented constructs, often used for code generation. NoFlo graphs instead are only the coordination layer that manages the control flow of your software. The components are still handcrafted and unit-tested code that NoFlo merely wires together at runtime, following the edges specified in a JSON file. No code generation here.

Why map out the control flow?

All software is inherently a graph. Functions call other functions, sending data along. Signals are emitted and connections are made. But outside of some debugging tools we rarely see this in a visual format. Instead, when starting to work on a program you have to parse the code and build this map in your head.

This imposes a lot of cognitive load that tools like NoFlo could avoid. When you can see visually how things are connected, you can focus on the bigger picture and build the software you need in a more efficient way. This is what Bret Victor talked about recently.

What is the role for code, then?

However, there still remains a role for text-based code. The actual components, the boxes in the graph, are still written out in JavaScript. But since they’re isolated from their surroundings until a NoFlo graph wires them together, each component can focus on accomplishing a single task well.

My original NoFlo UI prototype already included the code editor for modifying and creating new components. By the principles of TDD, each component is always edited alongside its unit tests, and the tests can be run at any point with a single click. We’re now bringing that back into the new UI:

For those who don’t want this UI, NoFlo is still fully usable also without it. As a matter of fact, we don’t have a UI before the Kickstarter succeeds, and yet many companies are already building their applications with NoFlo. One way is by using the .fbp language.

Why now?

Another reason why the NoFlo development environment may succeed where many others failed is that programming has changed.

We no longer target a single protocol — whether the Win32 API or HTTP — in our applications. Instead, we need to talk multiple protocols and with multiple devices at the same time. Even just dealing with both REST and WebSockets is difficult to many traditional programming environments.

The other big change is the usage of different web APIs as part of your applications. Authentication, handling asynchronous requests and services that are sometimes down can be a pain.

NoFlo brings a controlling layer to your software that allows you to map out these scenarios and isolate the handling of each protocol or API into its own set of small, simple components.

Where could this be used?

Many of the nodal editing tools built in the past have been very domain-specific. However, flow-based programming is a general software development paradigm. NoFlo has already been used for a wide variety of tasks, including:

• Business data extraction and reporting
• Automating billing workflows
• Receiving, routing, and sending text messages
• Static site generation
• Building server-side web applications

Now that we added browser support it will be possible to also build physics-based user interfaces with NoFlo. There should be some interesting examples of that coming up soon.

NoFlo being a general JavaScript library could in future enable us to coordinate the flows also in new kinds of places like desktop applications. There are several desktop environments where JavaScript is a first-class citizen. Combining flow-based interactions with declarative UI definitions could be something very powerful!

Many of the giants of the software industry, like Google, Facebook, Microsoft, Adobe, and Mozilla are all working on improving the JavaScript development tools. It would be awesome to have their support for what we’re doing.

How about open source?

If you’re following this blog, you probably know that I feel strongly for software freedom. Everything I’ve done during my professional career has been possible only thanks to the open source community.

NoFlo and the UI we’re building are and will remain open source available under the MIT license. As a matter of fact, the UI is designed to not only work with NoFlo, but also to be adaptable to work with other flow-based systems. I feel this is an area with a lot of potential for collaboration with the various functional and dataflow projects out there.

We will however be offering a hosted version of the software for a fee. The various Kickstarter rewards will give our backers an early and cheaper access to that.

But still, you’ll always be able to run the whole stack on your own infrastructure if you choose to do so.

What happens next?

Our Kickstarter campaign has been up for less than a day, and we’re already above 30% of our goal. This makes me quite optimistic for the effort. But of course we won’t succeed without the help of the wider open source and JavaScript community!

There are still quite a lot of days left in the campaign. Tonight I’ll be flying back to Europe, and then we’ll focus on the next steps.

One important area of attention is publishing more demos and examples of NoFlo in real-world use. I hope by early next week we’ll be able to show a few applications running on both browser and Node.js. I will also publish our flow-based port of the Jekyll static site generator. These should help people getting started with this style of programming.

We also have quite a lot of content lined up and waiting for editing. Something of interest to everybody working with flow-based or functional programming will be the long cut of the interview we made with J. Paul Morrison, the father of FBP. My hope is that video will be live by the end of the next week.

If you’re interested in the developments, make sure to follow NoFlo on Twitter, Facebook, or Google+. If you back our campaign, you’ll also receive the updates via Kickstarter.

Thanks everybody for helping to make this possible! Keep spreading the world and giving your support!

2 0

Here’s the thing: the Leap Motion is almost amazing. When using it to interact with my desktop and perform actions (clicking, dragging, scrolling), the experience is about 50 percent fluid intuition and 50 percent screaming frustration. There are moments, sometimes ten or fifteen seconds long, when everything magically clicks into place—the Leap doesn’t decide your hand is too far away or too close to be able to execute actions, and for a few seconds, boom, you’re scrolling, dragging, and clicking effortlessly. It feels totally natural. Then, almost capriciously, your gestures aren’t good enough any more and you spend ten more seconds trying to get a single click to register.

I’ve had a Leap sensor as part of their developer program for a while now, and the description above feels quite accurate to both the Leap, and our Kinect air cursor work.

Lacking interface concepts

The problem is that our current interface concepts are simply not suitable for this sort of interaction. There must be incredible things you can accomplish when the computer can see your fingers or whole body in motion. We just haven’t discovered them yet.

This is very similar to the situation we had for a long time with touchscreen devices. There have been tablets available since the 90s, but they didn’t take off until there was a new kind of OS and new kind of UI guidelines for this world. After all, your finger is not a mouse.

Once those guidelines were out there, and manufacturers built compelling enough all-touch devices, the market took off. I’ve said for a long time that the tablet form factor is winning, an estimate validated by the recent sales figures. And yet, the new kind of tablets have only been available since 2006 or 2010 depending how you look. The big shift is only getting started.

Right now tablets are still mostly used for recreation and light data processing, as productivity applications and culture always take a bit longer to adapt. We need to move beyond text as the main way of communicating ideas.

I’ve already taken a few steps that way for us programmers.

Behind the glass

While there are many great user interfaces built on the touch paradigm, interacting with a featureless piece of glass lacks a lot of the haptic feedback we get from physical interfaces. The author Douglas Adams commented on this in The Hitchhiker’s Guide to the Galaxyalready back in 1979:

A loud clatter of gunk music flooded through the Heart of Gold cabin as Zaphod searched the sub-etha radio wave bands for news of himself. The machine was rather difficult to operate. For years radios had been operated by means of pressing buttons and turning dials; then as the technology became more sophisticated the controls were made touch-sensitive–you merely had to brush the panels with your fingers; now all you had to do was wave your hand in the general direction of the components and hope. It saved a lot of muscular expenditure, of course, but meant that you had to sit infuriatingly still if you wanted to keep listening to the same program.

Zaphod waved a hand and the channel switched again.

(thanks to Ars commenter Joannemullen, I had forgotten about that quote!)

The book Invisible Computer argues for building physical appliances for different computing purposes. I don’t see that happening anytime soon given the strong onward march of ephemeralization, of moving everything to converged devices and increasingly capable software.

But even with software there would be so much more we could be doing.

Physical interaction

The devices we carry around us are filled with different sensors which could provide new kinds of interaction concepts.

For example, on many Nokia phones you’ve been able to mute the phone by flipping it around. In 2001 I had an Ericsson phone which used the proximity sensor to snooze the alarm when you waved your hand over the phone.

iOS devices use the gesture of shaking the device to shuffle a playlist or undo an action. On many Android devices you can use NFC touches to move data or interaction between devices. Touch a screen to send a picture there, or touch a speaker to make your music play from there.

These are all great, natural gestures that allow us to interact with the device in a physical way, not with just with the virtual interfaces behind the glass.

Gestures like these should be more universally available, just like touch UI concepts like pull to refresh. When people trust a gesture to work consistently everywhere, it will be used a lot more, and users will be happier.

We are after all interating with physical devices we can not only touch, but also move around.

0 0

Here’s the thing: the Leap Motion is almost amazing. When using it to interact with my desktop and perform actions (clicking, dragging, scrolling), the experience is about 50 percent fluid intuition and 50 percent screaming frustration. There are moments, sometimes ten or fifteen seconds long, when everything magically clicks into place—the Leap doesn’t decide your hand is too far away or too close to be able to execute actions, and for a few seconds, boom, you’re scrolling, dragging, and clicking effortlessly. It feels totally natural. Then, almost capriciously, your gestures aren’t good enough any more and you spend ten more seconds trying to get a single click to register.

I’ve had a Leap sensor as part of their developer program for a while now, and the description above feels quite accurate to both the Leap, and our Kinect air cursor work.

Lacking interface concepts

The problem is that our current interface concepts are simply not suitable for this sort of interaction. There must be incredible things you can accomplish when the computer can see your fingers or whole body in motion. We just haven’t discovered them yet.

This is very similar to the situation we had for a long time with touchscreen devices. There have been tablets available since the 90s, but they didn’t take off until there was a new kind of OS and new kind of UI guidelines for this world. After all, your finger is not a mouse.

Once those guidelines were out there, and manufacturers built compelling enough all-touch devices, the market took off. I’ve said for a long time that the tablet form factor is winning, an estimate validated by the recent sales figures. And yet, the new kind of tablets have only been available since 2006 or 2010 depending how you look. The big shift is only getting started.

Right now tablets are still mostly used for recreation and light data processing, as productivity applications and culture always take a bit longer to adapt. We need to move beyond text as the main way of communicating ideas.

I’ve already taken a few steps that way for us programmers.

Behind the glass

While there are many great user interfaces built on the touch paradigm, interacting with a featureless piece of glass lacks a lot of the haptic feedback we get from physical interfaces. The author Douglas Adams commented on this in The Hitchhiker’s Guide to the Galaxyalready back in 1979:

A loud clatter of gunk music flooded through the Heart of Gold cabin as Zaphod searched the sub-etha radio wave bands for news of himself. The machine was rather difficult to operate. For years radios had been operated by means of pressing buttons and turning dials; then as the technology became more sophisticated the controls were made touch-sensitive–you merely had to brush the panels with your fingers; now all you had to do was wave your hand in the general direction of the components and hope. It saved a lot of muscular expenditure, of course, but meant that you had to sit infuriatingly still if you wanted to keep listening to the same program.

Zaphod waved a hand and the channel switched again.

(thanks to Ars commenter Joannemullen, I had forgotten about that quote!)

The book Invisible Computer argues for building physical appliances for different computing purposes. I don’t see that happening anytime soon given the strong onward march of ephemeralization, of moving everything to converged devices and increasingly capable software.

But even with software there would be so much more we could be doing.

Physical interaction

The devices we carry around us are filled with different sensors which could provide new kinds of interaction concepts.

For example, on many Nokia phones you’ve been able to mute the phone by flipping it around. In 2001 I had an Ericsson phone which used the proximity sensor to snooze the alarm when you waved your hand over the phone.

iOS devices use the gesture of shaking the device to shuffle a playlist or undo an action. On many Android devices you can use NFC touches to move data or interaction between devices. Touch a screen to send a picture there, or touch a speaker to make your music play from there.

These are all great, natural gestures that allow us to interact with the device in a physical way, not with just with the virtual interfaces behind the glass.

Gestures like these should be more universally available, just like touch UI concepts like pull to refresh. When people trust a gesture to work consistently everywhere, it will be used a lot more, and users will be happier.

We are after all interating with physical devices we can not only touch, but also move around.

1 0

The number of mobile web users is already at 1.5 billion, which happens to be quite close to the total number of Internet users back in 2009.

And it is growing rapidly. In 2015 there will be an estimated 2 billion smartphone users which is quite close to the total number of Internet users currently.

In the developed world, this is likely to be a mixture of tablets, smartphones, and traditional desktop computers, with most users having at least two different web-capable devices. In the developing world, the smartphone is the computer.

Considering these statistics, it is insanity to design websites and services PC-first, with mobile only as an afterthought.

How to prepare

Just some years ago, the mobile web was a slum. Instead of getting full-featured websites, many sent us out to poorly-built and featureless m. sites. Now more and more sites go with responsive web design that makes the site itself adapt to different screen sizes and resolutions.

But even with responsive design, it is easy to go overboard. Tools like WordPress Jetpack and jQuery Mobile oversimplify the site itself by trying to make it look and feel like a native app. In the mobile-first world this is not the right way to go.

In The Rise of the Mobile-Only User content strategist Karen McGrane makes a valid point (emphasis added):

Mobile users should get the same content. It’s frustrating and confusing for them if you only give them a little bit of what you offer on your “real” website. If you try to guess which subset of your content the mobile user needs, you’re going to guess wrong. Deliver the same content as your desktop user sees. (If you think some of your content doesn’t deserve to be on mobile, guess what — it doesn’t deserve to be on the desktop either. Get rid of it.)

There is no pixel-perfect

In this new world users will access your content or software using a wildly varying set of devices. And each of them has the reasonable expectation of being able to access the full experience and the full set of features you’re providing.

This changes web design substantially. Even in the old world of different PC browsers, pixel perfect web design was rarely that. With responsive design, it is even less so.

Instead:

• Think in visual components instead of full pages. The composition of a page out of these components can vary for different screen sizes
• Design the compositions always for at three screen form factors: a full-sized desktop or tablet screen, a smartphone screen, and the 7” tablet in between
• Make your user interface elements big enough to be used on inaccurate touch screens
• Never, ever require a plugin to access some content or functionality

CSS Media Queries make responding to different form factors quite easy. And besides that, they also make it easy to optimize for the different screen densities we now have. This way your images will look sharp on anything the users have, from the “retina-class devices” to the lowest-specced Chinese smartphone, while requiring the user to only download the assets that their device can utilize.

The devices people use to access the services you provide will vary greatly not only in their display capabilities, but also in the ways you can do input. Some will have mice and physical keyboards, but an increasing amount will instead have a touchscreen. For these users, it is a big service to use the correct HTML5 input types so that the on-screen keyboards and widgets can adapt to the content being entered.

The web is not native

The web is its own platform, and as such it is foolish to try and mimic traditional desktop applications. It will never feel quite right whatever you do.

It is a lot better to accept this and fully embrace the unique advantages of the web platform:

• The web is an universal runtime that works on 100% of the computing devices your users have
• There are no gatekeepers telling what you can publish, and no middlemen taking a cut of whatever you sell online
• The web is built out of URLs that users can easily share with each other, and continue using when they switch devices
• URLs also allow any application on the web to link to any screen or state of another application
• It is just as easy to provide content as it is to provide functional applications on the web

Every major software company on the planet has their own web browser, and the competition between these is fierce. This will ensure that over time, the web will keep on getting better and faster. Compare this to traditional software platforms that can easily stagnate or get abandoned. Thanks to the standard protocols it also allows you to use any technology of your choosing for the server side of your software.

Paul Graham put it well in his The other road ahead from 2001:

And you don’t have to know if you bet on Web-based applications. No one can break that without breaking browsing. The Web may not be the only way to deliver software, but it’s one that works now and will continue to work for a long time. Web-based applications are cheap to develop, and easy for even the smallest startup to deliver.

Things are getting better

As somebody who has been developing for the web for nearly twenty years the rate the web developer experience keeps improving is sometimes dizzying.

We get used to some limitations in the stack, and then suddenly something comes along and removes that shortcoming. We’re still exploring the new kinds of designs and visual experiences technologies like Media Queries and WebGL make possible, just like it took years for the community to find best practices around things like AJAX.

And yet new amazing things keep pouring in. My personal favourite recently has been Web Components which gives a standard way to provide reusable widgets on the web, and to do things like data binding and templating. This alone will make a lot of the popular frameworks and libraries obsolete.

Just watch this video and see how much easier web development is becoming:

The current work on standardizing Web Payments together with more grassroots efforts like Bitcoin promise to add better ways to do business on the web. This should remove the last big advantage of native applications in that they’re easier (but very expensive) to monetize via app stores and in-app payments.

Linked Data on the web and new features like Yandex Islands make it easier to connect web applications and data on the web together. Tools like my Create.js make the web easier to edit, and Web Intents promise even closer integration between web apps.

Each of these will make the web richer and better. Each of them will allow new startups to be built, and new meaningful connections to happen over the internet, many of these using mobile devices.

It is an exciting time to be a web developer.

0 0

The number of mobile web users is already at 1.5 billion, which happens to be quite close to the total number of Internet users back in 2009.

And it is growing rapidly. In 2015 there will be an estimated 2 billion smartphone users which is quite close to the total number of Internet users currently.

In the developed world, this is likely to be a mixture of tablets, smartphones, and traditional desktop computers, with most users having at least two different web-capable devices. In the developing world, the smartphone is the computer.

Considering these statistics, it is insanity to design websites and services PC-first, with mobile only as an afterthought.

How to prepare

Just some years ago, the mobile web was a slum. Instead of getting full-featured websites, many sent us out to poorly-built and featureless m. sites. Now more and more sites go with responsive web design that makes the site itself adapt to different screen sizes and resolutions.

But even with responsive design, it is easy to go overboard. Tools like WordPress Jetpack and jQuery Mobile oversimplify the site itself by trying to make it look and feel like a native app. In the mobile-first world this is not the right way to go.

In The Rise of the Mobile-Only User content strategist Karen McGrane makes a valid point (emphasis added):

Mobile users should get the same content. It’s frustrating and confusing for them if you only give them a little bit of what you offer on your “real” website. If you try to guess which subset of your content the mobile user needs, you’re going to guess wrong. Deliver the same content as your desktop user sees. (If you think some of your content doesn’t deserve to be on mobile, guess what — it doesn’t deserve to be on the desktop either. Get rid of it.)

There is no pixel-perfect

In this new world users will access your content or software using a wildly varying set of devices. And each of them has the reasonable expectation of being able to access the full experience and the full set of features you’re providing.

This changes web design substantially. Even in the old world of different PC browsers, pixel perfect web design was rarely that. With responsive design, it is even less so.

Instead:

• Think in visual components instead of full pages. The composition of a page out of these components can vary for different screen sizes
• Design the compositions always for at three screen form factors: a full-sized desktop or tablet screen, a smartphone screen, and the 7” tablet in between
• Make your user interface elements big enough to be used on inaccurate touch screens
• Never, ever require a plugin to access some content or functionality

CSS Media Queries make responding to different form factors quite easy. And besides that, they also make it easy to optimize for the different screen densities we now have. This way your images will look sharp on anything the users have, from the “retina-class devices” to the lowest-specced Chinese smartphone, while requiring the user to only download the assets that their device can utilize.

The devices people use to access the services you provide will vary greatly not only in their display capabilities, but also in the ways you can do input. Some will have mice and physical keyboards, but an increasing amount will instead have a touchscreen. For these users, it is a big service to use the correct HTML5 input types so that the on-screen keyboards and widgets can adapt to the content being entered.

The web is not native

The web is its own platform, and as such it is foolish to try and mimic traditional desktop applications. It will never feel quite right whatever you do.

It is a lot better to accept this and fully embrace the unique advantages of the web platform:

• The web is an universal runtime that works on 100% of the computing devices your users have
• There are no gatekeepers telling what you can publish, and no middlemen taking a cut of whatever you sell online
• The web is built out of URLs that users can easily share with each other, and continue using when they switch devices
• URLs also allow any application on the web to link to any screen or state of another application
• It is just as easy to provide content as it is to provide functional applications on the web

Every major software company on the planet has their own web browser, and the competition between these is fierce. This will ensure that over time, the web will keep on getting better and faster. Compare this to traditional software platforms that can easily stagnate or get abandoned. Thanks to the standard protocols it also allows you to use any technology of your choosing for the server side of your software.

Paul Graham put it well in his The other road ahead from 2001:

And you don’t have to know if you bet on Web-based applications. No one can break that without breaking browsing. The Web may not be the only way to deliver software, but it’s one that works now and will continue to work for a long time. Web-based applications are cheap to develop, and easy for even the smallest startup to deliver.

Things are getting better

As somebody who has been developing for the web for nearly twenty years the rate the web developer experience keeps improving is sometimes dizzying.

We get used to some limitations in the stack, and then suddenly something comes along and removes that shortcoming. We’re still exploring the new kinds of designs and visual experiences technologies like Media Queries and WebGL make possible, just like it took years for the community to find best practices around things like AJAX.

And yet new amazing things keep pouring in. My personal favourite recently has been Web Components which gives a standard way to provide reusable widgets on the web, and to do things like data binding and templating. This alone will make a lot of the popular frameworks and libraries obsolete.

Just watch this video and see how much easier web development is becoming:

The current work on standardizing Web Payments together with more grassroots efforts like Bitcoin promise to add better ways to do business on the web. This should remove the last big advantage of native applications in that they’re easier (but very expensive) to monetize via app stores and in-app payments.

Linked Data on the web and new features like Yandex Islands make it easier to connect web applications and data on the web together. Tools like my Create.js make the web easier to edit, and Web Intents promise even closer integration between web apps.

Each of these will make the web richer and better. Each of them will allow new startups to be built, and new meaningful connections to happen over the internet, many of these using mobile devices.

It is an exciting time to be a web developer.

2 0

What I’ve been doing

I would love to tell stories of working from parks and cafes, like Mark O’Connor has on his iPad setup, but unfortunately we had a backlash of winter here in Berlin and the warm spring weather only came back this week.

Instead — quite atypically — I’ve been mostly desk-bound in this time. The EU projects that mandated a lot of travel have now ended, and my current projects are more about software development than evangelism.

But that actually makes this experiment even more useful, as it means most of the six week has been actual programming, which is what most of my readers also do.

For those who missed my setup in the previous post, this is how it looks in action:

On-screen are tmux, vim, and a Grunt test automation watcher running inside a MOSH client.

Here are some things I’ve done in the last month:

• Porting the NoFlo flow-based programming engine to run in both browser and Node.js with the same codebase, including a tutorial on how others can do the same
• Writing and publishing an implementation of Android-style Action Bars for web apps
• Adding multiple major features to the web-based NoFlo IDE
• Dealing with the issues raised with the release of jQuery UI 1.10 and Backbone 1.0.0 with Create.js and VIE
• Blogging, including publishing some quite popular posts

In general the experience has been a positive and productive one. I’ll write about some nuances here.

Web debugging

As you can see from the list above, much of my recent work has been client-side. With this, the unavailability of web debuggers on mobile browsers can become a problem.

I’ve tackled this issue in two ways:

• More tests. Instead of poking around in a debugger, I try to write Mocha tests for most aspects of my applications. This also has the benefit of automation, meaning that PhantomJS will test everything in my application every time I commit
• VNC and desktop browsers. When I really need one, I can still use the web debugging tools of traditional web browsers via VNC

See Mark O’Connor’s setup instructions for VNC on one of these tablet workstation setups.

Post-PC means post-Office

One area where tablets are really lacking is support for traditional office tools like word processors and spreadsheets. There is a Google Drive client, but it is very slow (even small spreadsheets can take minutes to load) and mostly non-functional (word processor doesn’t even support headlines).

There are also some other office suites available, but even these are better used for viewing documents than actually making changes to them.

But the bigger question is whether traditional office tools even have a place in this modern world. The commentary on constant delays with Microsoft Office for iOS and Android shows that people don’t see them as that relevant any longer:

For the longest time, Office was the ubiquitous productivity suite. Everybody used it. Nobody considered using anything else. However, since this mobile revolution started, even non-geeks are starting to question whether Office is still all that. I had breakfast this morning with a CPA who does all of his work in Google docs. There is an entire generation of future workers going through high school and college now who don’t even have Office installed on their computers. If Microsoft has any hopes of keeping Office relevant, it needs to be everywhere, including the iPad.

Personally, I might be a lot better off writing my documents in Markdown, versioning them with git, and maybe using custom data-gathering applications with CouchDB map-reduces for data visualization.

The story of tablet productivity is still evolving. The new tools and interaction techniques we have will eventually give rise to new kinds of productivity applications. That may signal the end of the Office hegemony.

Presenting from the tablet

On the first week of this experiment I was actually traveling. The final review meetings for both of the EU projects were being held in Brussels and Luxembourg, and I had to present our results.

As the presentation tools on Android are not very good, I took this as an opportunity to finally start using an HTML-based presentation system. For this, I picked DZSlides, with a custom Jekyll-based flow for constructing slide decks from individual assets stored in git.

The results are quite nice, and I love being able to embed live demos inside the slides.

With every computing platform, there is always some fiddling involved with getting your device connected to a beamer. I was positively surprised with how easily the Nexus 10 worked. Simply connect using a micro-HDMI to VGA adapter, and you’ll have the tablet screen up on the projector.

Minor annoyances

Everybody knows about the common gripes with Android on large tablets — most apps have been written with a narrow phone screen in mind, and simply look bad on a wide 10” screen. Twitter is a good example of the typical neglect of Android’s UI guidelines.

Somewhat surprisingly, this even applies to Google’s own tablet applications. Apps like Google+ and Google Drive are a lot more functional on an iPad than on a large Android tablet.

However, these are more of a problem when using something like my Nexus 10 as a media tablet, and don’t really affect how well it works as a programming workstation.

For programming work, what matters is things like the beautiful screen, Android’s reasonably good support for hardware keyboards, user-accessible file system, and the ability to share information between applications. These are the main reasons why I went with Android instead of an iPad.

However, not all is sunshine. So far, the main annoyances for me have been:

• Regressions in Magic Trackpad support mean that it is practically unusable when you also have a Bluetooth keyboard. A lot of character presses get duplicated, making typing near-impossible. I’m assuming other mouse devices would however work
• MOSH ConnectBot — which I’m using for my programming sessions — makes Ctrl and Esc keys not work. Luckily I was able to find a workaround
• Android 4.2.2 is buggy on the Nexus 10. Especially Chrome can cause the system to crash. Other browsers help here, and hopefully Google will fix the issue soon

Now, crashes and freezes may sound like a big deal. But thanks to using tmux, they just mean a short interruption, and not any lost work. I just restart my tablet or MOSH client, attach back to the tmux session I was working with, and I’m right back to where I was, cursor position, vim splits, and all.

Quantifying productivity

Calculating programming productivity is notoriusly difficult. While Mark was able to show impressive figures from his iPad setup, I don’t have anything similar because:

• I haven’t had the time to crunch the numbers on the work I do
• The ending of the EU projects meant that I’m now doing different things than I did with my laptop, and so comparing results from the two is hard

And in the end what matters more is the results of the work, not the effort spent getting there.

But still, it would be good to have a bit more data on how well the setup works besides the subjective “it feels like a good way to program”.

To this end, I recently started using the RescueTime tracker on both of my Android devices. It keeps calculates how much time I spend with different applications each day, and even allows me to give some sort of productivity scores for them:

I’ll be running this for a while, and will try to combine it with some statistics from my GitHub account. Those two should be able to paint a picture of how I work.

Conclusions

In the beginning, like with any new tool you have to start using, the Android tablet setup felt weird and limiting. But it has grown on me since, and right now I’m not regretting giving my laptop away.

But at the same time, if a new interesting device came out, the cost of switching to that would be minimal. After all, the Nexus 10 for me is essentially just a window into the web and my terminal running somewhere else.

In a way this decoupling is similar to the traditional desktop PC setup where you have a separate computer, screen, mouse, and a keyboard. The difference here is that none of those parts are bound to a desk or connected with cables. Instead, the peripherals talk with my screen over Bluetooth, and my screen with the “computer” over the Internet.

If I for instance want a better keyboard, I can just buy one and replace that part without having to change anything else with my setup.

Cost advantage

One aspect that people have remarked on is cost. Over the course of two years — which is the typical replacement cycle of a professional workstation — this setup is cheaper than a MacBook Air. And with that price I get a lot better screen and about double battery life, and even a smartphone.

What I lose is the ability to work fully offline, though somewhat alleviated by having local vim and git via Terminal IDE.

Moving forward

The experiment will keep continue. In these six weeks, I haven’t seen any negative impact on my productivity from working on an Android tablet instead of a laptop, and many positive ones. Portability, battery life, and the emphasis on tests and automation are probably the foremost.

Of course, new devices come to the market, and eventually something will come that beats the current setup. But then I’ll be able to switch without even losing my cursor position, so the only cost is the hardware itself.

In time, I will write more about how things are going.

Update: Read my Working on Android, 2017 edition post.

0 0

Many made the argument that the tablet was “just a big iPod Touch or iPhone”:

at the end of the day, the show’s centerpiece - the iPad – is just a big iPod Touch. Lots of folks will want it, in a hypothetical sort of way. But it’s hard to imagine all that many of them will fork over the initial $499 for a crippled version

The funny thing about this argument is that — while the skepticism was misplaced — the core point was true: The iPad is just a big iPhone.

Or to put it the other way, the iPhone is just a pocket-sized iPad.

Tablets of different sizes

There was a recent Time interview of the computer science legend Alan Kay. I shared it on Google+, and there was some discussion. I made the following argument:

In the end there are no phones any longer. Just tablets of different sizes, from tabletop (iPad, Surface) to pocketable (what we call smartphones)

The chances are that if you’re reading this post, you will have what you consider a smartphone. Look at it, and consider how you use it. Is it really a phone, or is the way you use it a lot closer to what you’d consider a tablet?

Back in mid-to-late 2000s, we had Nokia’s Internet Tablet devices. We considered them tablets, and did tablet-like things on them. The Nokia 770 tablet had a 4.13” screen. The iPhone has a 4” screen, and the Nexus 4 has a 4.7” screen. And yet somehow the first device was seen as a tablet, and the two latter as smartphones.

The software running on smartphones and tablets is nearly identical, as are the use cases.

Phablets meet in the middle

Smartphones and tablets are converging, quickly. Manufacturers know this, and the device-buying public is starting to see it too. But still the gadget blogs love to diss the large “phablet” devices, even though they sell quite well.

The problem here is that many technology bloggers still try to keep the two categories of devices — smartphones and tablets — separate, even though they really aren’t. Maybe these devices are two big for holding against your head for a phone call, but who really does that anymore?

In reality all of them are tablets — windows on the web — and the only difference is that some of them fit in your pocket, and others need a bag, providing a bigger canvas to see information and work on in exchange.

This is the use case I have. For most of my internet and communication needs, I’m using a Nexus 4 as a pocketable tablet. I also have a larger tablet, which I use as my workstation and for some things where a bigger screen is nicer, like graphic novels.

Some are even able to do all their work on the smaller phablets.

End of telephony

The big news here is that the telephony part of a smartphone is not going to matter for much longer. Internet-based communication tools like instant messengers, email, and hangouts provide richer ways to interact, and don’t tie you down to a specific device, or a specific telecommunications provider.

In the long run, this means an end to phone subscriptions. As I wrote on my “hacker’s toolkit”, buying prepaid data is cheaper and easier:

As the MiFi is only used for Internet access, I can buy cheap pre-paid SIMs from each country I travel to. Paying somewhere around ten euros for a month of Internet abroad certainly beats the usual roaming charges!

This is already happening. For example, Asymco reported on dropping SMS volumes in Spain:

After peaking at the end of 2008 at about €450/quarter, revenues have fallen by 60% to about €171 million in the third quarter of 2012. These figures represent almost 100% operating profit for operators so the impact is felt directly in the bottom line.

The culprit is IP-based messaging. Services like Whatsapp, iMessage and even Facebook offer “free” messaging to users who have a smartphone and a data plan. I’ve been told that 97% of Spanish smartphone users have Whatsapp installed. In some markets this “free” messaging is offered via BlackBerry Messaging.

This obviously is a development telecommunications companies are scared of. But while SMS and call volumes go down, the other change is that all these new devices will have SIM slots, and so the operators can sell a lot more of data plans.

Some of them are already targeting this new world. T-Mobile recently killed their traditional subscriptions and subsidies in favor of prepaid plans.

This is a new world

The computing world is switching to tablets rapidly. These tablets can be smaller or bigger depending on the requirements of the user, but they all are internet-connected, touch-capable and full of sensors. They already fit use cases from software development to watching media or social networking.

This will mean eventual changes in our culture:

My generation will be at something of a loss when this new world comes about. In my life, I’ve been rewarded for communicating effectively online via text. I’m a reasonably effective verbal communicator, but not nearly as good as I’ll need to be to compete with the telepresence-native adults that the children of today will grow up to be.

Today’s digital natives will be tomorrow’s telegraph operators. The only way to survive will be to understand the impact of pervasive video communication before it sweeps us under our keyboards.

PCs will still remain as the main productivity tool for some years, mainly thanks to all the legacy software built around that ecosystem. But the VisiCalc moment of tablets will come, sooner or later:

Every major shift in computing has brought its new big companies. PCs gave us Microsoft, web Google and Facebook. In the tablet space the focus has so far been on hardware and platforms, but I’m quite certain there will be winners in the productivity software space as well, companies that we may not have even heard of yet. Maybe your company is going to be one of them?

For now, I’m considering my adventures in the tablet productivity world an experiment. But day by day, my work tablet setup is feeling more and more comfortable.

Right now I don’t miss my laptop. Or having a phone number.

0 0

The privacy scare

Much of the Glass marketing focuses on the camera aspect, as does the criticism and fearmongering around it. Some feel that the device would bring about an era of ubiquitous surveillance and loss of privacy. Some even label these people as Luddites:

StopTheCyborgs and other Neo-Luddite groups make some pretty compelling arguments against AR, including the credible threat that people could be recorded at any time without their knowledge or permission. Nobody could ever consider conversation in the presence of Google Glass wearers completely private. Any entity that could tap into and process all the information from Google Glass on a minute-to-minute basis could track anyone or any conversation anywhere near the wearers. Google Glass may well begin the destruction of our current definitions of privacy.

The mislabeling aside (the historical Luddites were more concerned about machinery taking away their jobs, as they and globalization together eventually did), the criticism is generally baseless.

We live in cities filled with thousands of surveillance cameras monitoring our every move. We pay for our groceries with credit cards that leave a mark on everything you bought — when, and where — into a database. We carry smartphones that are constantly reporting their location back to the operator. We post our private pictures and current activities to social networks. There is very little privacy left. If you’d see how detailed information law enforcement can get out of our every movement from your operator, you’d know that too.

In a world where a hidden, pencil-shaped spy camera can be bought for about 30$, is a very obviously visible Google Glass device really the biggest risk to privacy? And yet, places are banning them already, a lot before they actually hit the market.

This is the future

The anti-Glass movement has a benefit. While not exactly a flying car, it is yet another sign that we’re living in the future that science fiction authors envisioned. I mean, consider the fact that there is A movement to ban cyborgs.

They even sell stickers.

Consider the possibilities

Since the privacy battle is already lost, shouldn’t we instead focus on the possibilities they can bring? While Google has shown very little of the capabilities of the Glass software apart from videoconferencing and making web searches, various books can provide clues.

Many of them describe a world where you can share with anybody what you see with your own eyes, or access any information without having to hold a bulky device.

It is a little hard to provide good context of the use cases from the books just by short snippets, but here are some. Feel free to provide your own in the comments.

In Daniel Suarez’s Daemon series, Glass-like devices are used for connecting and coordinating a full economy, called the Darknet. Here is a view of an office with augmented reality displays and motion controls:

The center of the room looked to be a staging area, bustling with your people, all wearing eyewear and gloves. To the side was a raised platformn lined with office chairs where a dozen people were grabbing, pulling, and pushing at invisible objects in the air. They were all speaking to unseen people, as though it were a call center.

I could easily imagine manipulating my flow based programs in this way.

The Daemon series also feature other Google products, like driverless cars and an augmented reality game much like Ingress.

Vernor Vinge’s Rainbows End has everybody using augmented reality glasses as an everyday matter. They even change how you see a city like Barcelona:

Vaz strolled to the stone barrier and looked down. If he blocked out all the tourism fantasy [augmented by the Glass], he could see the freight harbor almost two hundred meters below and a kilometer away. The place was an immensity of freight containers rambling this way and that, chaos. If he invoked his government powers, he could see the flow of carge, even see the security certificates…

And of course people interact with the augmented reality projections of others just like they were present:

Even when her friends were gone physically, they were often still around, invisible presences [without the Glass] like Robert’s doctors. Miri walked around the backyard talking and arguing with nobody—a parody of all the cellphone discourtesy that Robert remembered

For the less pleasant view of something like Glass, Neal Stephenson’s Snow Crash focuses on the surveillance aspects, with people using AR displays being called Gargoyles:

Gargoyles are no fun to talk to. They never finish a sentence. They are adrift in a laser-drawn world, scanning retinas in all directions, doing background checks on everyone within a thousand yards, seeing everything in visual light, infrared, millimeter-wave radar, and ultrasound all at once. You think they’re talking to you, but they’re actually poring over the credit record of some stranger on the other side of the room, or identifying the make and model of airplanes flying overhead.

Then again, talking with a Facebook addict can already be a lot like that.

How this will play out

If the glass does what people want it to do, it will become popular.

Just like we’ve taken smartphones and tablets into our lives and homes, eventually we will live in a world where you will feel naked if you step out of your home door without the glass on. How am I supposed to connect and communicate now?

I’d love to have Glass, even if it is clunky in the beginning.

Wearable computing is one of the possible trajectories we’re going with, from room-sized mainframes to desk-bound personal computers to these amazing small slabs of glass and antennas we now carry in our pockets.

Just like with smart spaces, or tablets many years ago, I’m keen to explore the new interaction possibilities.

0 0