Tuesday, November 5, 2013

How I Became an Artist

The IOIO Plotter strikes again!

I'll start with a story and move on to some technical detail on my cool new image processing algorithm.
A couple of weeks ago I was presenting in East Bay Maker Faire with my friend Al. I was showing off my plotter among other things, taking pictures of by-passers, making their edge-detection-portrait and giving it to them as a gift. At some point, a woman came in, introduced herself as a curator for an art and technology festival called Codame and invited us to come and present our works there.

Frankly, at this point I had no idea what Codame was and pretty much decided not to go. Fortunately, I have friends that are not as stupid as me, and a couple of days later Al drew my attention to the fact that some reading on codame.com and our favorite search engine suggested that this is going to be pretty badass.

OK, so we both said yes, and now we're a week from D-day and I decide that presenting the same thing again would be boring, and anyway edge detection is not artistic enough... I had a rough vision in my head about some random scribbles that would make sense when viewed from a distance, but didn't quite have an idea how to achieve that. But at this point I become mildly obsessed with this challenge, as well as mildly stressed by having to present the thing by the end of the week, so I gave up sleep and started trying out my rusty arsenal of computer vision and image processing tools from days long forgotten.

I got lucky! Two nights later, and there it was. Just like I imagined it!
If you step away from your monitor a few meters (or feet if you insist) the right picture will start looking like the left one. And if you zoom in, it looks like nothing really. It's all, by the way, a single long stroke, made out of straight line segments.

From this point, all that was left is to not sleep a few more nights and integrate this algorithm into my Android app that controls the plotter. Barely made it, as usual. The first end-to-end plot saw the light of day in the morning of the opening.

The event was amazing! Seriously, everyone reading this blog who happens to be around San-Francisco should check it out next time. The Codame organization is committed to giving a stage and other kinds of help to works of art that the conventional art institution refuses to acknowledge: art made by code, by circuits, by geeks and by engineers. The exhibits included a lot of video art, computer games and mind-boggling installations as well as tech-heavy performance arts (music and dance).

My plotter worked non-stop and I was on a mission to cover an entire wall with its pictures. By the end of two evenings, covered it was! I even gave away some. The reactions where very positive - watching people stand in front of this piece fascinated and smiling joyfully was worth all the hard work. The first time one of them approached me and asked "are you the artist?" I looked behind me to see who he was talking to. "No", I said, "I'm an engineer". But after the tenth time, I just smiled and said "yes" :)


Now for the geeky part of our show.

How Does It Work?

After struggling with complicated modifications on Hough transform and whatnot, the final algorithm is surprisingly simple. So much so, that I wouldn't be surprised if somebody has already invented it.
The main idea is:
  • At every step (generates one segment), generate N (say, 1000) random candiate lines within the frame. Each line can have both its points random or one point random and the other one forced to where the previous step ended, in order to generate a one-stroke drawing line the one above.
  • From those N lines, choose the one for which the average darkness it covers in the input image is the greatest.
  • Subtract this line from the input image.
  • Repeat until the input image becomes sufficiently light.
A small but important refinement is now required. If we do just that, the algorithm will helplessly try to cover every non-white pixel with black and especially will have an annoying tendency to get stuck in any dark area for too long and darken it too much. To fix that, instead of thinking of a black line as a black line of thickness 1, let's think about it as a 20% gray line of thickness 5 (or any other such reciprocal pair). The intuition behind this is that when viewed from far (or down-sampling), a thin black line darkens its entire environment slightly rather than covers a very small area in black.

In practice, an efficient way to implement this is simply to work on an input image resized at 20% and draw 20% gray lines on it. Saves a lot of computations too. The coordinates of the actual output lines can still be of arbitrarily fine resolution, limited only by the resolution of our output medium.

Source code, you way? Here: https://github.com/ytai/ScribblerDemo

Sunday, October 20, 2013

Who Wants a IOIO-OTG for $30?


No, this is picture not a negative: it's the new IOIO-OTG from SeeedStudio!

What's new?


  • It's black.
  • It's selling for $30, including male and female headers and a USB-OTG cable.
Otherwise it's exactly the same.

Why?


Since I set out to develop the IOIO-OTG I had a goal of making the end-user price be $30. For various reasons that didn't quite work out on the first attempt, but I kept pursuing this goal, with the belief that this is the right price for it. I got strong recommendations on SeeedStudio from Shenzhen, China and soon contacted them with an offer. I was not disappointed! They were very enthusiastic to collaborate and very professional. Working together with them, I closely inspected their manufacturing and testing procedures and I'm happy to report that the quality of their boards meets my highest standards, and I feel that they are very serious and committed to quality and user happiness.
We decided to make it black in order to differentiate it from the existing boards and get people's attention. I also personally love the way it came out looking.
So a big shout-out to those guys for all their hard and excellent work, and I encourage you to support them and the healthy competition they are bringing.

What's next for IOIO?

I'm now working full-steam on the next software release. It will include:
  • The motor control library mentioned on my previous post. Took me a while to get it from "working" to "polished" state.
  • The IOIO-OTG can also work as a USB slave with Android devices that support USB-OTG. That means that the IOIO can be powered by the Android without needing an external power source. This is a contribution by my friends Misha and Nadir (thanks!).
  • Some bug fixes and cleanup.
I'm also planning a small hardware revision that will add some improved protection against input voltage surges. My friends from Seeed drew my attention to a subtle problem with the current design which I'm intending to fix with that. Stay tuned!

Tuesday, May 14, 2013

IOIO Plotter and the Motor Control Library


This is the story of how I built my Android/IOIO/OpenCV-based interactive plotter, as well as the soon-to-be-released motor control library for the IOIO. I'll tell the story of the plotter from the bottom up, which is the order in which I've designed and built it. But before diving into the geeky technical details, a few words about the final product.
It started with me looking for a cool example application for the motor control library I was about to develop for the IOIO, and have this application something I can present in Maker Faire. I wanted it to demonstrate the ability to control some multi-axis machine in a simple, reliable and precise way using high-level Java code running on an Android or a PC. I finally decided on the plotter, as it seemed like a fun thing to make as well as an interesting piece to exhibit. The plotter is based on a very elegant design, which unfortunately isn't my idea (just Google for "whiteboard plotter"). Since building yet-another-one is boring, I wanted to make this one with a twist, taking advantage of the fact that I can easily put an Android device in the system. So my plotter is interactive in that you take a picture with it and it would immediately convert it to paths (via edge detection) and plot them.
Now we're ready for the geeky stuff :)

Ultra-Productive Development Environment

My friend and colleague Anton Staaf, has introduced me to a really cool development strategy he's been using on his projects. He developed this nice little "shell" library, which is essentially a simplified, very portable command shell, to which it is very easy to add new commands. I borrowed his code, and ported it to the PIC24, using the USB CDC as the underlying serial link. Shortly after, I had a IOIO board which I could plug into the USB port of my PC, open a "screen" session to and run commands to exercise whichever new features I'm working on. Combined with the device-mode bootloader of the IOIO-OTG, I was able to have super fast cycles of code-compile-flash-test. It was really fun to work like that. I'm hoping to eventually release this shell-app for the IOIO for others to hack with - it's totally awesome!
Another tool which served me really well here is the XProtoLab from Gabotronics. Since this library is all about generating perfectly synchronized, precisely-timed signals, I needed a way to validate the output signals. I don't have a scope or a digital analyzer, but the XProtoLab is a tiny, beautifully designed, tiny oscilloscope, logic analyzer and signal generator. I bought this one second hand for $30 a while back and it is worth its weight in gold. Highly recommended!
Now I was in good position to start playing around with some ideas for how my library will work, which I eventually ported into the proper IOIO app firmware.

H-A-R-D-R-E-A-L-T-I-M-E

OK, so I develop realtime software for a living, but I never before got to this level of realtime...
I initially drafted the following design principles:

  • The IOIO will play the role of a sequencer. It will have a buffer of "cues", which keeps getting filled by the client (Android or PC). Those cues are essentially "over this period of time, I want this channel to do this and that channel to do that". A channel can be for example a stepper motor pulse train, a PWM signal for a DC motor or a servo or a digital output pin.
  • As usual with the IOIO library, I want this to happen with as little as possible CPU intervention, so that it can run in parallel with all the other IOIO functions. So I decided I'll use the output compare modules for generating all the pulse signals and a timer for timing the cues.
  • But stepper signals are slightly tricky, as you want a precise number of steps over the period of the cue. Not one too many, not one too little. Ever. So one might generate those pulses one-by-one, but that would place a lot of burden on the CPU and would be very difficult to time correctly when multiple channels are involved. So I decided I'll just let the pulse trains run freely during a cue, and just be really really precise about stopping them at the right time, before they generate an extra pulse.
So, OK, one might think that setting a timer to interrupt at the highest possible priority should suffice, but it really doesn't, unless you want to be really way too conservative with how close you allow your last step in every cue to get to the cue point. But that would put a very serious limitation on the maximum pulse rate, which I didn't want to do.
It took my some time to convince myself that C can't cut it. You just can't really know how many cycles your code is going to take. And even if you could, this could change the next time you upgrade your compiler. So I reverted to assembly. It was actually a lot more fun than I expected, after not having done this for years. And the end-result is something I'm really really happy with! Cycle-accurate timings for everything. I know exactly when each instruction runs with respect to the output waveforms and everything is super-fast, so it doesn't place any significant overhead on the CPU. You can do up to about 30KHz signals, up to 9 such in parallel, in addition to twenty-something binary outputs (e.g. for controlling the stepper direction, solenoids, or LEDs), without ever missing a step jittering on the timing by as little as one CPU cycle.
Of course, it took me about 10 times to complete than it would have in C...
Here's a little (underground) video I shared a while ago, demonstrating some early stages of the motor control library:

Protocol Glue

From this point, bootstrapping this library to the IOIO protocol was pretty straightforward, and shortly after I had a semi-baked Java API for feeding the sequencer. The current API is pretty bare-metal, and I'm thinking about providing it with some higher level abstraction layer when I release it, or at least a decent set of handy utilities. For the time being, I developed some utilities that are specific to stepper motors, which is what I needed for the plotter.

Plotter Design

Finally came the time to make the plotter. The design is pretty simple: two pulleys driven by stepper motors controlling the length of two strings. The two strings are joined in one point attached to the carriage. A rather simple geometric transformation can tell you how long you want each of the strings to be in order to get the carriage to a given point on the sheet in XY coordinates.

On the carriage is a sharpie. In order to minimize the effect of the carriage swinging on the position of the tip of the sharpie, the hanging point, where the strings connect needs to be as close as possible to the tip. This way, even if the carriage tilts a little, the tip won't deviate by a whole lot.


I found a neat little trick for the carriage design: when drawing, the carriage is supported by two ball casters and the tip of the sharpie. This way the sharpie stays perpendicular to the sheet. In order to raise the sharpie (for moving the carriage without plotting), a third ball caster is mounted on an actuated linear bearing, driven by a small hobby servo. The servo can push this third caster into the sheet, so that it is "deeper" than the tip of the sharpie, thus causing the carriage to be supported on the three casters and the sharpie tip to float.
Oh, and remember that I just used the fancy term "linear bearing", causing you to imagine some precision machined awesomeness? Think more like a piece of a shampoo bottle pump in this case :) What can I say, I just hate waiting for parts to arrive or worry about how to fabricate the perfect bearing, when all I really care about is getting this app up and running...


Another little trick I found is for mounting the main assembly (with the motors and pulleys) on the easel: I used some square pieces of plastic, originally intended for hot cups, and bent them with the hot air gun I used for soldering. It ended up pretty cool, and I can easily unmount the thing for transportation.
And of course, the IOIO is mounted on the front, and a pair of DRV8825 stepper drivers behind it to help with the heavy-lifting. I dialed them to about 1A per motor, which seems to work well.


Now, Plot!

There's this part in a project when the hardware is pretty much done and now it's all "just a simple matter of software", as my friend Ed likes to joke. But at this point, having all these million layers of infrastructure at my disposal, I just couldn't wait to draw something with it and see if it's any good. For all I know, I might have a million problems hidden. I coded like crazy for a couple of days, until finally plotting a first circle! Well, let's call it circle-ish, since it did uncover some small mechanical issues that needed addressing. But pretty much, the entire stack of electronics and software worked flawlessly! I wrote some basic utilities to do the coordinate transformations and to expose a high-level Plotter API, which gets an arbitrary list of paths, each represented by x(t), y(t) and plots them.

Pimp Your Plotter



Since this piece is to be presented, some aesthetics can't hurt. I coincidentally noticed that the pulleys look a lot like yo-yo's and decided to have fun with this concept. My friend Ali took me for an awesome tour in his workshop and let me cut some pieces of MDF on his laser cutter. It's funny that the only precision part in this entire system is the decoration :D

Finally: The App Layer

So many layers on layers on layers, I finally had a working plotter and it was time for the application. It's been a long time since I wanted to get my hands dirty with OpenCV for Android. It is 100% pure awesome! Makes image manipulation and standard computer vision algorithms really simple to implement. I developed a simple GUI that allows you to pick a picture from your Android gallery (local  storage, Web albums or capture an image from the camera), and then interactively tweak the parameters of a Canny edge detector. The detected edges are displayed in red against a grayscale image, so you can easily see what result you are going to get and keep moving the sliders until you're happy.thinning the resulting edges, or otherwise a single edge may actually be two-pixel thick at times, which makes it really annoying to convert into paths. I used the algorithm proposed here (thanks, guys!) and implemented it pretty easily with OpenCV. Last, I needed to trace the edges into paths. I did this step pretty dumbly, because I started to run out of time and juice. In the future, it could be pretty nice to:

Then, I found that an essential step I needed to take is

  • Smooth the edges rather than just connect pixels coordinates with straight lines. A nice algorithm to borrow ideas from is here.
  • Be clever about the ordering of paths within an image, so that to attempt minimize the total length of travel of the carriage. In the current implementation, the ordering is pretty stupid, causing the carriage to move from side to side way more than is necessary, significantly slowing down the process.

Aftermath

This weekend I'm going to present this project, along with some other IOIO projects in the Maker Faire Bay Area 2013. If you happen to be around, drop by to say hi. Special thanks to my good friend Al Linke, who made the video shown at the top. Al has made some awesome IOIO-based projects himself, and is going to share the booth with me in Maker Faire.

I intend to release the motor control library within a few weeks for everybody to enjoy. I hope it will pave the way for driving 3D printers and other CNC machines with a IOIO/Android combo, which seems to me like an elegant way for giving these machines a great user interface, standalone computing and connectivity on the cheap.

Some more fun pictures for those who persisted this far.

Friday, January 25, 2013

Go, Go, IOIO-On-The-Go!

A year and nine months passed since the first announcement on the IOIO. Eight months flew by since my announcement on the upcoming IOIO-OTG. Almost a year since I started development. Waaaaaay longer than planned. Everything that could have gone wrong went wrong. Hell, even some things that couldn't have gone wrong went wrong! But finally:

IOIO-OTG is HERE!!!

Let's start from scratch, as for some of the readers (who've spent the last two years on Mars) this might be your first encounter.

I/O for the I/O-less

The IOIO-OTG is a printed circuit board for electronics hobbyists and prototypers, which addresses a very common problem: how do I use my {computer, tablet, phone} to control my {robot, dish-washer, cat-feeder, etc.}. The original IOIO board was the first to offer a complete solution to this problem for Android devices. The IOIO-OTG adds PC support (Windows, Linux, OSX).

Unlike most other solutions, requiring you to write two pieces of software (board and client), at least one of which is typically complicated, the IOIO takes a different approach, by only requiring you to write the client-side (Android, PC) software, using a high-level programming language (currently Java) with a rich and intuitive API that allows you to manipulate the I/O pins and hardware peripherals on the board, giving you the experience that those I/O capabilities are actually an integral part of your client machine. This feature of the IOIO makes prototyping and development with it very fast and simple. Another great selling point is that it can work with virtually any Android device, even very old ones (Android 1.5 or higher), while most other boards would only work with the latest and greatest.

You can connect the IOIO to its client either over USB or over Bluetooth (using a standard dongle). The pre-installed firmware and the provided software library completely hide away the gory details of the underlying connection. The same code you write will work over either connection type seamlessly.

Smaller, Cheaper, Stronger

The IOIO-OTG is significantly smaller in size and/or cheaper and/or richer in features than alternative solutions. For example, in the Android world, the IOIO-OTG's I/O specs are comparable with an Arduino Mega ADK at a fraction of the size and cost and with all the software advantages mentioned above. In the PC world it would provide an attractive competition as well to boards of similar capabilities, and once again, the savings on software development time are huge.

To make it super-loud-and-clear: I'm not making a case that IOIO-OTG is better than Arduino or vice-versa. Arduino is awesome for what it's good at, which is IMHO for standalone operation. The IOIO boards were never intended to work standalone, but excel when it comes to offering I/O to an existing machine that lacks it. There is an ever increasing number of applications that fall into this category, especially due to the attractiveness of using Android devices in physical computing applications, getting tons of sensors, internet connectivity, lots of computation power, touch screen, etc. in a cheap, easy to use package.

Constantly Improving

The IOIO-OTG is all open-source, software, firmware and hardware. The development has been an ongoing process, with new features and bug fixes introduced on a regular basis. New versions of firmware are distributed in a way that makes it very simple for the user to upgrade: connect the IOIO-OTG to a PC and run a simple program to flash the latest version, or your own custom one if you're into such adventures. Download the software package with libraries and examples from the website and you're good to go.

Hardware Specs

That's what we're here for, aren't we? Here are the main features:
  • USB-OTG dual-role (host, device).
  • Input voltage: 5V-15V, from external source or through USB (when connected to a computer).
  • Output voltage: 5V, up to 3A (!), 3.3V, up to 500mA.
  • 46 I/O pins (digital I/O), built-in pull-ups / pull-downs / open-drain on all pins.
  • 16 Analog inputs.
  • 9 PWM (for driving servos, DC motors, dimming LEDs, etc).
  • 4 UART.
  • 3 TWI (I2C, SMBUS).
  • 3 SPI.
  • 6 Pulse Input (precise pulse-width / frequency measurement).
  • USB current limiting when acting as USB host (useful in Android mode).
  • Switch for forcing host mode (for using non-standard USB cables, which are more common than the standard ones...)
  • On-board LED under user control.
For those who know the IOIO V1, the main changes are the dual-role stuff, the beefier 5V regulator and two less I/O pins. There's also improved circuitry for cleaner analog input and better protection against user error (you can still fry it if you really want to :D)

Cutting Costs

My main goal in this project is to make it available and useful for as many people as possible. Really. No marketing BS here. This is not my day-job. One of the key factors in meeting this goal (assuming the product is great, of course) is making it affordable for as many people as possible. Making it cheaper also means people will be less frustrated if they happen to fry the board, and make people feel more comfortable leaving the board permanently attached to their project and buying a new one for the next project.

In that, I feel that I have yet to improve. We set our goal at $30 end-user price. It is currently about $40, and during the struggle to cut costs I decided to forego my own royalties of the product. I'm not particularly happy about it, to be honest, but I feel like I have done the right thing. I'm currently considering my options for how to reduce the end-user price while leaving a little something for myself too.

Coming Up

  • Raspberry Pi support. Already have a working prototype, but need to polish.
  • More applicative features: capacitive sensing, extended stepper motor control, encoder interface.

Thanks

Luckily, I didn't do this alone!
Aaron Weiss from SparkFun took part of the board design and walked me through the winding road of getting a product out the door.
David Stadler designed the beautiful new graphics, which preserves (and improves) the yo-yo from the IOIO, but gets rid of the only-for-Android feel.
Kustaa Nyholm contributed his PureJavaComm library to the community, which opened the door for easy PC integration.
All the IOIO users, who are slowly turning into a nice community, gave me all the inspiration and motivation to keep working when things went the hard way.
My dear wife, kids and friends, who gave me the huge amount of support required for such a project and for patiently listening to my boring geeky stories all along.

Read More

Main IOIO homepage: https://github.com/ytai/ioio/wiki
IOIO project gallery (links I'm collecting): http://pinterest.com/ytaibt/ioio

Sunday, May 27, 2012

The Second Generation of IOIO is in the Works


It's been a little over a year since IOIO has been released. During this period it has been used by thousands of users world-wide, who published tens of amazing projects. IOIO has become the first and leading product for interfacing Android with external hardware. Several software / firmware upgrades have been successfully rolled out, which added new features and fixed bugs. Two different manufacturers (SparkFun, Jaycon Systems) currently make IOIOs and an alternative form factor boards from SeeedStudio are just starting to sell. A book has been published on making Android accessories with IOIO. I've given a few workshops on IOIO, one of which at the MIT Media Lab, which has always been a dream place for me. Quite a trip! I was expecting something much more modest when I started, but happily jumped on the train.

Why a New Version?

During the whole time, I've been constantly gathering user feedback and looking at other products in the same field and have kept asking myself "what is the most important thing to do next?". Eventually, two main points started to emerge, that could not be addressed by software alone:

  1. I find that the paradigm of controlling I/O pins remotely (i.e. from an off-board processor) using a high-level Java API works really well. It has proven really great when you can seamlessly move your connection from wired to wireless or when you can easily integrate the capabilities of the host (Android) with the capabilities that IOIO adds. It has proven great when users with Java-only background could now easily be able to communicate with hardware, without needing to write any embedded code and without needing to develop a communication protocol themselves. I believe this paradigm can be just as useful for the PC world. Currently, the standard way to control I/O from a PC is to use an Arduino or a similar board, and having to write two separate programs which communicate with each other. The bandwidth in this case would be typically limited to 115Kb/s, which is far less than what USB is capable of.
  2. IOIO is too expensive. Not for business reasons, because there's hardly any competition, and making it cheaper might actually decrease net revenue. But my goal is increasing the number of units sold and the number of happy users. I'd consider this project a success if IOIO became something everybody knows and loves, and considered the standard and obvious solution for enhancing a host computer with I/O capabilities.
It became apparent that there's enough motivation for a second generation. And so I started! SparkFun were as great as always, and have been very supportive on both goals.

So... What's New?

The next generation of IOIO will be a USB on-the-go (OTG) device. What this means in practice, is that the new IOIO will be capable of acting either as a USB host (like the current IOIO) or as a USB device (allowing it to connect to a PC as well as be powered by it). Moreover, it will be able to auto-detect which is the right role, according to whichever cable is connected to it (a micro-A or micro-B).
Making the IOIO an OTG device required some modifications to the power-supply module. While I'm at it, I've managed to design a module that is both cheaper and beefier (2A) than the current one. It will also have a resettable protection fuse and will handle current limiting more elegantly than the current IOIO (which simply has a resistor on the VBUS line).
I considered upgrading the microcontroller, but was happy to find out after searching a little, that the current one (PIC24F) is one of the most peripheral-rich micros out there if not the richest. Since this is the most important aspect of the microcontroller for this application, I decided to leave it alone. Other than that, I've made a few more nice-to-have changes, such as reducing the noise on the analog inputs and reorganizing the power supply pins in a more accessible way (having GND next to each supply).

The Development Process

In order to save time and money, and according to my beloved tradition, I made the first prototype by home-etching. In this case, I took an old IOIO and gave it a heart transplant to replace all that needed replacing. I wish there was an "ugly but works" contest...

Then it's software time! The firmware took some time to develop, but it is now close to completion. The IOIOLib part was the most fun. It ported from Android Java to PC Java totally smoothly. Finally, just in time for Maker Faire, I finished a fully functional demo, in which I'm running HelloIOIO on a PC, and controlling the IOIO LED over USB or Bluetooth!
SparkFun also moved quickly, and provided me with the "pretty but doesn't work" prototype pictured at the top of this post. The "doesn't work" part is probably the fault of my soldering, though :D. But it was working enough for demonstrating the new USB device functionality at Maker Faire.
So we're probably a couple of months away from production and official launch. It is possible that the initial release will ship new hardware, but with software only capable of USB host and I'll release the device mode in Beta first. Not sure yet. We shall see depending on my estimate of the risk at the time of launch.

Thanks!

I'd like to thank all of the users for their trust, support and feedback; to thank SparkFun for being the most awesome partners on the planet and most of all to thank for my family for putting up with my crazy hobbies and excessive work hours.

Friday, October 28, 2011

IOIO Over Bluetooth (or: Who Needs Cables Anyway?)

Pheeeew.... a few long weeks of crunch-mode right about when I moved to California and then to a new house. However, I felt I had to get this done and the Android Open conference in San-Francisco seemed like a good target date. I made it. Barely, but definitely made it.

OK, now that I got it off my chest, I can tell you what it's all about.

The Short Story

With a firmware upgrade on the IOIO, it now supports connecting a standard Bluetooth dongle into its USB jack and is able to establish its connection to the Android phone wirelessly! This actually makes IOIO one of the cheapest, simplest and most powerful Bluetooth-enabled prototyping platforms out there. And some more good news: your application code stays exactly the same. That's the way it should be as far as I'm concerned. End-users should care about what they want to do with their hardware for their project, not about how the heck (or how the hack) to communicate with it. So you only need to write the application-specific code (the source code for the application demonstrated above takes less than 30 lines of Java for the IOIO-related stuff), and it seamlessly works on any kind of connection and can even switch from one to another while running. I don't know of any existing platforms that will let you do that so easily and cheaply. The closest one probably being Amarino. Keep in mind that IOIO is also capable of USB connectivity to Android of course (ADB or OpenAccessory), giving superior reliability latency and bandwidth compared to Bluetooth. You do the comparison.

The Long Story

Although I think there is some real kick-ass little revolution here, this post is going to be more of a story than my usual bunch of technical specs. I'm just in this kind of mood more than the check-out-this-awesome-stuff mood.
Back when I published my original announcement on IOIO, one of the commenters (Inopia, thanks, man!) cleverly noted that since IOIO is a USB host, using a standard Bluetooth dongle in order to make the connection wireless is just a matter of writing the right firmware. He was right! And I immediately fell in love with the idea and with the challenge it presented. I felt that from all the million features I could develop next, this one will be the real killer. Just imagine: a couple of bucks (cheapest dongle I found is $1.80 including shipping from DealExtreme) and you have yourself a whole new range of possibilities: home automation, R/C toys, and much more.
Slowly I began to realize some really cool side-effects that this will have. First, the current inability (or more precisely, complexity) to use IOIO and debug your Android at the same time would go away. Second, we're no longer limited to an Android - control IOIO from any Bluetooth-enabled device: IOIOLib is just a bunch of Java code that can easily be ported to other platforms (or rewritten if need be). Third, we're no longer limited to just one IOIO controlled by a single host application.
You get the point. I just had to do it. One problem: I don't know Jack about Bluetooth. Only know enough to know that it's definitely doable. That's where the second key actor in this story comes in. I'm digging the Web for open-source Bluetooth stack implementations. Pretty soon I come across btstack, developed by Matthias Ringwald. I also found other options, and at that point, I was not completely sure which one to choose. So I contact Matthias and I start checking out the code of several projects, and throw some of them away for lack of maintenance and others for having Spaghetti code. But btstack turns out to be just perfect. Easy to port, very clean code, doesn't use the memory heap (embedded heaven), active maintenance and great discussion and support forum. Matthias really got it right (at least my idea of getting this sort of things right). Two nights later (mostly struggling with implementing the USB driver for the dongle), and I'm able to open an end-to-end connection from my phone to the IOIO. Then a few weeks of finding these tiny, cruel bugs and getting everything nicely packaged and documented, etc.
And as I said, not a moment too soon! I got to Android Open two days after having a working demo. There I met Aaron Weiss from SparkFun face-to-face for the first time. Aaron is the engineer from SparkFun's side that worked on IOIO from day one. Meeting him and having him stand next to me while presenting was really great!
At the conference, I attended a keynote by Massimo Banzi, one of the two founders of Arduino. I really admire his work, especially after having taught a course on Arduino that enabled non-techie designers build the most awesome stuff. Quite a great keynote he gave, and a little later I've had the honor of presenting myself and inviting him to see my demo. And he came, and was so kind and positive and that really meant a lot to me.

Next exciting event was an interview by Make magazine folks. Needless to say I admire their work too. I think they honestly liked my Bluetooth demo and agreed that this is a little breakthrough in the field.

The moment I came home after the conference I fell ill for a few days. Totally exhausted. Haven't had a decent sleep in a few weeks. I took a few days off, and then back to work: a demo is nice, but I gotta get this thing released. Fortunately, when preparing the demo I wasn't playing quick 'n' dirty. So I just needed some polish, but no throw-away code. And finally, I'm happy with it and feel confident enough releasing it. It's not perfect-perfect, as multi-device support still needs some care. But it's reliable and definitely useful as-is. I made a video explaining users how to upgrade their IOIO to the new feature, building on top of the firmware upgrade capabilities that I previously enabled. Some have already reported success.

Links

More information (and the instructional video) can be found here.
IOIO can be purchased from SparkFun (about $50) here.
The cheapest ($1.80 incl. shipping) Bluetooth dongle I found and tested is here.
Questions are happily answered on the ioio-users discussion group.

What's next?

There are several possible directions I'm considering (your comments welcome):
  • Supporting the multi-IOIO use-case properly.
  • Supporting WiFi dongle (imagine that!).
  • Releasing OpenAccessory support in non-Beta mode (now the ground is properly laid, with new bootloader and connection abstraction layers).
  • Adding long overdue features that users requested such as infrared remote control protocol, synchronous parallel I/O, quadrature encoder, PPM output, etc.
Tough choice. All seem to add great value. We shall see...

Wednesday, July 13, 2011

IOIO Manager Unleashes the Power of Firmware Upgrades

Exciting news once again, folks!
Since the original design of IOIO, it was planned to make it capable of upgrading its own software, by pulling updates from the Android device it is connected to. This gives a lot of flexibility for where the upgrades come from, be it the Web, a file containing custom firmware that a user has built or even embedded inside an Android application that needs the IOIO to behave in some non-standard way.
To achieve that, the IOIO firmware included a bootloader, which kicks in on every IOIO restart and checks for new firmware. For security purposes, it has been decided that the bootloader will only get its upgrades from a single application, called the IOIO Manager, and check that this application is authentic by enforcing it to be signed be the official IOIO key (i.e. by me). Alas, this IOIO Manager had not yet been born when IOIO was first released, and this cool feature, although available on every IOIO firmware, remained dormant. Until yesterday, that is!
The IOIO Manager application is now available on the Android Market. Install it by scanning the QR code below or by clicking it:

Installing New Application Firmware Using the Bootloader

This is the most common use case: the IOIO Manager application lets you maintain a library of application images, and then choose the one you want installed on the IOIO at any point in time. Once selected, the IOIO will pull this image after its next restart. This takes less than two seconds and does not require any additional hardware (such as a programmer).
To remove all doubt: this upgrade mode works great with the existing bootloader version, which all current IOIO owners have. At least in the near future, the bootloader is not expected to change, and those users are able to use IOIO manager to receive upgrades!

Upgrading the Bootloader

The bootloader cannot upgrade itself by design. This is in order to protect the user from accidentally bricking the board by installing a faulty image that renders the bootloader invalid. My original thought was that the bootloader will never have to be modified anyway. Errrrr, wrong! When designing the bootloader I figured that since the bootloader already contains all the code for establishing an ADB connection to the phone, it might as well provide this service to the application and save the code duplication. While this seemed reasonable enough back then, I overlooked something important - what if the application wants to use some other protocol over USB? One example is of course Open Accessory. Moreover, what if I found a bug in the bootloader, or want to introduce an optimization? Bottom line - we need a way to upgrade the bootloader from time to time and for that we must have a programmer.
Wait, what?
But the SparkFun page specifically used to say "you'll never need a programmer", until we figured out this is true only 99% of the time. And yes, we could just fix the bootloader design bug and have the new units shipped with a better bootloader, but what about the users that already bought it and don't have a programmer? I wanted to be as fair as possible to these people who were first to believe in my product, so I decided to go the extra mile...
So you need a programmer, no doubt about that, but who said you need an off-the-shelf programmer? Let's make one out of a IOIO! And so I did. And it works! So right now, if you want to reprogram your IOIO's bootloader, all you need is another IOIO, be it your friend's or your fellow hacker. Well, people can still argue this is not a free solution in all cases, but I hope they will appreciate I really did anything I could to make things better for them.
Here's what it looks like when one IOIO programs another:
Simply connect 5 wires (two of which are supply and ground) between them, select the image and go!

New Possibilities

First, this new feature opened the gate for me to start pushing new features to IOIO on a regular basis. Some of which have been itching me to release for quite some time.
Next, this allows people interested in modifying the code running on the IOIO to do so very easily as well as to distribute the firmware they are producing for others to install with no trouble. I'm really curious to see what will come out of this.
For more information, read the detailed guide to the IOIO Manager on the IOIO Wiki.

P.S. in retrospect I realized this is a homage to an older project of mine: AVR programmer using the PICMAN (the great-grandfather of the IOIO)