This is the story of how I built my Android/IOIO/OpenCV-based interactive plotter, as well as the soon-to-be-released motor control library for the IOIO. I'll tell the story of the plotter from the bottom up, which is the order in which I've designed and built it. But before diving into the geeky technical details, a few words about the final product.
It started with me looking for a cool example application for the motor control library I was about to develop for the IOIO, and have this application something I can present in Maker Faire. I wanted it to demonstrate the ability to control some multi-axis machine in a simple, reliable and precise way using high-level Java code running on an Android or a PC. I finally decided on the plotter, as it seemed like a fun thing to make as well as an interesting piece to exhibit. The plotter is based on a very elegant design, which unfortunately isn't my idea (just Google for "whiteboard plotter"). Since building yet-another-one is boring, I wanted to make this one with a twist, taking advantage of the fact that I can easily put an Android device in the system. So my plotter is interactive in that you take a picture with it and it would immediately convert it to paths (via edge detection) and plot them.
Now we're ready for the geeky stuff :)
Ultra-Productive Development Environment
My friend and colleague Anton Staaf, has introduced me to a really cool development strategy he's been using on his projects. He developed this nice little "shell" library, which is essentially a simplified, very portable command shell, to which it is very easy to add new commands. I borrowed his code, and ported it to the PIC24, using the USB CDC as the underlying serial link. Shortly after, I had a IOIO board which I could plug into the USB port of my PC, open a "screen" session to and run commands to exercise whichever new features I'm working on. Combined with the device-mode bootloader of the IOIO-OTG, I was able to have super fast cycles of code-compile-flash-test. It was really fun to work like that. I'm hoping to eventually release this shell-app for the IOIO for others to hack with - it's totally awesome!
Another tool which served me really well here is the XProtoLab from Gabotronics. Since this library is all about generating perfectly synchronized, precisely-timed signals, I needed a way to validate the output signals. I don't have a scope or a digital analyzer, but the XProtoLab is a tiny, beautifully designed, tiny oscilloscope, logic analyzer and signal generator. I bought this one second hand for $30 a while back and it is worth its weight in gold. Highly recommended!
Now I was in good position to start playing around with some ideas for how my library will work, which I eventually ported into the proper IOIO app firmware.
H-A-R-D-R-E-A-L-T-I-M-E
OK, so I develop realtime software for a living, but I never before got to
this level of realtime...
I initially drafted the following design principles:
- The IOIO will play the role of a sequencer. It will have a buffer of "cues", which keeps getting filled by the client (Android or PC). Those cues are essentially "over this period of time, I want this channel to do this and that channel to do that". A channel can be for example a stepper motor pulse train, a PWM signal for a DC motor or a servo or a digital output pin.
- As usual with the IOIO library, I want this to happen with as little as possible CPU intervention, so that it can run in parallel with all the other IOIO functions. So I decided I'll use the output compare modules for generating all the pulse signals and a timer for timing the cues.
- But stepper signals are slightly tricky, as you want a precise number of steps over the period of the cue. Not one too many, not one too little. Ever. So one might generate those pulses one-by-one, but that would place a lot of burden on the CPU and would be very difficult to time correctly when multiple channels are involved. So I decided I'll just let the pulse trains run freely during a cue, and just be really really precise about stopping them at the right time, before they generate an extra pulse.
So, OK, one might think that setting a timer to interrupt at the highest possible priority should suffice, but it really doesn't, unless you want to be really way too conservative with how close you allow your last step in every cue to get to the cue point. But that would put a very serious limitation on the maximum pulse rate, which I didn't want to do.
It took my some time to convince myself that C can't cut it. You just can't really know how many cycles your code is going to take. And even if you could, this could change the next time you upgrade your compiler. So I reverted to assembly. It was actually a lot more fun than I expected, after not having done this for years. And the end-result is something I'm really really happy with! Cycle-accurate timings for everything. I know exactly when each instruction runs with respect to the output waveforms and everything is super-fast, so it doesn't place any significant overhead on the CPU. You can do up to about 30KHz signals, up to 9 such in parallel, in addition to twenty-something binary outputs (e.g. for controlling the stepper direction, solenoids, or LEDs), without ever missing a step jittering on the timing by as little as one CPU cycle.
Of course, it took me about 10 times to complete than it would have in C...
Here's a little (underground) video I shared a while ago, demonstrating some early stages of the motor control library:
Protocol Glue
From this point, bootstrapping this library to the IOIO protocol was pretty straightforward, and shortly after I had a semi-baked Java API for feeding the sequencer. The current API is pretty bare-metal, and I'm thinking about providing it with some higher level abstraction layer when I release it, or at least a decent set of handy utilities. For the time being, I developed some utilities that are specific to stepper motors, which is what I needed for the plotter.
Plotter Design
Finally came the time to make the plotter. The design is pretty simple: two pulleys driven by stepper motors controlling the length of two strings. The two strings are joined in one point attached to the carriage. A rather simple geometric transformation can tell you how long you want each of the strings to be in order to get the carriage to a given point on the sheet in XY coordinates.
On the carriage is a sharpie. In order to minimize the effect of the carriage swinging on the position of the tip of the sharpie, the hanging point, where the strings connect needs to be as close as possible to the tip. This way, even if the carriage tilts a little, the tip won't deviate by a whole lot.
I found a neat little trick for the carriage design: when drawing, the carriage is supported by two ball casters and the tip of the sharpie. This way the sharpie stays perpendicular to the sheet. In order to raise the sharpie (for moving the carriage without plotting), a third ball caster is mounted on an actuated linear bearing, driven by a small hobby servo. The servo can push this third caster into the sheet, so that it is "deeper" than the tip of the sharpie, thus causing the carriage to be supported on the three casters and the sharpie tip to float.
Oh, and remember that I just used the fancy term "linear bearing", causing you to imagine some precision machined awesomeness? Think more like a piece of a shampoo bottle pump in this case :) What can I say, I just hate waiting for parts to arrive or worry about how to fabricate the perfect bearing, when all I really care about is getting this app up and running...
Another little trick I found is for mounting the main assembly (with the motors and pulleys) on the easel: I used some square pieces of plastic, originally intended for hot cups, and bent them with the hot air gun I used for soldering. It ended up pretty cool, and I can easily unmount the thing for transportation.
And of course, the IOIO is mounted on the front, and a pair of DRV8825 stepper drivers behind it to help with the heavy-lifting. I dialed them to about 1A per motor, which seems to work well.
Now, Plot!
There's this part in a project when the hardware is pretty much done and now it's all "just a simple matter of software", as my friend Ed likes to joke. But at this point, having all these million layers of infrastructure at my disposal, I just couldn't wait to draw something with it and see if it's any good. For all I know, I might have a million problems hidden. I coded like crazy for a couple of days, until finally plotting a first circle! Well, let's call it circle-ish, since it did uncover some small mechanical issues that needed addressing. But pretty much, the entire stack of electronics and software worked flawlessly! I wrote some basic utilities to do the coordinate transformations and to expose a high-level Plotter API, which gets an arbitrary list of paths, each represented by x(t), y(t) and plots them.
Pimp Your Plotter
Since this piece is to be presented, some aesthetics can't hurt. I coincidentally noticed that the pulleys look a lot like yo-yo's and decided to have fun with this concept. My friend Ali took me for an awesome tour in his workshop and let me cut some pieces of MDF on his laser cutter. It's funny that the only precision part in this entire system is the decoration :D
Finally: The App Layer
So many layers on layers on layers, I finally had a working plotter and it was time for the application. It's been a long time since I wanted to get my hands dirty with OpenCV for Android. It is 100% pure awesome! Makes image manipulation and standard computer vision algorithms really simple to implement. I developed a simple GUI that allows you to pick a picture from your Android gallery (local storage, Web albums or capture an image from the camera), and then interactively tweak the parameters of a Canny edge detector. The detected edges are displayed in red against a grayscale image, so you can easily see what result you are going to get and keep moving the sliders until you're happy.
thinning the resulting edges, or otherwise a single edge may actually be two-pixel thick at times, which makes it really annoying to convert into paths. I used the algorithm proposed
here (thanks, guys!) and implemented it pretty easily with OpenCV. Last, I needed to trace the edges into paths. I did this step pretty dumbly, because I started to run out of time and juice. In the future, it could be pretty nice to:
Then, I found that an essential step I needed to take is
- Smooth the edges rather than just connect pixels coordinates with straight lines. A nice algorithm to borrow ideas from is here.
- Be clever about the ordering of paths within an image, so that to attempt minimize the total length of travel of the carriage. In the current implementation, the ordering is pretty stupid, causing the carriage to move from side to side way more than is necessary, significantly slowing down the process.
Aftermath
This weekend I'm going to present this project, along with some other IOIO projects in the Maker Faire Bay Area 2013. If you happen to be around, drop by to say hi. Special thanks to my good friend Al Linke, who made the video shown at the top. Al has made some awesome IOIO-based projects himself, and is going to share the booth with me in Maker Faire.
I intend to release the motor control library within a few weeks for everybody to enjoy. I hope it will pave the way for driving 3D printers and other CNC machines with a IOIO/Android combo, which seems to me like an elegant way for giving these machines a great user interface, standalone computing and connectivity on the cheap.
Some more fun pictures for those who persisted this far.
I recently backed up a Kickstarter campaign for the Espruino Pico. While being "yet another" small microcontroller-based board, this one has quite a neat software stack: it runs a JavaScript interpreter and exposes a very extensive JavaScript API for controlling its I/O. I was thinking this would make a great tool for super-quick hacks that I sometimes need for testing purposes / one-off tooling. But quickly it dawned on me that this would be a perfect way to get my son into programming! The Espruino has an IDE that is a Chrome extension with Blockly support, allowing us to code without having to do much typing and without risking syntax errors (which is often a big source of frustration for beginners). I also found that it was fairly easy to add new blocks, so I quickly made a couple of useful ones (for example, there wasn't a stock block for controlling a servo).
