Teaser: Makeblock Robot Starter Kit Cellbot + Sensors + shxtty Path Tracking

Well this was fun. Ingredients:

  • Makeblock Bluetooth module
    To execute instructions from smartphone.

  • LG Optimus Dynamic
    Strapped on robot for sensors. WiFi to talk with Mission Control. Bluetooth to control robot.

  • Tornado server
    Runs on PC. Connects to phone via SL4A. Talks with web UI using websockets.

  • Websockets
    To stream sensor data from phone to web UI. Also relays commands from web UI to robot.

  • Web UI
    Basic webpage with Javascript to handle keyboard events and draw things

  • D3.js
    For rendering the accelerometer G-meter (not shown in video)

  • HTML Canvas
    I just used a 2-D grid and rectangles (fillRect) to draw the robot’s snail trail.

  • Physics
    v = d/t; d = v*t
    Estimate distance traveled by robot using motor “velocity” and time.

  • Ye Olde Trigonometry
    To convert bearing + distance traveled to x-y position in room.

You know, this could make a good school project. Lots of applied basic science, math and programming.

I promise to put up details and code in a later post 🙂

(If I forget, please email aaron@secretsciencelab.com and I’ll share everything with you.)

How to fix SL4A webCamFacade exception

On some phones, if you try starting the webcam using SL4A, you will get the following exception:

java.lang.RuntimeException: setParameters failed

Yuck!

Thankfully, the fix is simple. I found Paola Garcia‘s solution everywhere I looked on the internet. She was cheerfully answering everyone’s questions all over the place. She said she fixed the problem by changing the preview format from JPEG to NV21:

NV21

Thanks Paola!

But while the fix is simple, the hard part is you have to build a new SL4A app (APK) with the fix. Good news! It’s not really that hard because someone was kind enough to leave instructions. It was a good learning experience for me.

First, follow the first 2/3rds of this post:
http://jokar-johnk.blogspot.com.es/2011/02/how-to-make-android-app-with-sl4a.html
Do only the “compile” steps. The last “how to package your app” part doesn’t really apply to you, since you just want to build SL4A.

I’m writing this post because I ran into some problems using the instructions above. I’m sharing my solutions in case anyone else runs into them too. I think most of the problems are because the post was written in 2011, and the latest libraries and SDKs in 2014 don’t play as nicely together.

The step you’ll want to substitute is when you Import your project source code. Instead of the “official” SL4A, Use hg to clone the SL4A repository from Paola Garcia‘s branch:

hg clone https://olapaola@code.google.com/r/olapaola-android-scripting/

(FYI: I used Tortoise Hg on Windows.)

Now here are the problems I ran into and how I fixed them. (Just in case anyone else runs into them too.)

  1. Missing Android SDK files when compiling
    SL4A needs APIs from the older Android SDK platforms. Check C:\Users\yourname\AppData\Local\Android\sdk\platforms. If you’re missing the directories android-3, android-4, android-7, android-8, you need to download/install them.

    In Eclipse: Window -> Android SDK Manager. Select all the Android SDKs til API Level 8 (Android 2.2).

    Download & install

    Download & install

  2. conversion to dalvik format failed with error 1
    I got this when trying to export the APK after compiling. I couldn’t figure out what the problem was for a while because stupid Eclipse just gave me the error message with no console logs. I finally found the problem when I did “Project -> Run As…” It gave me a helpful error message I could debug. Something about the JDK libraries was conflicting. I made two changes:

    Download JDK 1.6.0_26 and select it here

    Download JDK 1.6.0_26 and select it here

    Drop the Java compiler to 1.6

    Drop the Java compiler to 1.6

  3. Export APK
    I’m an Android noob so this step tripped me up too.
    Export signed package

    Export a signed package. Create a new keystore. I filled in blanks with B.S. and set passwords to “android”.

Finally, out pops your fresh APK with WebCamFacade fixed. Transfer the APK to your phone, uninstall the old SL4A app and install this new one.

Happy hacking! If these instructions were helpful, leave a comment below. Let us know what silly things you used your camera for. I know I have my eye on OpenCV 🙂

How to build a Makeblock Bluetooth/WiFi Cellbot car

Ingredients:
– LG Optimus Dynamic
– PC game controller (joystick)
– pygame for joystick input
– python script runs on PC to translate joystick input to movement commands
– SL4A to send movement commands from PC to smartphone over WiFi
– smartphone sends movement commands to Arduino via Bluetooth (also SL4A)
– Arduino translates commands to movement

First, we taught Bao to listen with Bluetooth

That was using pyserial, connecting directly from PC to Arduino via COM17.

I used that to build a basic instruction set to control movement. E.g.,:
!fwd 10
!bwd 5
!left 3
!right 10
!fwdleft 10 5
!bwdright 6 6
!stop

Those instructions are parsed and executed on the Arduino. The numbers tell how much the joystick was deflected.

But it wasn’t very interesting to connect directly from PC to robot via Bluetooth. I wanted to strap my smartphone on to add that extra layer of sensors and computing power. So my scheme was for my PC to talk to my smartphone and for the smartphone to be like the Jewel Wasp, controlling the robot via Bluetooth.

This was the main Python code running on my PC:

And this was what I had on the Arduino:

The main changes were to handle commands sent via Bluetooth. Bytes were read one at a time and accumulated into a buffer. The buffer was parsed for commands and the commands were executed. I had to tune the vectors a little to take advantage of the analog joystick input and to smooth out transitions to/from any direction.

I rigged the smartphone onto the robot with Legos. Then I downloaded and streamed the robot’s POV using the “IP Webcam” app because why not?

It’s ALIVE! Robot POV

Pilot POV

It worked better than I expected. The controller lag was not too bad, even with the inefficient command string processing scheme.

Tee hee

When I was done playing I restored it back to the old IR + Ultrasonic setup. I still haven’t figured out what’s the best next step from IR to Bluetooth for a kid.

LG Optimus Dynamic – perfect cellbot phone?

It is the cheapest, full-featured Android phone that I’ve found. Only $20 on Amazon NEW. It feels like finding hidden treasure because the only reason it’s so cheap is that the phone is targeted for the bottom market. And, normal people who buy the phone later add minutes to use it as a phone. But not us! If everyone did what we want to do with the phone, they’d lose money because there’s no way this package can be produced and sold for a profit at $20. So shhhh… let’s just keep this between you and me.

Here’s what you do after you get it:

  1. (optional) Root it. That link takes you to a super simple one-click script and all you need is a PC and the USB cable that came with the phone. I don’t know why I rooted mine, but it’s Android and you can, so why not? You could install this custom ROM too if you want (I didn’t). And if you screw something up like I did, you always go back to “stock” with this.
  2. Install SL4A — the Scripting Layer for Android. This lets you interface with Android and everything your phone’s hardware allows… from a Python script.
    * Install the APK
    * Menu > view > interpreters
    * Menu > add > Python
    * Install the Python APK > OPEN the Python app > Install Python (+ whatever modules you want, like pySerial + pyBluez)

  3. In SL4A, go to Menu > view > interpreters > Menu > start server
    (Note the IP address and port, e.g. 192.168.1.3:12345)

  4. From your PC, write a simple Python script called hello.py to talk to the server on your phone:
    hello.py

  5. Notice the “import android” above. That’s from here. Download android.py and put it next to hello.py above.
  6. python hello.py
  7. CHEER

The hardware isn’t great and it’s still on Gingerbread, but I’m pretty sure it’s still more powerful than NASA’s Lunar Lander. It’s just a steal at $20 for the nice package. Battery life is super and everything works fine. It even has an external micro SD slot! Oh one thing you might want to upgrade for our example above is the Text-to-Speech voice. Gingerbread is old so you don’t get Google’s latest TTS software. So I downloaded IVONA. It’s free, it works for Gingerbread and I like Amy’s UK English voice.

By the way, the SL4A website taught me that I’m not the first brilliant person to think of using smartphones for robots. And, the term for robots that use smartphones is Cellbots. But it still makes me wonder why people still want to mess with things like a $30 XBee WiFi module. Especially when phones like the Optimus Dynamic gives you WiFi (plus everything else) for $20. Who knows.

Happy Hacking!

Robot Kit – Day 7

Our new ultrasonic car + IR code seems to work well, even in the living room. You only wanted to switch to MAX SPEED to watch it go nuts:

We made an interesting discovery: mama’s fuzzy bunny slippers are invisible to the ultrasonic sensor. I tried to demonstrate why by bouncing some balls off different surfaces. But I’m not sure if you got it.

PC to android smartphone recipe

Bao has been taking a break. I think having a robot that moves on its own stresses you out. You’ve been running to flip the off switch after playing with Bao a little. Maybe it’s too much for a 3 year old to deal with. On the plus side, now you know the deal with batteries and recharging, and how Bao gets slow when he’s low on energy. The “energy” and “battery” concept is handy for explaining why you have to eat and sleep.

Anyway, while Bao has been resting, we’ve moved on to the next phase of Bao’s evolution. After some research, I’ve found the perfect recipe. But first, check out the ingredients to see if you can guess what we’re making:

1) LG Optimus Dynamic
This is a $20 prepaid smartphone. Prepaid meaning you can actually buy minutes to use it as a smartphone (e.g. if you need a spare phone for family visiting from out of town). And this one’s nice because it has “triple minutes” for life. So you can buy 200 minutes and the phone will give you 600 minutes magically. But what I’m really buying this for is the hardware and software. It comes packed with the usual sensors: GPS, accelerometer, compass, light sensor; and communication interfaces: Bluetooth, WiFi, PLUS a camera, video recorder, audio recorder, a 1GHz processor, some RAM and 4GB of storage space, AND whatever apps you want and the Android OS to tie it all together. All for $20! In the robot world, each of these devices would cost $20 or more by themselves. But thanks to the economics of smartphones, you can get them ALL for $20. What a wonderful time we live in. I really feel that smartphone tech will accelerate many more industries. (Example: Oculus Rift.) So instead of buying loosey goosey electronics for your robo, the only thing you need is Bluetooth… to pair with a smartphone.

2) Scripting Layer for Android (SL4A)
This is what you install on your Android phone. SL4A lets you tap into every function on your phone from your favorite scripting language (e.g., Python). SL4A can either run a script stored on the phone, or you can connect to SL4A to run a script from your PC. E.g., check this out:

From your PC:

This tells your phone at 192.168.1.3 to, "Capture what I say, then say it back to me." How crazy is that? (Browse the API to see what else it can do.) You can tell it to take a picture, make a phone call, or even drive a robot... (hint hint)

3) web2py
This is what I used to make a toddler-proof web interface or "app" or GUI. The front-end (View) is HTML/Javascript, and the backend (Controller) is Python -- perfect for hooking up with SL4A. Right now, it doesn't do much. But it still has you rolling on the floor laughing, because I send the words you type to the phone and the phone says them (using SL4A). You've been having fun typing different words to see how the text-to-speech voice screws it up. Anyway, this bomb-proof custom toddler web UI works much better than VI, which we were using earlier...

Oh, and here's my latest IR + Ultrasonic car code. It's tuned for the 3-wheel car and my kitchen:

Robot Kit – Day Six

Oh the wonderful things a robot can do

It can go CHOO CHOO

It can go BEEP BEEP

I’m going to resist making any more mods until you suggest them. I also want to give mama a chance to tinker. Because she said, “But I wanted to do the BEEP!”

One thing we might have to do soon is give Bao a bumper. The ultrasonic sensor takes a beating from all the crashes. Anyway, with the new 3-wheel configuration, we haven’t had to recharge it for 3 days. The 4-wheel tank ran empty in just one day. But then again, we’ve only been in IR mode. Maybe the Ultrasonic processing is a power hog too.

And finally, it was time for new batteries. I strapped some Legos to the frame. You put "one door for going in, and one door for going out."

And finally, it was time for new batteries. Just for fun, I strapped some Legos to the frame. You made “one door for going in, and one door for going out.”

Robot Kit – Day Six (“What’s that sound downstairs?”)

What’s that sound coming from downstairs?

“I want his name to still be Bawa.”

Bonus: The mailman brought gifts! An 8-pack of Eneloops and one fast charger. I got that charger because it has a DC 12V input, which can be driven by cigarette lighter sockets and a foldable solar panel I’ve been thinking of getting.

Choo choo!

Mama gets a turn too, somewhat. And we learn to share

Robot Kit – Day Five

Here’s Bao with fresh batteries and the new program

I was impressed with how quickly you understood its algorithm.

We had a play date in the afternoon and Bao was an excellent babysitter

Later, I got excited when the CR2025 batteries came in the mail for the IR remote. So we upgraded Bao with IR control. You were a terrible pilot at first, and as you can see, we found out why when you ran away in the middle of playing.

It turned out that you had to go potty. You were a much better pilot after that

I was starting to wonder if we really wanted the tank mode. The tracks kept getting stuck, it ate too much power, it was heavier, and it wasn’t efficient (2 wheels driving 4 wheels, slack in the tracks, etc). The 3-wheeled car was starting to look like a better design. It’s simple like a FWD car, with each motor driving one wheel directly. So we converted Bao to a 3-wheeler tonight and you renamed him to Bawa.

Robot Kit – Day Four

“I want to call it a Sister Kit, not a Starter Kit.”

Its name is now Bow (as in Bow-wow). And it’s a girl.

Batteries are still charging. Meanwhile, I showed you the code and the instruction “Forward”. We’re learning how to “teach” the robot how to do things by sending it instructions. The first thing we’ll do is show it how to go forward. You said, “And then we’ll teach it to spin.” Good idea. Then I was thinking of teaching it how to use its “eyes” so it spins when you put your hand in front of it. Then maybe we’ll load the ultrasonic obstacle-avoidance program.

It’s alive! (Forward only)

New trick (Turn left – but not enough power)

After adding more juice to motors:

Peekaboo mode: “Ultrasonic Car” demo program with Forward() replaced with Stop():

Then we ran out of power 🙁

Update: Somehow I kept tinkering until 5am without batteries, but with the board attached to my computer. I couldn’t move the motors, but I could simulate it by printing to the SoftwareSerial. I added a “handbrake” mode so it pulls the handbrake if it’s stuck or if it gets bored. Then, to start it again, you first have to turn it so it’s no longer facing any obstacle. Then, give it the “Go” signal by waving in front of its eyes.