Raspberry pi robot: real-time object detection using remote Tensorflow server

Finished product first:

The video above shows real-time objection detection using a neural network model called Single Shot MultiBox Detector (SSD). The neural network model was trained on (and runs with) Tensorflow.

The “AI” model works much better than older Computer Vision (OpenCV) techniques. For example, here’s an example of a previous attempt to track objects with the color “tomato”. The way that worked was by filtering a camera frame for a specific color, then masking out that color to find the contour of an object.

There were many problems with this approach. For example, color-based detection is sensitive to lighting changes. It performs differently depending on whether it’s day or night time. Also, we get false detections because many other objects in the scene may have the same color.

But, one advantage of classic OpenCV techniques is they’re simple and fast. Fast enough to run in real time on a live camera feed on a tiny Raspberry Pi processor.

On the other hand, AI models are more accurate and powerful. They can detect many different objects in various orientations (even if they’re partially hidden or chopped out of frame). They’re less sensitive to lighting changes. 

But the downside is they’re expensive to run. The Raspberry Pi doesn’t have quite enough horsepower. I tried it and it’s possible… but I wasn’t happy with the performance. I found some interesting USB AI accelerators like Google’s Coral but they’re pricey. 

So I decided to go with the “Mars Rover” approach and use a remote inference server (running on a PC in my living room). This buys us “infinite” computing power for a fixed cost of network latency. As long as your network is fast, this scheme works well.

One thing going for us is even though the Raspberry Pi camera can capture HD images, the SSD AI model only needs 300×300 pixel inputs. So we can downsize the images before sending them over the network to improve performance.

Here’s the Raspberry Pi code. All it does is capture frames, encode them as Base64 strings and makes requests to our remote inference server: 

Now for the object detection neural network. Google provides a set of pre-trained models for object detection in their Model Zoo. The model I picked was ssdlite_mobilenet_v2_coco. You’ll also need this file to convert the detection IDs to text labels: coco-labels-paper.txt 

I made a Python module to load and run the pre-trained Tensorflow object detection model. The class Predictor in coco_predictor.py  below loads the saved model and provides a predict() function for running inference on new input images.


Finally, the Flask inference server that runs on a PC. The Raspberry Pi sends it images and it replies with detections:


Oh, and if you’re curious about the Robot Kit I’m using, it’s made by Adeept. They call it the Mars Rover PiCar-B:

Mars Rover PiCar-B

The hardware is excellent and well-designed. All the pieces fit together perfectly and mount onto a solid acrylic chassis with bolts. There is a steering rack and a RWD drivetrain that sends power from one motor to both rear wheels. You can pretty much follow the instructions and everything fits together like a Lego set. And when you’re not using it in robot mode, it’s a good looking “case” that lets you use it like a regular Raspberry Pi on your desk.

The software, however, is so-so. You definitely need programming experience. Some things might not work right out of the box, and some things need fiddling around with. On the plus side, they do provide tons of working code for all the various sensors, servos and components. So as long as you are comfortable with Python, you can use their code as excellent references. You can pick and choose and mix and match what you need.

I like this robot kit because it uses a Raspberry Pi (which I already have). So it uses standard Linux stuff. You can use state-of-the-art software and write regular programs like a civilized person. And because it uses a Pi, you get WiFi, Ethernet, Bluetooth, USB, HDMI, etc for free. No painful caveman Arduino programming. It runs on CR 18650 lithium ion batteries (which I also have from salvaging old laptop batteries). It can also run on the regular Pi USB power source.

“Behind the scenes”: Driving school. Programming the head.

I hope to make it autonomous and self-recharging one day!

Happy hacking!

How to make a webapp/server that can read & react to emails

In the last post, we talked about how to teach your Google Home device (or Google Assistant on your Android phone) to send emails.

But that itself is not too interesting (unless you really enjoy spamming yourself). What would be more useful is if you create a program to read and act on the emails automatically. Because, then, you have the ingredients for building your own Virtual Assistant.

The simplest (and cheapest way) I found to do this is using Google App Engine. It has a generous daily free quota. For all my home automation projects, I haven’t run above the free limit.

Start by following the “hello world” instructions to make your first web app:

Then, add the files/code below to enhance your web app to handle emails. (Note: the code I provided below is an example of how I use email to control my home sprinkler system. This lets me say “Turn on sprinkler 1” to my Google Home Mini, which triggers IFTTT to send an email to my web app, which reads the email and triggers my home sprinkler system to turn on zone 1.)




After you set up the above, you can send emails to your Google App Engine web app using its special address:

Now that you have a web app, there’s much more you can do beyond handling emails. It opens up a couple more opportunities for your home automation. E.g.:

  • Use this web app as a trusted HTTP server that your smart devices can fetch instructions from
  • Use this web app to serve a web GUI for your devices/projects
  • Use this web app (which supports HTTPS) to authenticate users and securely control your smart devices

Give it a go. I think the code I gave you above should work. If I missed something, please let me know at aaron@secretsciencelab.com. Happy hacking!

How to send emails with Google Assistant / Google Home

I got a Google Home Mini for $19 last Black Friday, and my family has been having fun with it. My kids like to ask it questions and play music on it. I’ve set it up with voice shortcuts so my kids can say, “Call mom” or “Call dad” to phone me from the Google Home itself (in case of an emergency). It works as a regular Bluetooth speaker and it also knows many useless party tricks.

But most interesting to me is I now have a natural language voice interface for all my projects! With the Google Home / Google Assistant, I can control things with my voice, like how Tony Stark talks to J.A.R.V.I.S. And as someone who was born before the Internet, it is amazing to me that I can do this with a $19 device that sits on my kitchen counter.

But what good is an “Assistant” if it can’t send emails? Not very, if you ask me. So in this guide, I’m going to share with you how I set up my Google Home Mini to send emails. By the way, this works for any Google Assistant interface (like on your Android phone), so it works even if you don’t have a Google Home.

To send emails with Google Home, there are 2 ingredients: IFTTT and Gmail. I’ll assume you have a Gmail account, so we’ll just focus on setting up IFTTT.

IFTTT is a free service at ifttt.com. IFTTT stands for “If this, then that.” Or more specifically, “If x happens, then do y.” IFTTT hooks up with all kinds of standards and protocols and services so that you can replace the x and y with anything.

Some example IFTTT “recipes” are:
If it is dark outside, then turn on the lights.
If it is going to rain tomorrow, then disable the sprinklers.
If the temperature drops below ___, then set thermostat to ___.
If you tell Google Home “Email Joe and say ___”, then send gmail to joe@joe.com with ___ .

I’m going to explain how to do that last “recipe”. It’s not that hard, and just a matter of setting up IFTTT.

1) Make a new “If this then that” applet on ifttt.com. Google Assistant will be the “this” part of the recipe and Gmail will be the “that” part.
2) Connect Google Assistant to your IFTTT account. You can program it to trigger an action when you speak a special phrase.
3) Connect a Gmail account to your IFTTT account. It does not have to be your main Gmail account. You can make a new account just for IFTTT mails. When you say your special phrase to Google Assistant, you can program it to send a special email.
(More detailed instructions here.)

Next, I’ll show you how to configure a Google App Engine web app to receive emails. This setup can be used to relay information to your smart devices or control them for free. (I’ll be using it to control my home sprinkler system.)

How to start using NodeMCU for WiFi projects (Goodbye Arduino?)

Ah Arduino. You were my first gateway to making tiny electronic widgets that interact with the world. I’ve played with platforms that range from small and simple to expensive and complex like FPGAs, but nothing beats the cost and ease-of-use of Arduinos for making tiny devices you sprinkle around your home. Until now…

See the thing is, WiFi has become as common as electricity in many countries. So I keep finding that I want all my devices to connect to the internet — something that Arduino doesn’t do very well. Especially if you want to talk at a higher level; like with HTTP or MQTT. Plus, getting WiFi-enabled Arduino chips or going with alternative platforms like Electric Imp can get pricey.

So that led me to a family of devices built around the ESP8266 chip. Not only do these tend to be cheaper than Ardunios, they have integrated WiFi! (E.g.: D1 mini NodeMCU).

But I noticed something popular among users of these devices. They don’t use the normal Arduino platform, but instead a different NodeMCU firmware. I looked it up and what I liked about it was that its firmware libraries help you do many complex things you normally want to, easily. E.g., you can make your NodeMCU device a web server, or talk MQTT with just a few lines of code.

So I got a NodeMCU chip for under $3 to play with one weekend. I figured out how to flash the firmware and upload my code to it. The process is slightly different from using an Arduino. So I compiled all my notes, scripts and whatever else I needed into a bare-bones package. I plan to use this as my personal template for starting new NodeMCU projects. Since it might be useful to others too, I posted it to github here:

The broad steps for starting a NodeMCU project are:

  1. Download and build the firmware. I recommend using the Docker option. That way, all you do is download the firmware source code from git, customize it (optional), then use docker to build it.
  2. Flash the firmware. The flash command is also in the link in the previous step. Or check out “flash.sh” in my nodemcu-skeleton git.
  3. Upload your code using luatool. See “upload.sh” in my git’s ‘test’ dir for reference.

Although the steps are more ‘manual’ than with Arduino, I actually like it better. I never liked working in the clunky Arduino IDE. I prefer coding in my favorite editor, then uploading to the device via command-line. Working with NodeMCU does come with its own quirks, but they’re not too bad once you get into the groove.

I am still experimenting with these $3 NodeMCU chips, but so far I’m liking it. My first project was a sound monitor to keep an ear on the house while we were away for a few days. This was mainly because my wife was worried about leaks from her new aquarium setup. So first, we placed regular water alarms under the aquariums. The sound monitor measures the sound level continuously and streams the data to the web (via Thingspeak). Then I have a cron job that watches the data stream and emails me if anything looks funny. It’s sensitive enough to detect anything from doorbells, loud conversations and microwave beeps… to water alarms and fire alarms.

I might switch to these NodeMCUs as my go-to device for all my future projects!

Happy hacking 🙂

DIY home automation almost-instant-response WIFI smart switch for < $16 with MQTT

Parts list:

Total: $15.55 ($7.28 if you go with the mini NodeMCU)

WARNING!!! this switch can switch up to 110/240V AC @ 10A. This is more than enough to cook your insides or “stop” your heart and kill you. Do not play with your house mains unless you know what you’re doing and you understand the risks. One mistake and you’re dead. If you’re not sure, close this page. It’s not worth it.

I made this WiFi switch to add an IP camera to the outside of my house, without having to run new power lines to it. The idea was to tap into the 110V AC of an existing outdoor porch light to power my camera. I would then keep the switch to the light ON all the time, to power my camera ON all the time. But I didn’t want my porch light to be on all the time. So, a WiFi smart switch will replace the wall switch to control the light. This was acceptable, since we don’t turn the porch light on/off frequently, and it’s better off scheduled (turn on at night) or automated (turn on when motion detected).

My only constraint was that everything had to fit into an in-wall power conduit or switch box. The largest part was the AC/DC transformer/power supply. I managed to find a 1.25A power supply that is 70 x 39 x 31 mm. As a bonus, you can open its metal case and pack your microcontroller and other electronics inside it:

Open the metal case, pack it in. AC comes in the top. DC goes out the bottom to the Relay, microcontroller and my IP camera.

Open the metal case, pack it in. AC comes in the top. DC goes out the bottom to the Relay, microcontroller and my IP camera.

The input of the power supply is 110/240VAC and the output is 12V DC. My IP Camera uses 12V DC, so it took the raw 12V DC. But the Cactus Micro and the Relay need lower voltage. So I used the DC-DC converter to step down 12V DC to 5V DC.

Closer look at the DC bits

Closer look at the DC bits. The blue board I’m lifting up is the microcontroller that talks over WiFi and switches the relay. USB for scale.

Closer look at my camera

Closer look at my camera

Closed up. Compact enough to fit into a wall conduit/switch box.

Closed up. Compact enough to fit into a wall conduit/switch box.

And that’s the WiFi switch. Wire the switch up like this:

Your WiFi switch controls your light or whatever AC load

Your WiFi switch can now control your light or whatever AC load you want up to 240V 10A

The rest of the magic happens in software. There are 2 parts to the software. 1 part is the code you upload to the Arduino microcontroller. That connects to WiFi, listens for on/off commands and switches the relay. The 2nd part is what sends on/off commands to your smart switch.

Software Part 1:
Let’s set up your MQTT broker first. MQTT is the protocol that we’ll use to send commands to your Smart Switch and switch it on/off almost instantly. You can think of MQTT as an “IoT Twitter”. It lets your switch “subscribe” to a topic. Then something else can “publish” to that topic to turn it on/off. You need a MQTT broker because that’s what passes your messages between publisher and sender.

The easiest way to get a MQTT broker is to sign up for CloudMQTT’s free “Cute Cat” plan: https://www.cloudmqtt.com/plans.html. The 4 pieces of info you need after you set it up are:
– Your broker’s hostname or IP address
– Your broker’s port
– Your broker’s user
– Your broker’s password

Software Part 2:
Arduino code. I used a Cactus Micro, which has 2 chips in one package: an Arduino chip and an ESP8266 for WiFi. 1st step is to program the ESP8266 with Espduino. Espduino is an Arduino library that makes it easy to connect to WiFi and talk MQTT.

  1. Set up your Cactus Micro arduino to flash the ESP8266’s firmware — upload this Arduino progrmamer sketch: http://wiki.aprbrother.com/wiki/How_to_made_Cactus_Micro_R2_as_ESP8266_programmer.
  2. Flash the ESP8266 with the “espduino firmware”: https://github.com/AprilBrother/espduino/tree/master/esp8266/release. Use the NodeMCU Flasher if you’re on Windows and the esptool if you’re on Linux.
  3. Upload the Arduino smartswitch sketch below to your Cactus Micro…

IMPORTANT: Use Arduino IDE version 1.0.6. If you use version 1.0.5 your Cactus Micro will be unstable and randomly crash.

Disclaimer: The code below has a little more than the MQTT on/off stuff. I wanted to also show you how to sync time and also upload data to thingspeak.com. If you don’t need those, just delete them. I included them because I’ve found them useful in all my projects and they show you how you can upload/download regular HTTP via Espduino. I also included some bulletproofing code that reboots the ESP8266 if it malfunctions.

Replace all the “TODO_…” strings with your own info

Once you have everything set up, you can power your switch and test it using the CloudMQTT “Websocket UI.” The code above subscribes your switch to the “/homebot/switches/MY_SMARTSWITCH/cmd” topic. Send a “1” over MQTT to turn it on. Send a “0” or whatever else to turn it off.

Here is mine in action:

Happy switching!

P.S. If you get stuck or have any questions, email me at aaron@secretsciencelab.com

P.P.S Please don’t die by being stupid with electricity. Actually it’s probably best if you don’t try this project.

“Sprinkler Brain” – Cactus Micro Arduino Wi-Fi multi-zone Smart Sprinklers (DIY for under $20)

In some parts of the world, water usage is serious business. For example, there is a drought in California. If you live here, you know that they want you to water your lawn only twice a week. And if they catch you using too much water, they slap you with a higher rate for being a water pig. So water is serious business. But you love your garden and vegetables and trees and such. And they need water. So what can you do?

To be honest, I’m struggling to figure that out myself. My neighbor is getting hip with lush succulents that barely need any water. I might go that route too, but even if you do, you still need to be smart about your water. In fact, as you sprinkle more diverse plants around your landscape, watering might get more complicated since different plants need different amounts of water. And the first step to optimizing water usage is being able to CONTROL water flow from a computer program. On, off. Once we can do that, you can smart it up any which way to optimize when to switch sprinklers on or off. So that’s the main goal of this write-up — to show you how to set up a cheap and bulletproof base rig to programmatically switch your sprinklers.

But since we’re building, how about we make a wishlist of all the things we can do better than grandpa’s garage wall sprinkler controller:

  • Program sprinkler schedule from anywhere, without going to the wall unit
  • Never lose your programs when you lose power or forget to replace the backup battery
  • Automatically skip sprinkling when it rains
  • Measure how much water each sprinkler zone uses
  • Never have to physically be at the wall unit to turn it on or off
  • Must be able to manually turn individual sprinklers on and off from your phone, when you want to test and fix sprinklers around the yard
  • Must be able to spray your kids/neighbors/dog/squirrels with the tap of your finger, from the comfort of your armchair
  • Etc.

Parts list:

Total: $17.14

Now here’s what we’ll be building…

The secret plans

My secret schematic. Disclaimer: use at your own risk, I’m not responsible for death or damages, don’t play with electricity, etc etc

S1 and S2 at the bottom are your sprinklers. 24 AC on the right is the standard power supply for your sprinkler valves.
Block (A) on the left is the 8-channel relay.
Block (B) in the middle-ish is the Cactus Micro microcontroller.
Block (C) is the AC/DC converter to tap 24V AC from the standard sprinkler circuit and convert it to 5V DC for our control circuit.

At this point you might be wondering why is there a transistor in the middle of the schematic, if we already have a relay. Well, you don’t need it. But I put it in there as a safety feature. Remember how water is serious business? I figured that I’m okay if my chip malfunctions and it doesn’t turn on the water. But what’s worse is if it malfunctions and the water gets stuck OPEN and floods the yard and the sidewalk and the street and neighbors while I’m away. That would be an expensive disaster.

The possibility crossed my mind because I fried one of my Cactus Micro pins while soldering the headers, and it got stuck and pulled one relay open. Had that relay been connected to a sprinkler, and had that sprinkler locked open while I wasn’t home, I would have returned to a $1000 water bill before I could shut it off.

So that’s why I added that extra transistor switch. (I would have added 5 more safety mechanisms if I had the patience.) It functions like a Two-man rule control. Like when they needed two keys to arm the nuclear missile in the Hunt for Red October. You can see that the transistor guards the power supply of the relay. What this means is that TWO things have to work to open a sprinkler relay:
1) an “open” signal from the Cactus Micro microcontroller D1-D8 pins to the relay’s IN1-IN8 pins
2) an “enable” signal from the Cactus Micro’s D15 pin to the transistor

This way, two pins have to fail for the relay to be stuck open — which is still possible — but less likely than one pin failing. Can’t be too sure.

OK. Now we build. First, prepare the Cactus Micro. The original firmware is garbage. We want to replace it with espduino. Espduino gives you rock-solid WiFi and lets you work like a civilized person via a REST API, not a caveman via serial commands.

Follow instructions in the 2 links below. First upload the Arduino sketch to configure the Cactus as a serial programmer. Then flash the ESP8266 firmware through the Cactus host board.

After the espduino firmware is on the ESP8266, replace the “serial programmer” sketch above with our real “Sprinkler Brain” sketch below:

Some noteworthy bits in the sketch above:
– syncClock() syncs the time on your Cactus Micro with timeapi.org every 15 minutes
– readHygro() is an example of how you might add a sensor to your system
– fetchCommands() fetches commands from my “cloud” mothership (in the form of a string like

where 1 is force on, 0 is force off, – is follow program).
– uploadData() pushes Cactus data/state to thingspeak.com servers. I use this thingspeak data stream to render my smartphone “app” UI
– restGet() is the wrapper function I use to make HTTP REST calls. It counts errors so that if I see too many consecutive ESP8266 errors, I can power-cycle the ESP8266.
MinuteMap is the compact data structure I use to store my sprinkler schedule
– The Time.h Arduino library is from here

I set up my “server” on Google App Engine. It’s awesome and Google gives you a very generous free daily quota. Unlike Amazon’s EC2 which rails you with no lube and annoying bills even if you do something innocent like leave one terminal connected to your instance. With Google App Engine, I have never needed more than the free daily quota, even with my many projects bashing one app.

(Email me at aaron@secretsciencelab.com if you want my AppEngine code. I didn’t have time to carve it out and clean it up to post here)

Then I made a simple web app UI which serves 2 purposes:
1) manually turn sprinklers on/off to impress friends
2) monitor the sprinklers in action and to make sure it’s working

Here we can see that Sprinkler Channel 1 is running and it has 7 mins left on its schedule

Here we can see that Sprinkler Channel 1 is running and it has 7 mins left on its schedule. I can tap the on/off buttons on the left to manually override the schedule.

(Email me at aaron@secretsciencelab.com if you want my HTML/Javascript web app code. I’m happy to share, just too lazy right now to package it nicely to post here)

After you have the software loaded, hook it up following the schematic. Hopefully you will end up with something that looks better (and is less of a fire hazard) than this:

AC/DC converter at top left. Mini breadboard to connect wires at top right. Transistor is on breadboard. 8-channel relay at bottom of box. Cactus micro with gazillion wires going to the breadboard.

AC/DC converter at top left. Mini breadboard to connect wires at top right. Transistor is on breadboard. 8-channel relay at bottom of box. Cactus micro with gazillion wires going to the relay and breadboard. I used the Rev1 Cactus in this pic. You should get the Rev2.

Action shot:

Happy Sprinkling!

How to flash espduino firmware on Cactus Micro ESP8266 rev1

Modify Cactus Micro:
1) convert default serial port to hardware serial (cut/solder)
2) connect Digital pin 5 to ESP8266’s GPIO0 (so arduino sketch at the bottom of this page can pull ESP8266’s GPIO0 to GND – this puts the ESP8266 in program mode to receive firmware update)

Load Arduino programmer sketch – sends computer COM serial straight to ESP8266 hardware serial

Flash firmware. Do NOT use esptool.py. I kept getting this error:

Instead, use the GUI for Node MCU

Click Config tab
Uncheck the internal Node MCU firmware (selected by default)
Browse for espduino 0x00000 bin
Browse for espduino 0x40000 bin

Enjoy espduino!

Important: when using the serial passthrough Arduino sketch, don’t change the default baud rate of espduino – I tried to be smart and set it at 9600 — didn’t work.

Below is the “passthrough” sketch I used to stream the firmware from USB through Arduino serial through ESP8266 hardware serial. It’s ugly and there are many other similar sketches, but I’m saving the one I used here, just in case.

P.S. You’ll probably want this Arduino library next:

DIY Electric Imp WiFi garage opener + sensor for under $50 (weekend project)

Background story / motivation
One night, I opened my garage door to take out the trash. Then I came back in and went to bed. The next morning, as I was strapping my son into his car seat, he asked me, “Daddy, why is that exercise ball over there?” “What? What exercise ball?” “That one. Who put that ball there?” I replied without looking, “I don’t know, I didn’t put it there.” He said, “Maybe mama put it there.” “Maybe she did.” “Maybe it fell.” I looked back and saw it in the middle of the floor. Strange. Didn’t seem like her to leave it like that. Then I looked around. My toolbox was gone. I looked left. My bicycle was gone. I peeked into my wife’s car. Every compartment was open and trashed.

I had left the garage door open and we had been robbed.

Since then, whenever I left the house, I found myself checking and double-checking the garage door. And sometimes even after triple-checking… I would drive back to check ONE MORE TIME. Because you can’t be too sure. Even when I was home, I found myself constantly peeking into the garage to make sure I didn’t leave the door open again.

It was driving me nuts. So I decided to do something about it. If you’re like me, this weekend project will finally give you some peace of mind. If you’re handy with electronics and code, you could even get it done in one evening. When you’re done, you’ll get to open your garage door from your phone. And best of all, you’ll always know if your door is open or closed.

(By the way, before going for this solution to the garage door problem, I considered leaving my garage door open again. I really wanted to set a trap to destroy whoever dared to rob me again. But my wife said no. So here’s just some instructions for a garage door opener… instead of an automated crossbow turret.)

Parts list

Total: $44.75

Tools list

Finished pictures

Mini breadboard ties everything together. The Imp is sitting in the April Breakout board. The April board's soldered header pins plug into the breadboard. The blue thing is the relay with 3 pins in the breadboard. You see a switch in the picture but I replaced that with an Ultrasonic sensor. The problem with the switch was my garage door doesn't always stop at the same place, so no physical switch was going to reliably trigger. On the other hand, the Ultrasonic sensor measures accurate distances up to 3m, so you can make a software switch that you can tune to perfection. Dinosaur for scale.

Mini breadboard ties everything together. The Imp is sitting in the April Breakout board. The April board’s soldered header pins plug into the breadboard. The blue thing is the relay with 3 pins in the breadboard. You see a switch in the picture but I replaced that with an Ultrasonic sensor. The problem with the switch was my garage door doesn’t always stop at the same place, so no switch was going to reliably trigger. On the other hand, the Ultrasonic sensor measures accurate distances up to 3m, so you can make a software switch that you can tune to perfection. Dinosaur for scale.

I SEE YOU, garage door

I SEE YOU, garage door

Goodies tucked into small cardboard box on the side. USB power from ceiling.

Goodies tucked into small cardboard box on the side. USB power from ceiling.

Black wires go from relay to the same terminals used by the wall switch.

The tiny black wires connect the relay to the same terminals used by the wall switch.

Smartphone UI screenshot. Just a big button. The three checkboxes below it are to arm the button. This is so I can't accidentally press the big button.

Smartphone UI screenshot. Just a big button. The three checkboxes below it are to arm the button. This is so I can’t accidentally press the big button.

Circuit diagram

I drew this with Fritzing!

I drew this with Fritzing!

Electric Imp setup/prep

Solder male headers onto Imp

Electric Imp Agent code

Electric Imp Device code

(Optional) rolling authentication code
I made a Google AppEngine app to be a gatekeeper and for the smartphone UI to open/close the door. The idea is:

  • Take advantage of AppEngine’s built-in authentication to control who can access your app
  • When an authorized person taps “Open”, save a random “rolling code” into your web app’s DB
  • Then, tell your Electric Imp to “Open” and pass it the rolling code you just generated
  • When your Electric Imp Agent receives the “Open” command, it checks the received rolling code with your AppEngine webapp
  • If the rolling code matches the latest code in the DB, the webapp destroys the code (so it cannot be reused) and the Imp Agent tells the Imp Device to open the door

If you want to go this route and need more info, email me at aaron@secretsciencelab.com. If enough of you want it I’ll put it here. I’m just too lazy right now!

The hardest part is soldering header pins to the Imp, and soldering wires to the Ultrasonic sensor. But other than that, everything else is pretty much plug and play. That’s the beauty of using the Imp, the Keyes Relay and an Ultrasonic sensor. All the electronics are nicely packaged in each of them, so all you need to do is connect them with wires.

I stuffed everything into a small cardboard box and tied it to the frame of the garage door control box. I plugged the Imp’s USB adapter into the same outlet that powered the garage door control box. Lastly, I secured the Ultrasonic sensor to a bolt under the garage door rail using a twist-tie. Dirty, but simple!

Bonus (advanced)

  1. Push your sensor data to data.sparkfun.com for free:
    E.g., door open, door close

    E.g., door open, door close

  2. Convert your SparkFun CSV stream into an Atom RSS using my URL: http://secretsciencelab.com/wp-content/scripts/phant/feed.php?csv=http://data.sparkfun.com/output/YOUR_PUBLIC_KEY.csv&key=state
    (replace the bolded parts).

  3. Plug your Atom RSS URL above into IFTTT as a Feed recipe.
  4. Now you can tell IFTTT to: “If new Feed item, then do something.” E.g., “If my garage door opens or closes, text me.”

Happy hacking 🙂


How to teach Spyno to spy on (almost) anything

Spyno likes JSON. It’s a JSONivore. So when you make a Spyno agent, you need to feed it a source URL in JSON format.

But what if you want to spy on things that are not in JSON? No worries. We have two tricks:

Trick 1. Yahoo Query Language YQL free service

“Use YQL to convert XML to JSON & vice versa. Access atom, rss, micro formats and more. You can even load CSV files from anywhere.”

So whether your favorite site is in RSS or XML, no problem. Just send it to Google Feed to transform it to JSON.

For example, say one of your guilty pleasures is Perez Hilton (don’t ask):

  1. Find the RSS feed URL (E.g., http://i.perezhilton.com/?feed=rss2)
  2. Drop the following query into this YQL form:

  3. That returns you the following Endpoint, which you can copy-paste into Spyno

Trick 2. Google Feed API

“With the Feed API you can download any public Atom, RSS, or Media RSS feed using only JavaScript, so you can easily mash up feeds with your content and other APIs.”

Trick 3. Kimonolabs API

Kimono is like the Swiss Army kitchen sink that turns any website into a JSON feed. It’s designed for scraping websites. But what’s special about it is that you can teach it what you want scraped. All you have to do is open the website you’re interested in using Kimono, then click elements on the page you want:

And these two tricks are about all you need to make JSON feeds for your new Spyno Agent.

Happy scraping, Spyno trainers!

How to view Spyno from anywhere

When you’re on your PC, open Spyno by clicking on him at the top right of Chrome:



But what if you’re not at your PC? Visit your Public URL, of course!

Here’s how you get the Public URL for your Spyno page:

  1. Type chrome://extensions in your Chrome’s address bar. Hit Enter.
  2. Visit Spyno’s options page
  3. Click


  4. Scroll to the bottom. Click “Get code”:


  5. Sign in with Google and you’ll see this:
    Code Page

    The link marked (1) is your public URL. Send that to your phone to view Spyno from your phone. Copy the code marked (2) to your PC’s Spyno option page. Spyno uses that code to update your public page.

    Paste your code here and click Save

    Paste your code here and click Save

  6. Ta-da!

Now you can take Spyno with you anywhere.

Next: How to teach Spyno to spy on (almost) anything