How to teach Spyno to spy on (almost) anything

Spyno likes JSON. It’s a JSONivore. So when you make a Spyno agent, you need to feed it a source URL in JSON format.

But what if you want to spy on things that are not in JSON? No worries. We have two tricks:

Trick 1. Yahoo Query Language YQL free service

“Use YQL to convert XML to JSON & vice versa. Access atom, rss, micro formats and more. You can even load CSV files from anywhere.”

So whether your favorite site is in RSS or XML, no problem. Just send it to Google Feed to transform it to JSON.

For example, say one of your guilty pleasures is Perez Hilton (don’t ask):

  1. Find the RSS feed URL (E.g., http://i.perezhilton.com/?feed=rss2)
  2. Drop the following query into this YQL form:

  3. That returns you the following Endpoint, which you can copy-paste into Spyno

Trick 2. Google Feed API

“With the Feed API you can download any public Atom, RSS, or Media RSS feed using only JavaScript, so you can easily mash up feeds with your content and other APIs.”

Trick 3. Kimonolabs API

Kimono is like the Swiss Army kitchen sink that turns any website into a JSON feed. It’s designed for scraping websites. But what’s special about it is that you can teach it what you want scraped. All you have to do is open the website you’re interested in using Kimono, then click elements on the page you want:

And these two tricks are about all you need to make JSON feeds for your new Spyno Agent.

Happy scraping, Spyno trainers!

Leave a Reply

Your email address will not be published. Required fields are marked *