It seems like most sites offer RSS feeds (a method of subscribing to new postings), but every so often I’ll run into a site that doesn’t, but should. For example, say you want to be notified every time a certain company posts a new job listing, or every time a library adds a book on a certain topic.
For those situations, you may find Dapper useful. Dapper is part of a new breed of Web ventures called (unpleasantly enough) “scrapers” — they scrape data from other sites and turn it into usable chunks of information that you can manipulate.
I like Dapper because it’s very effective and easy to use. I used it because I wanted to create an RSS feed of movies I added to my list on the movie social-networking site Flixster. This may seem weird, but I wanted to create a WordPress sidebar widget that lists movies I’ve just watched. Flixster offers its own widget, but it’s hideous:
Yuck. Flixster doesn’t have an RSS feed of your data, so using Dapper’s Dapp Factory, I specified which types of data I wanted, gave each type a name, and grouped them together. Dapper did a really remarkable job of deducing which types of data were similar. (From reading other, older posts about Dapper on the Web, I gather that it’s gotten much more effective in the last year or so.)
Once you’ve scraped the data you want, you can get it from Dapper in a number of forms: RSS, XML, Google Gadget, iCalendar, Netvibes, and Image Loop. Dapper can also create its own widget for you automatically.
WordPress has its own simple RSS widget, so I just entered the address for my Dapper-created RSS feed, and you can see the results at right. It’s interesting to think about other applications for this. Some companies are extremely nervous about the possible effects of data-scrapers, and Wired has an interesting article detailing the various ways data scrapers could be abused, or could simply violate other companies’ intellectual property.
There are other ways to create an RSS feed. Yahoo Pipes offers a greater degree of flexibility, but it requires more programming expertise. Kapow features a service similar (to my layperson’s eyes) to Dapper, but is aimed at companies rather than individuals. FeedYes did a nice job creating an RSS feed from my Flixster data, but it doesn’t offer as many output options as Dapper. Feed43 can also create an RSS feed, but the process is a little more complex.
I found the Dapp Factory to be the easiest, most intuitive solution to my specific problem. It’s free, no registration required.