File Under: Web Services

Craigslist to Developers: Look Elsewhere For Mashup Data

Thinking of building a mashup using Craigslist data? You might want to look elsewhere for classified listings.

Craigslist has a long history of being openly hostile to outside developers, but the site took its walled garden mentality to new levels with a recent decision to block Yahoo’s popular Pipes tool from accessing the site’s data.

According to developer Romy Maxwell, Craigslist has blocked Yahoo Pipes primarily to stop his mapping mashup, Flippity. The Flippity mashup was using Yahoo Pipes to gather Craigslist posts from its public RSS feeds and plot them on a map. This apparently rubbed Craiglist the wrong way, but rather than just block the offending Pipe, Craigslist decided to block the whole service — effectively killing thousands of Yahoo Pipes-based mashups.

This isn’t the first time Craigslist has shutdown outsiders trying to improve on the site’s famously antiquated tools — the site previous blocked ListPic, a tool designed to help Craigslist users browse by images and a tool to search all the Craigslist sites at once.

However, unlike ListPic, which Craigslist claimed was using resources excessively, the latest ban doesn’t seem to be about traffic.

In his blog post, Maxwell claims his application complies with Craigslist’s terms of service and doesn’t use excessive bandwidth. In fact, according his post, the Yahoo Pipe in question is private and only runs once every fifteen minutes. Couple that with the fact that Yahoo Pipes caches the data it scrapes, and one could conclude that bandwidth concerns should not be a factor in the shutdown.

As Wired magazine reported in its cover story about Craigslist earlier this year, the company has been quick to ban third-party search services which “[subvert] Craigslist’s mission to enable local, face-to-face transactions” or services which “increase the risk of scams and can be exploited to snatch up bargains, giving technically sophisticated users an advantage over casual browsers.” Flippity doesn’t appear to have been introducing any feature that would enable such behavior.

So, perhaps Flippty was in violation of the TOS? Maxwell asked founder Craig Newmark about the app and Newmark told him, “as a rule of thumb, [it's] okay to use RSS feeds for noncommercial purposes.”

If it’s unlikely to be about bandwidth and there’s no overt violation of the TOS, why ban Yahoo Pipes?

Frankly, we’re not sure. Craigslist isn’t talking — we’ve contacted the company and asked for an explanation, but as of this writing, nobody has gotten back to us, nor has any public statement been issued.

One thing that is clear though, if you’re thinking of adding Craigslist data to your application, think again.

For its part, Maxwell says Flippity will continue, but plans to use data from developer-friendly sites like eBay and Oodle.

See Also: