All posts tagged ‘DIY’

File Under: APIs, Web Services

It’s Time to Build a Twitter-Free Twitter

Image: Twitter.

Twitter dropped a bombshell on third-party application developers last Friday — the social network built on the backs of third-party developers and clever, innovative clients has decided it no longer needs them.

Twitter’s blog post is short on specific details, but the gist of it is that Twitter is tightening up its API access for third-party developers. The company has long viewed third-party apps as unnecessary and previously warned developers not to “build client apps that mimic or reproduce the mainstream Twitter consumer client experience.” But thus far it hasn’t enforced that guideline. Now it seems it will.

In a post on the Twitter developer blog titled Delivering a consistent Twitter experience, Michael Sippey, Twitter’s director of product, seems to say that the company wants its official clients — and only its clients — to be the way people use Twitter. Instead of building clients that pull out of Twitter, the company wants developers to “build into Twitter.” In other words, kiss your Tweetbot, Twicca or Hibari goodbye and get ready for some embedded widgets instead of good ol’ tweets.

Much digital ink was spilled over the weekend denouncing Twitter’s policy change or lamenting the potential loss of alternative Twitter clients. Of course Twitter is in charge of Twitter and when you use its service — or build apps on its API — you must suffer its whims.

But Twitter’s decision to start enforcing its API restrictions “more thoroughly” could end up a great thing if it inspires developers to take the essence of what makes Twitter great — succinct, timely messages to and from your friends — and free it from Twitter the company.

An independent and decentralized equivalent to Twitter is certainly not a new idea. The basic building blocks you’d need to build such a system have been with us for many years — a combination RSS, OPML and perhaps PubSubHubbub would cover of most of it — but until now there hasn’t been widespread client developer support for such a system. After all, why go to all the trouble of building a decentralized network on top of open web standards when using the Twitter API is so much easier?

Twitter’s third-party developers now have the answer to that question — because you can’t be locked out of the open web.

Developer Brent Simmons, perhaps best known for creating the Mac-based RSS reading app NetNewsWire, has a basic outline of how Twitter app developers could band together and make something that not only sidesteps Twitter’s coming API restrictions, but the service itself.

“The interesting (to geeks like us) part,” writes Simmons on his blog, is “what system that works like Twitter could exist without a company behind it?”

Simmons then proceeds to break Twitter down to its essentials: “under the hood, following somebody is really just subscribing to a feed of their statuses. Posting is really just updating a feed of your own statuses. So you standardize on a feed format. RSS would work great, of course, and there’s a ton of RSS reading and writing code out there already.”

Instead of Twitter clients, what you’d really be building is a real-time RSS client. That’s not a far-fetched idea. Dave Winer, the forefather of blogging and creator of RSS, has been building one for years. (He’s also been telling everyone to build a distributed Twitter-like publishing system for years.)

Simmons doesn’t address it directly, but it’s worth noting that building such a system doesn’t preclude using Twitter. It’s not either/or, it can be both. In this scenario you’d write a post in a client like Tweetbot and Tweetbot could automatically send it Twitter and to your own feed. Start with both and then a migration away from Twitter would be smoother. Those that want to dump Twitter immediately could do so, but still keep posting to anyone with a client that supports the open structure. Then, if Twitter really does cut out third-party apps completely, the infrastructure necessary to support an open alternative is already up and running.

Simmons has more details for developers on his blog and in a follow-up post that delves more into the logistical complexities, but the basic message to developers is simple: Twitter’s changes means you need to find a better network for your clients to use.

The better network is the one that’s always been there — the web. The advantage for app developers feeling threatened by Twitter’s API changes is obvious. As Simmons writes, “there’s a practical reason to use the open web: your app can’t be shut down.”

The question is, if there were an open alternative would disgruntled Twitter users embrace it? The main argument against any alternative is the so-called network effect: Everyone I know is on Twitter; why would I go somewhere else? But not too long ago no one used Twitter and everyone used Myspace. Everyone used Friendster. Everyone use AOL. People change; networks move. A distributed version of Twitter sans Twitter might well be the web to Twitter’s AOL, but there’s one certainty: We’ll never know until we build it.

File Under: servers, Web Basics

What to Do When Your Website Is Hacked

One drawback to the otherwise awesome sauce of the do-it-yourself web is that you’re also responsible for fixing it yourself when something goes wrong — call it the FIY corollary to the DIY web.

For example, what happens if the bad guys attack your website?

In some cases your web hosting service may be able to help, but most of the time undoing the damage is your responsibility. Websites are attacked every day; well-tested though they may be, frameworks and publishing tools inevitably have security flaws and eventually you may be bitten by one. Or it might not even be the tools that end up being the problem, it might be something far less obvious. Developer Martin Sutherland’s server was recently hacked because one file on a shared server had the wrong file permissions.

Sutherland’s write-up of how he discovered and fixed the attack on his server is well worth a read and makes an excellent primer on how to handle being hacked. While Sutherland’s situation may be specific to the attack that his site suffered, his diagnostic steps make an excellent starting point even if you use a completely different publishing system. (Sutherland uses Movable Type.)

Sutherland’s strategy (once he realizes he’s been hacked) is to scan through all the files on his server to see which ones had recently been changed. He then filters that list, ignoring files that should have changed (log files, etc.) and narrowing it down to suspicious file changes.

How much this approach will tell you if your own site has been hacked depends on what the attacker has done and what your server setup looks like, but it should help you get moving in the right direction. Read through the full post for the specific command line tools Sutherland uses to inspect his files. If you’re not comfortable on the command line or don’t have shell access to your server you may be able to use something like Exploit Scanner (if you’re using WordPress) or a similar tool for your publishing system.

Once you know what happened and which files were affected it’s just a matter of rolling back the changes using your backups. You do have backups right? As Sutherland writes, “it’s not a matter of if something goes wrong, it’s a matter of when.” Remember: backups are only useful if you have them before you need them.

We sincerely hope your site is never hacked, however, it does happen all too frequently. As Sutherland’s write-up illustrates, one of the keys to making sure that you recover quickly is to have good backups. Do yourself a favor and spend a few minutes creating an automated backup system before something goes wrong. Now excuse me while I go make sure my pg_dump cron script is running properly.

File Under: Social

ThinkUp Wants to Liberate Your Online Social Life

The corporate social web still sucks

Expert Labs, the non-profit organization behind ThinkUp, a web-based data-liberation and analytics application, is rebooting into a commercial entity.

No need to panic if you use ThinkUp to back up your social network life; the application will remain open source and freely available.

But Expert Labs is going away and ThinkUp is refocusing on a larger goal — liberating your online social life from the clutches of corporate web entities.

In its own words the new ThinkUp wants to build “an information network that connects to today’s social networks, but isn’t centralized and dependent on a company or investors.”

That’s not an entirely new idea. Diaspora and some other projects are trying to do the same thing, but ThinkUp is taking a different approach — it wants to build an app first and focus on the user experience rather than the underlying technology.

In fact ThinkUp already is an app that’s pretty close to what it’s aiming to do. ThinkUp is a web-based app that pulls your data out of social silos like Facebook or Twitter and stores it on your own server. You control your own data, and have a record of your conversations potentially long after Facebook, Twitter and the rest have become mere footnotes in the history of the web.

For more on how ThinkUp works and how you can use it be sure to check out our earlier coverage and then grab the code and try it for yourself.

So what of ThinkUp’s new, loftier goals? Is any attempt to replace Facebook doomed to failure? Of course not. Everything is replaceable, just ask MySpace. And ThinkUp believes its approach is different. “Prior attempts have tried to solve this problem based on the assumption that it is a technical challenge,” says ThinkUp’s Knight News Challenge application. “We believe it to be a social one.” ThinkUp’s focus going forward will be on the social and the interface:

We will draw people in through a compelling media site that encourages participation via our decentralized platform… a peer-to-peer network that powers a great media property with broad appeal — imagine if Digg or Reddit were open, decentralized and powered by a network instead of votes.

If you’re curious to know what that might look like, head on over to the ThinkUp proposal for the Knight News Challenge and click the heart icon to “like” it (incidentally if the Knight New Challenge sounds familiar, that might be because it’s also the birthplace of EveryBlock). In the meantime, work on the ThinkUp app continues with a new release that improves the charts and graphs and paves the way for the coming Foursquare support. Check out the ThinkUp GitHub page for more details.

File Under: APIs, Web Services

Backup Your Flickr Images in Your Own Parallel Dimension

Like most of the web you’re probably waiting for the other shoe to drop on the much-loved, but seemingly beleaguered, Flickr photo service.

Let’s face it, Flickr’s parent, Yahoo, hasn’t exactly had a banner year and the company already all but killed developer-favorite Delicious. If you aren’t worried about the future of Flickr it’s probably because you aren’t paying attention.

Or, it might be because you’ve got a complete and total backup of all your Flickr images running on your own URL, complete with all the metadata, permissions and privacy settings you’ve stored on Flickr.

What’s that? You don’t have a parallel version of Flickr on your own server? For shame.

Lucky for you, former Flickr employee Aaron Straup Cope created Parallel-Flickr which, as the name suggests, mirrors Flickr on your own domain. Parallel-Flickr is, in Cope’s words, “still a work in progress… it ain’t pretty or classy yet but it works.”

In a nutshell Parallel-Flickr is a set of PHP scripts for backing up your Flickr photos and generating a database-backed website to display them. The feature list includes downloading and storing your original images (along with the 640px version) and grabbing all of Flickr’s metadata about each image as a JSON file. With that info Parallel-Flickr then constructs a database and generates a website with the same URL structure that Flickr uses. Parallel-Flickr also “honours the viewing permissions you’ve chosen on Flickr.” It’s that last part of that description that’s intriguing. Here’s Cope’s description of what the code does:

The thing that’s most interesting to me though is the last piece on that list: The part where the site uses Flickr to authenticate logged in users. What that means is that I can replicate Flickr’s privacy settings locally. It means that I can have a local copy of my photos and keep private things private…

If you come to my site and you’re not logged in (via Flickr) you just won’t see non-public photos. Neither will I, for that matter. But if you do log in then because you’ve logged in via the Flickr API auth dance I have a auth token for you and can look up your Flickr ID and whether you’re a contact and see when and where you have permissions to see all those other photos.

In other words, so long as Flickr is around, Parallel-Flickr allows your site to act exactly like Flickr. From the URLs to the privacy settings, you’ll have your data backed up and online. Should the unthinkable happen to Flickr your site will still continue to function, save your private images which will be hidden safely away.

As noted above, Parallel-Flicker is a work in progress, but if you’d like to try it out, head on over to the GitHub page and grab the code. If you prefer to wait for features like cron jobs for syncing, geodata backups, S3 archiving and more, keep an eye on the project, all that and more is already on the todo list.

See Also: