Scaling on a Shoestring, Lessons from NewsBlur

NewsBlur survives a traffic surge after news of Google Reader’s pending demise gets around.
Image: NewsBlur.

One of the more interesting stories to emerge from the demise of Google Reader is that of NewsBlur, a previously small, but very nice, open source alternative RSS reader.

NewsBlur is a one-man operation that was humming along quite nicely, but when Google announced Reader would shutdown, NewsBlur saw a massive traffic spike — in a few short days NewsBlur more than doubled its user base. How NewsBlur developer Samuel Clay handled the influx of new users should be required reading for anyone working on a small site without loads of funding and armies of developers.

“I was able to handle the 1,500 users who were using the service everyday,” writes Clay, “but when 50,000 users hit an uncachable and resource intensive backend, unless you’ve done your homework and load tested the living crap out of your entire stack, there’s going to be trouble brewing.”

Having tested NewsBlur a few times right after Google announced Reader was closing, I can vouch for the fact that there were times when the site was reduced to a crawl, but it came back to life remarkably quickly for a one-man operation.

In his postmortem, Clay details the moves he had to make to keep NewsBlur functioning under the heavy load — switching to new servers, adding a new mailing service (which then accidentally mailed Clay 250,000 error reports) and other moments of rapid, awkward growth.

It’s also worth noting that Clay credits the ability to scale to his premium subscription model, writing that, “the immediate benefits of revenue have been very clear over the past few days.”

As for the future, Clay says he plans to work on “scaling, scaling, scaling,” launching a visual refresh (which you can preview at dev.newsblur.com) and listening to feedback from the service’s host of new users.

If you’re looking for a Google Reader replacement, give NewsBlur a try. There’s a free version you can test out (the number of feeds is limited). A premium account runs $24/year and you can also host NewsBlur on your own server if you prefer.