Archive for the ‘Backend’ Category

File Under: Backend, Identity, Web Basics

We Should Retire Aaron’s Number

Aaron Swartz. Image: Wikimedia Commons.

[Editor’s Note: Coder and activist Aaron Swartz committed suicide Jan. 11, 2013 in New York. He was 26 years old. See Wired’s early coverage for details.]

When a great baseball or basketball player leaves the game they retire his or her number. That means the jersey hangs from the ceiling, or there’s a plaque at the stadium, and no player on the team ever wears that number again.

Babe Ruth’s number, 3, is retired. Michael Jordan’s too (23). Jackie Robinson’s number, 42, is retired for all baseball teams.

On the web, retiring a number would mean the website is permanently registered, and the content is preserved so it lasts as long as the web does. That means the contents of will be there forever. It will never become a porn site, or a landing page, or whatever.

Right now there is no way to do this. Isn’t that strange. We could fix it if we want. The internet is just software. It would be a small but worthwhile hack and could set a precedent for future memorials.

Something to think about!

This post first appeared on Scripting News. Also see the related thread on Hacker News.

Dave Winer, a former researcher at NYU and Harvard, pioneered the development of weblogs, syndication (RSS), podcasting, outlining, and web content management software. A former contributing editor at Wired magazine, Dave won the Wired Tech Renegade award in 2001.
Follow @davewiner on Twitter.

File Under: Backend, servers, Web Services

Host Your Static Website on Amazon S3, No WWW Necessary

Amazon’s S3 file storage service started life as just that — a simple way to store static files and pay for only the data you used. When you don’t need an always-on server, S3 fits the bill.

But if you can store static files, why not whole static websites? In 2011 Amazon began allowing you to point your own domain to an S3 “bucket”, a folder in Amazon parlance. Custom domain support made it simple to host entire static sites; the catch was that you needed to use a subdomain — for example, www.

Now the www restriction has been lifted and you can point any root domain at S3 and serve your files directly. The only catch is that Amazon has created its own non-standard DNS workaround, which means you must use Amazon’s Route 53 service to host the DNS data for your domain.

Unfortunately, while the new root domain support is great news for anyone using a static blog generator like Jekyll, Amazon’s documentation leaves much to be desired. To help you get started with S3 hosting, here’s a quick guide to setting up S3 to serve files from a root domain (rather than making the root domain redirect to, as the Amazon blog post instructions do).

First, register a domain name and point your DNS records to Amazon’s Route 53 service (the Route 53 docs have detailed instructions on how to do this). The next step is to create an S3 bucket for your domain. In other words, a bucket named

Now click the Properties button, select the Website tab and make sure that the option is enabled and the Index Document is set to index.html. You’ll also need to click the Permissions tab and set a bucket policy (you can use this basic example from Amazon).

Now upload your site to that bucket and head back to Route 53. Here comes the magic. To make this work you need to create an A “Alias” DNS record. Make sure you name it the same as your domain name. Sticking with the earlier example, that would be Now click the Alias Target field and select the S3 endpoint you created earlier when you set up the bucket.

And that’s it. Behind the scenes that Route 53 “Alias” record looks like a normal DNS A record. That means things like email will continue to work for your domain and at the same time Route 53 directs requests to your S3 bucket. If you want to make www redirect to the root domain you can either set that up through Route 53 (see Amazon’s instructions) or handle it through another service.

File Under: APIs, Backend, Web Services

Google Drive’s New ‘Site Publishing’ Takes on Amazon, Dropbox

Google’s demo site, served entirely by Google Drive. Image: Screenshot/Webmonkey

Google has unveiled a new feature dubbed “site publishing” for the company’s Drive cloud hosting service. Drive’s new site publishing is somewhere between a full-featured static file hosting service like Amazon S3 and Dropbox’s public folders, which can make hosted files available on the web.

Google has set up a simple demo site served entirely from Google Drive to give you an idea of what’s possible with the site publishing feature. Essentially site publishing gives your public folders a URL on the web — anything you drop in that folder can then be referenced relative to the root URL. It’s unclear from the announcement how these new features fit with Google’s existing answer to Amazon S3, Google Cloud Storage.

The API behind site publishing works a lot like what you’ll find in Amazon’s S3 offering. If you use the Drive API’s files.insert method to upload a file to Drive, it will return a webViewLink attribute, something like That ugly, but functional URL becomes the base URL for your content. So, if you uploaded a folder named images, with a file named kittens.jpg, you could access it on the web at

There’s one drawback though, Drive’s site publishing doesn’t appear to support custom domains, which means it works fine for assets like images, CSS or JavaScript, but unless you don’t mind serving your site from some funky URLs, it’s probably not the best choice for hosting an entire site.

There are already numerous static file hosting solutions on the web including Dropbox and Amazon’s S3, as well as whole publishing systems that use Dropbox and S3 to host files, but for those who would prefer a Google-based solution, now you have it.

For more details on the new API see the Google Apps Developer Blog and be sure to read through the Drive SDK docs. If you need help, Google is answering questions over on Stack Overflow.

Behind the Scenes at Instagram: Tools for Building Reliable Web Services

In case you missed it, yesterday Facebook acquired Instagram, a photo-sharing service with some 30 million users and hundreds of millions of images on its servers.

The reported sale price of one billion dollars no doubt has many developers dreaming of riches, but how do you build a service and scale it to the size and success of Instagram? At least part of the answer lies in choosing your tools wisely.

Fortunately for outside developers, Instagram’s devs have been documenting the tools they used all along. The company’s engineering blog outlined its development stack last year and has further detailed how it uses several of the tools it’s chosen.

Instagram uses an interesting mashup of tried-and-true technologies alongside more cutting-edge tools, mixing SQL databases with NoSQL tools like Redis, and chosing to host its traditional Ubuntu servers in Amazon’s cloud.

In a blog post last year Instagram outlined its core principles when it comes to chosing tools, writing, “keep it very simple, don’t reinvent the wheel [and] go with proven and solid technologies when you can.”

In other words, go with the boring stuff that just works.

For Instagram that means a Django-based stack that runs on Ubuntu 11.04 servers and uses PostgreSQL for storage. There are several additional layers for load balancing, push notifications, queues and other tasks, but overwhelmingly Instagram’s stack consists of stolid, proven tools.

Among the newer stuff is Instagram’s use of Redis to store hundreds of millions of key-value pairs for fast feeds, and Gunicorn instead of Apache as a web server.

All in all it’s a very impressive setup that has, thus far, helped Instagram avoid the down time that has plague many similar services hit with the same kind of exponential growth. (Twitter, I’m looking at you.) For more details on how Instagram looks behind the scenes and which tools the company uses, be sure to check out the blog post as well as the archives.

Microsoft Unveils New Plan to Speed Up the Web

Microsoft wants in on the drive to speed up the web. The company plans to submit its proposal for a faster internet protocol to the standards body charged with creating HTTP 2.0.

Not coincidentally, that standards body, the Internet Engineering Task Force (IETF), is meeting this week to discuss the future of the venerable Hypertext Transfer Protocol, better known as HTTP. On the agenda is creating HTTP 2.0, a faster, modern approach to internet communication.

One candidate for HTTP 2.0 is Google’s SPDY protocol. Pronounced “speedy,” Google’s proposal would replace the HTTP protocol — the language currently used when your browser talks to a web server. When you request a webpage or a file from a server, chances are your browser sends that request using HTTP. The server answers using HTTP, too. This is why “http” appears at the beginning of most web addresses.

The SPDY protocol handles all the same tasks as HTTP, but SPDY can do it all about 50 percent faster. Chrome and Firefox both support SPDY and several large sites, including Google and Twitter, are already serving pages over SPDY where possible.

Part of the IETF’s agenda this week is to discuss the SPDY proposal, and the possibility of turning it into a standard.

But now Microsoft is submitting another proposal for the IETF to consider.

Microsoft’s new HTTP Speed+Mobility lacks a catchy name, but otherwise appears to cover much of the same territory SPDY has staked out. Though details on exactly what HTTP Speed+Mobility entails are thin, judging by the blog post announcing it, HTTP Speed+Mobility builds on SPDY but also includes improvements drawn from work on the HTML5 WebSockets API. The emphasis is on not just the web and web browsers, but mobile apps.

“We think that apps — not just browsers — should get faster,” writes Microsoft’s Jean Paoli, General Manager of Interoperability Strategy.

To do that, Microsoft’s HTTP Speed+Mobility “starts from both the Google SPDY protocol and the work the industry has done around WebSockets.” What’s unclear from the initial post is exactly where HTTP Speed+Mobility goes from that hybrid starting point.

But clearly Microsoft isn’t opposed to SPDY. “SPDY has done a great job raising awareness of web performance and taking a ‘clean slate’ approach to improving HTTP,” writes Paoli. “The main departures from SPDY are to address the needs of mobile devices and applications.”

SPDY co-inventor Mike Belshe writes on Google+ that he welcomes Microsoft’s efforts and looks forward to “real-world performance metrics and open source implementations so that we can all evaluate them.”

Belshe also notes that Microsoft’s implication that SPDY is not optimized for mobile “is not true.” Belshe says that the available evidence suggests that developers are generally happy using SPDY in mobile apps, “but it could always be better, of course.”

The process of creating a faster HTTP replacement will not mean simply picking any one vendor’s protocol and standardizing it. Hopefully the IETF will take the best ideas from all sides and combine them into a single protocol that can speed up the web. The exact details — and any potential speed gains — from Microsoft’s HTTP Speed+Mobility contribution remain to be seen, but the more input the IETF gets the better HTTP 2.0 will likely be.