File Under: servers, Web Services

New Arq 3 Taps Amazon Glacier for Backup Nirvana

Arq 3 makes it easy to navigate Amazon’s Glacier file storage service. Original Image: Christine Zenino/Flickr

Amazon’s Glacier file storage service costs less than a penny per gigabyte per month. It’s hard to think of a cheaper, better way to create and store an offsite backup of your files.

Of course backups are only useful if you actually create them on a regular basis. Unfortunately, getting your files into Glacier’s dirt-cheap storage requires either a manual effort on your part or some scripting-fu to automate your own system.

Back when Glacier first launched we speculated that it would be a perfect fit for a backup utility like the OS X backup app Arq. Now Arq 3 has been released and among its new features is built-in support for Amazon Glacier. Arq 3 is $29 per computer, upgrading from v2 is $15.

Arq creator Stefan Reitshamer sent over a preview of Arq 3 a while back and, having used it for the better part of a week now, I can attest that it, combined with Glacier, does indeed make for a near-perfect low-cost off-site backup solution.

Using Arq 3 with Glacier is simple. Just sign up for an Amazon Web services account and create a set of access keys. Then fire up Arq, enter your keys and select which files you want to back up. Choose Glacier for the storage type and then make any customizations you’d like (for example, excluding folders and files you don’t want backed up).

That’s all there is to it; close Arq and it will back up your files in the background. By default Arq 3 is set to make Glacier backups every day at 12 a.m., but you can change that in the preferences.

Should disaster strike and you need to get your files out of Glacier (or S3), just fire up Arq, select the files you need and click “restore.” Arq will give you an estimate of your costs and you can adjust the download speed — the slower the download the cheaper it is to pull files out of Glacier. There’s also an open source command line client available on GitHub in the event that the Arq app is no longer around when you need to get your files back.

Estimating costs with Arq’s Glacier restore screen. Image: Screenshot/Webmonkey

Existing Arq users should note that Amazon currently doesn’t offer an API for moving from S3 to Glacier (though the company says one is in the works). That means if you want to switch any current S3 backups to Glacier you’ll need to first remove the folder from Arq and then re-add it to trigger the storage type dialog.

In order to get the most out of Arq 3 and Glacier it helps to understand how Glacier works. Unlike Amazon S3, which is designed for cheap but accessible file storage, Glacier is, as the name implies, playing the long, slow game. Glacier is intended for long-term storage that’s not accessed frequently. If you need to grab your files on a regular basis Glacier will likely end up costing you more than S3, but for secondary (or tertiary) backups of large files like images, videos or databases Glacier works wonderfully.

My backup scenario works like this: For local backups I have two external drives. One is nearly always connected and makes a Time Machine backup every night. Once a week I clone my entire drive off to the second external drive. For offsite backups I use rsync and cron to backup key documents to my own server (most are also stored in Dropbox, which is not really a backup service, but can, in a pinch, be used like one).

But my server was running out of space. Photo and video libraries are only getting bigger and most web hosting services tend to get very expensive once you pass the 100GB mark. That’s where Arq and Glacier come in. It took a while, but I now have all 120GB of my photos backed up to Glacier, which will cost me $1.20/month.

The only catch to using Glacier is that getting the data back out can take some time. There are also some additional charges for pulling down your data, but as noted above, Arq will give you an estimate of your costs and you can adjust the download speed to make things cheaper. The slow speeds aren’t ideal when you actually need your data, but these are secondary, worst-case scenario backups anyway. If my laptop drive dies, I can just copy the clone or Time Machine backup drive to get my files back. The Glacier backup is only there if my house burns down or floods or something else destroys my local backups. While it would, according to Arq’s estimate, cost about $60 and take over four days to get my data out of Glacier, that would likely seem like a bargain when I’d have otherwise lost everything.