Webmonkey » Amazon

Amazon Takes on Dropbox With New Desktop File Syncing

Scott Gilbertson — Tue, 02 Apr 2013 15:40:15 +0000

Amazon’s desktop-centric Cloud Drive syncing. Image: Screenshot/Webmonkey

Amazon has quietly joined the ranks of cloud-based file syncing services like Dropbox, Google Drive and Microsoft’s SkyDrive. The company’s Amazon Cloud Drive — previously limited to a rather primitive web-based interface — now offers desktop file syncing tools like those found in Dropbox.

To test out the new Cloud Drive syncing, grab the new desktop app for Windows or OS X (sorry Linux fans, currently there is no desktop client for Linux).

Once you’ve installed the new Cloud Drive app, you’ll find a new folder on your drive — drop whichever files you’d like to sync into that folder and they’ll automatically be sent to Amazon’s servers. You’ll then have access to them on any computer with Cloud Drive installed and through the Cloud Drive web interface, though what you can do with files in the web interface is extremely limited.

It’s worth noting that the Cloud Drive app requires Java. As our friends at Ars Technica point out, that means users with newer Macs will be prompted to install Java as well (the Windows app comes with Java bundled).

There’s also no mobile apps for any platform (there is an Android Photo app, but all it does is send photos from your phone to Cloud Drive). In fact, while Cloud Drive will sync files between desktops, beyond that there isn’t much to see yet.

Part of the appeal of any web-based sync tool is ubiquitous access, not just via the web but in your favorite mobile apps as well and in that space Dropbox clearly has a huge lead over Cloud Drive.

Amazon offers 5GB of Cloud Drive storage for free, with additional storage available at roughly $.50/GB, which is down from the $1/GB price back when Cloud Drive first launched. That’s on par with SkyDrive’s pricing and roughly half the price of Dropbox. In this case though — at least right now — you get what you pay for. Amazon has the makings of a Dropbox competitor but it still has a lot of catching up to do.

Amazon Tackles Web Video With New Conversion Service

Scott Gilbertson — Tue, 29 Jan 2013 16:29:16 +0000

Image: Kevin Dooley/Flickr.

Amazon is getting into the web video game with a new video transcoding service aimed at making it easy to build the next YouTube.

Transcoding video is the process of taking a user uploaded video and converting it to a video format that works on the web, typically MP4 and WebM. Consumer video services like YouTube and Vimeo handle this for you behind the scenes. But if you want to actually build the next Vimeo or YouTube you’re going to have transcode video.

Open source tools like ffmpeg simplify the video transcoding process, but require considerable server power to operate at scale. And server power is something Amazon has in spades.

Amazon’s foray into video is hardly the first cloud-powered video transcoding service — Zencoder is another popular service (and runs on Amazon servers) — but Amazon’s offering is marginally cheaper and well-integrated with the company’s other services.

The Amazon Elastic Transcoder works in conjunction with the company’s other cloud offerings like S3 file storage. You send a video from one S3 “bucket” to Transcoder, which then converts it to the formats you need and writes the resulting files to another S3 bucket.

For now the Elastic Transcoder will only output MP4 video containers with Apple-friendly H.264 video and AAC audio. The new Transcoder options in the Amazon Web Services control panel allow you to create various quality presets if, for example, you’re delivering video to both mobile and desktop clients.

As with all Amazon Web Services the new Transcoder has a pay-as-you-go pricing model with rates starting at $0.015 per minute for standard definition video (less than 720p) and $0.030 per minute for HD video. That means transcoding a 10 minute video (the max on YouTube) would cost you $.15 for SD output and $.30 for HD, which sounds cheap until you start looking at transcoding several hundred 10-minute videos a day (200 a day would set you back $60 a day for HD). Amazon’s free usage tier will get you 20 minutes of SD video or 10 minutes of HD video encoded for free each month.

Amazon’s rates are marginally cheaper than Zencoder, which charges $0.020/minute for SD and double that for HD. Zencoder does have a considerable edge when it comes to output format though, offering pretty much anything you’d need for the web, including live streaming, while, at least for now, Amazon’s offering is limited to MP4.

Amazon Autopsy Reveals Causes of Cloud Death

Scott Gilbertson — Fri, 29 Apr 2011 13:43:09 +0000

Amazon has apologized to customers affected by last week’s EC2 outage and offered a detailed post mortem about exactly what went wrong. The short answer is that a network update shifted traffic to the wrong router, which then wrecked havoc on Amazon’s US East Region Availability Zone.

In addition to apologizing, Amazon is giving affected customers “a 10 day credit equal to 100 percent of their usage of EBS Volumes, EC2 Instances and RDS database instances that were running in the affected Availability Zone.”

Amazon is also promising to improve its communication with customers when things go wrong, but as we pointed out earlier, the real problem is not necessarily Amazon. While Amazon’s services unquestionably failed, those sites that had a true distributed system in place (e.g. Netflix, SmugMug, SimpleGeo) were not affected.

In the end it depends how you were using EC2. If you were simply using it as a scalable web hosting service, your site went down. If you were using EC2 as a platform to build your own cloud architecture, then your services did not go down. The later is a very complex thing to do, and it’s telling that the sites that survived unaffected were all large companies with entire engineering teams dedicated to creating reliable EC2-based systems.

That may be the real lesson of Amazon’s failure — EC2 is no substitute for quality engineers.

See Also:

Lessons From a Cloud Failure: It’s Not Amazon, It’s You

Scott Gilbertson — Mon, 25 Apr 2011 15:11:17 +0000

Chaos Monkey will eat your cloud.

Amazon’s cloud-hosted Web Services experienced a catastrophic failure last week, knocking hundreds of sites off the web. Some developers saw the AWS outage as a warning about what happens when we rely too much on the cloud. But the real failure of Amazon’s downtime is not AWS, but the sites that use it.

The problem for those sites that were brought down by the AWS outage is the sites’ own failure to implement the one key design principle of the cloud: Design with failure in mind.

That’s not to say that Amazon didn’t fail rather spectacularly, taking out huge sites like Quora, Reddit, FourSquare and Everyblock, but as Paul Smith of Everyblock admits, while Amazon bears some of the responsibility, Everyblock failed as well:

Frankly, we screwed up. AWS explicitly advises that developers should design a site’s architecture so that it is resilient to occasional failures and outages such as what occurred yesterday, and we did not follow that advice

But perhaps the most instructive lesson comes from those sites that were not affected, notably Netflix, SimpleGeo and SmugMug. Netflix published a look at how it uses AWS last year and, by all appearances, those lessons served the company well, because Netflix remained unaffected by the recent failure.

Among Netflix’s suggestions is to always design for failure: “We’ve sometimes referred to the Netflix software architecture in AWS as our Rambo Architecture. Each system has to be able to succeed, no matter what, even all on its own.”

To ensure that each system can stand on its own, Netflix uses something it calls the Chaos Monkey (no relation). The Chaos Monkey is a set of scripts that run through Netflix’s AWS process and randomly shuts them down to ensure that the rest of the system is able to keep running. Think of it as a system where the parts are greater than the whole.

The photo sharing site SmugMug has also detailed its approach to designing for failure and why SmugMug was largely unaffected by the recent AWS outage. SmugMug co-founder and CEO Don MacAskill, echos Netflix’s redundancy mantra, writing, “Each component (EC2 instance, etc.) should be able to die without affecting the whole system as much as possible. Your product or design may make that hard or impossible to do 100 percent — but I promise large portions of your system can be designed that way.”

MacAskill also has strong words for those who think the recent AWS outage is a good argument for sticking with your own data center: “[SmugMug's] data-center related outages have all been far worse … we’re working hard to get our remaining services out of our control and into Amazon’s.”

“Cloud computing is just a tool,” writes MacAskill, “Some companies, like Netflix and SimpleGeo, likely understand the tool better.”

If you’d like to learn more about how designing for cloud services differs from traditional data-center setups, check out this excellent post on O’Reilly. Also, be sure to read Netflix’s advice and learn from Everyblock’s downtime by following the guidelines in Amazon’s own documentation.

Photo: Technically not a monkey. (DBoy/Flickr/CC)

See Also:

Amazon’s New ‘Cloud Drive’: Your Music, Everywhere You Go

Scott Gilbertson — Tue, 29 Mar 2011 15:15:59 +0000

Apple and Google are both rumored to be working on music streaming services, but the first real competitor to enter the streaming music battle is Amazon.com.

The company has announced Amazon Cloud Drive, a web-based backup service where you can store your documents, photos music and other files. To go along with Cloud Drive, Amazon is offering Cloud Player, which can stream your music library to any web browser or Android mobile device. Cloud Player also allows you to download files and create playlists through its web-based interface. Note, however, that Amazon is using Flash Player to upload and stream music which means it won’t work on Apple’s iOS devices.

Amazon is offering 5GB of Cloud Drive storage for free to anyone with an Amazon account. That number can be bumped to 20GB with the purchase of an Amazon MP3 album. Additional storage works out to $1 per GB, with plans at 20GB, 50GB, 100GB, 200GB and 1000GB.

At the high end 1000GB will set you back $1000 per year, which is enough to buy roughly 10 TB worth of external hard drives every year. Of course your external hard drives don’t live in the cloud and won’t let you listen to music wherever you are, but if it’s just backups you want, clearly there are cheaper alternatives.

The real appeal of Cloud Drive only comes into play when it’s used with Cloud Player. Bring the two together and you can stream your music library to any web browser or Android mobile device.

While an online music locker capable of sending your music to any device feels almost inevitable at this point, Amazon’s offering is, thus far, disappointing. The interface is awkward and looks a bit like Hotmail did when it first launched — primitive. For example, using the web interface, there’s no way to download more than one file at a time.

Any music you already own must be manually uploaded, there is no automatic syncing like you’ll find in file backup services like Dropbox. The Amazon MP3 Uploader can scan your iTunes library and makes uploading a bit smoother, but if you’ve got a sizable music collection — several hundred gigabytes of music — you’re looking at days, if not weeks, to upload everything to Cloud Drive.

Cloud Drive is much better if you’re buying all your music from the Amazon MP3 Store, and clearly this is what Amazon would like you to do. To encourage that, songs purchased from the Amazon MP3 store and saved directly to your Amazon Cloud Drive never count toward your storage limit and are free to store forever. Couple that with rumors of Amazon working on an Android device and it isn’t hard to see how Cloud Drive, though basic at the moment, might one day become a serious iTunes competitor.

But there are possible legal problems Cloud Drive may have to overcome. The record industry still believes that you should buy a new copy of every song for every device you own, and Amazon has no way to determine whether you actually purchased your existing music files. MP3tunes, which also offers to store and stream your music library, was sued by music labels. Amazon, however, doesn’t believe licensing will be a problem. “We don’t need a license to store music,” Craig Pape, director of music at Amazon, tells the New York Times. “The functionality is the same as an external hard drive.”

Amazon may be first to the market among its sizable competitors like Apple and Google, but Cloud Drive in its current form isn’t particularly innovative, nor does it offer many of the features long-standing competitors like Dropbox or MP3Tunes already have. Still, Amazon’s Cloud Drive already has Google and Apple beat on one count — it exist.

See Also:

Amazon Is Building a Better Browser for Kindle

Michael Calore — Tue, 09 Mar 2010 01:40:27 +0000

Browsing the web on one of Amazon’s Kindle e-readers is like taking a step backwards in time. It’s clunky and has only limited support for web standards, and bare-bones JavaScript capabilities.

But now Amazon may be looking to add browser engineers to the Kindle team, according to the job listings on the company’s website.

A job posting for a browser engineer at Lab126, the division of Amazon that develops the Kindle, indicates the company is looking for somebody to develop “an innovative embedded web browser” for a consumer product.

The role at Lab126 includes designing new features for a new browser while supporting the existing code. Job requirements include familiarity with current web standards and web rendering engines, as well as experience with Java and embedded Linux, both of which the Kindle runs.

The Kindle’s current browsing experience is notably subpar. It’s good enough to check your e-mail, post to Twitter or read Wikipedia, but it doesn’t handle images or more complex web apps particularly well. It certainly doesn’t live up to the same vision of the mobile web being outlined by the iPhone, or Android phones like the Droid or Nexus One. And with the coming of the Apple iPad and other threats to Amazon’s dominant e-reader, which should behave on the web about as well as (if not better than) the iPhone, the Kindle had better improve its browser if the device is going to continue to compete with these more capable devices.

Amazon recently launched a beta program for third-party app developers who want to build software for the Kindle.

Apparently, the job listing has been up for a month, but I only became aware of it once CNET’s Stephen Shankland tweeted about it.

Calls to Lab126 and Amazon on Monday morning went unreturned. I’ll update this post if and when I get more information from Amazon or anyone else.

Meanwhile, if you have any advice about improving the Kindle’s browsing mojo, leave it in the comments.

Photo: Charlie Sorrel/Wired.com

Amazon Contest Eyes AWS Developers

Adam Duvander — Wed, 03 Sep 2008 17:53:29 +0000

Amazon Web Services today launched a contest for developers building their web business off of services like EC2 and S3. The Startup Challenge will award one winner $50,000 in cash and $50,000 in AWS credits, plus potential investment from Amazon.

New startups are commonly using one or more of these web services available from Amazon:

EC2 hosts web applications. Our tutorial helps you Get Started With Amazon Cloud Computing.
S3 is the “simple storage solution” used by even big name startups, like Twitter.
EBS provides persistent storage to EC2.
SimpleDB is in beta and provides access to structured data.

In early October Amazon will pick five finalists in the contest, which the public can vote on. A panel of judges will determine the eventual winner. The contest application form is straightforward, with seven long form questions to answer, including the problem being addressed and target customers. Anyone with a qualified entry (I’m assuming this means a site that uses AWS services) receives $25 in AWS credits.

Need some inspiration? Amazon has a list of AWS case studies that show how sites are using their services.

See also:

Amazon Expands Elastic Computing With New Storage Service

Adam Duvander — Thu, 21 Aug 2008 18:56:37 +0000

Amazon Web Services has released a new storage service that, at first, may look a lot like their S3 offering. The new service, Elastic Block Storage (EBS), is meant to increase the usefulness of their EC2 computing cloud.

In case you’re confused by all these services, here’s how Amazon describes their latest addition:

EBS gives you persistent, high-performance, high-availability block-level storage which you can attach to a running instance of EC2. You can format it and mount it as a file system, or you can access the raw storage directly. You can, of course, host a database on an EBS volume.

While S3 is great for storage, EBS is more flexible with its uses. EBS is used in tandem with EC2 instances. But normally, when an EC2 instance goes away, its storage disappears, too. EBS is, as Amazon says, persistent. It sticks around.

Cloud Computing providers RightScale say EBS opens up Amazon’s services to many new customers. Applications not written directly for Amazon’s other offerings are easier to incorporate with EBS. Amazon even points to a tutorial for running MySQL on EBS.

The costs are similar to other Amazon Web Services, which charge by usage. Storage is 10 cents a GB per month. I/O requests are 10 cents per million. There’s a AWS calculator to help you figure out your own costs.

See also:

Amazon’s Mechanical Turk Simplifies Automation

Adam Duvander — Wed, 30 Jul 2008 19:13:27 +0000

Amazon’s automated human intelligence service, Mechanical Turk, has become more useful to a general audience. To use the service in the past required advanced programming skills. Now Amazon has created an interface to bulk load tasks that require human eyes.

Mechanical Turk can help with data collection and correction. Many are already using the service for filtering out obscene photos/comments, or appropriately categorizing and tagging items. Most pay just a few pennies for the answers.

The site is equally catering itself to the workers. There seems to be plenty of interest in the work, called Human Intelligence Tasks, or HITS. I accidentally posted my test HIT, and received eight responses before I could take it offline.

One of the recent changes to Turk are the HIT templates that give requestors an example to start with. Then Amazon provides a comma-separated file to fill in with your data and re-upload. Previously this had to interface with your database via web services.

The name Mechanical Turk comes from an 18th century hoax. A chess-playing machine invented to impress a royal was later revealed to be controlled by a human. As such, Amazon calls its service “artificial artificial intelligence,” because HITs are automated, yet completed by a human.

See also:

S3 Outage Makes Developers Consider Redundancy

Adam Duvander — Mon, 21 Jul 2008 23:09:03 +0000

At least the downtime gods picked a notoriously low traffic day to punish Amazon’s S3 storage service. The darling of many web apps, including popular Twitter, was down for eight hours Sunday.

Amazon’s service provides storage and transfer of data for a small fee, so that developers can let their own servers focus on more important issues. For small upstart sites, it’s cheaper to buy their data a tiny slice at a time than to invest in a lot of hardware. S3 is especially popular for image hosting, which can be large files in comparison to trim HTML. Also, there are often many images per web page, putting extra strain on a server.

Among the revelations for some developers after a third of a day without S3 is that no single service can be counted on for 100% uptime. Of course, there’s oodles of redundancy built in to Amazon’s service. Yet, still it can go down, leaving many sites that count on it with a single point of failure.

Dave Winer sees a business opportunity in S3 redundancy:

“It would be easy to hook up an external service to S3, and for a fee, keep a mirror on another server. Then it would be a matter of redirecting domains to point at the other server when S3 goes down.”

Developers could achieve the same result Winer mentions on their own. Robert Accettura notes how WordPress.com weathered the S3 outage gracefully:

“They have (slower) back up’s in house for when S3 is down and can failover if S3 has a problem. This means they can leverage S3 to their advantage, but aren’t down because of S3.”

As many have noted, true web scalability and redundancy can be a tough sheep to sheer. While Larry Dignan questions if S3 is too complicated, I think the larger issue is that it is too simple. Too many have viewed it as a silver bullet, with Amazon doing the dirty work for them. This outage (and another back in February) has shown that S3 and services like it can help us a lot, but we still need to do our own work.

See also: