So many screens, so many images (testing responsive sites with Adobe Shadow). Photo: Adobe

The average webpage is now 1.2 megabytes and around 60 percent of that rather large payload comes from images. That’s a lot of data, whether you’re handling images responsively or just trying to speed up a desktop site.

You might think, if images are the bulk of what your browser is downloading, that browsers would be working hard to speed up the image downloads, perhaps trying alternate, space-saving image formats, but you’d be wrong.

You might also think that, as Google’s Ilya Grigorik writes, “innovating on better image formats would be a top agenda item” for the web. But again you’d be wrong. The web is still using the same image formats it’s been using virtually since the first images appeared online.

Grigorik thinks it’s high time that changed and we agree.

In a recent post looking at what it would take to deploy new image formats on the web, he writes, “if we really want to make an impact on web performance, then image formats is the place to do it… there is absolutely no reason why we shouldn’t have dozens of specialized formats, each tailored for a specific case and type of image.”

Of course no web developer wants to deal with dozens of specialized image formats. Nor should they need to — that’s a job for servers. “In a world with dozens of image formats,” continues Grigorik, “the human solution does not scale — read, markup does not scale… whereas computers are fantastic at doing exactly the kind of optimization work required to solve the problem.”

Grigorik isn’t alone in calling for new image formats, nor is he the first to suggest handing these tasks off to the server. Developer and responsive images proponent Matt Wilcox has argued for a similar solution, as have others.

The basic premise of these arguments is that deciding which image to serve up to which device and browser should be a server-side problem. And in fact there’s already a way to solve this problem with HTTP headers, namely the Accepts header, which tells the server which image formats the browser supports. Based on that information the server could then “re-encode, recompress, resize, strip unnecessary metadata and deliver the optimal format.”

The problem is that web browsers (with the exception of Opera) don’t actually send useful information in the Accepts header.

Thus, the first step in creating a server-side solution for smaller images is to get other browsers to send useful Accepts headers.

The Accepts header isn’t a magic bullet by any means, but it’s a problem that’s not hard to solve provided browser makers prioritize it. But to really get server side image solutions working the web would also need new server tools (fortunately, several already exist). There are other stumbling blocks as well. Grigorik addresses half a dozen potential problems and objections that you can read through in his post.

Even if browser makers come around to the idea and do start improving Accepts headers, bringing better image formats to the web is going to be an uphill battle. But Grigorik is determined to chase the idea. “Some uphill battles are worth fighting,” he writes in a comment, “I think this a good one. Wish me luck.”

That's a lot of screens, are your images ready? (testing using Adobe Shadow). Photo: Adobe.

No one wants to waste bandwidth sending large images over limited mobile pipes, but everyone wants their images to look good on the multitude of screens connecting to today’s web. Finding the balance between the myriad factors — screen resolution, screen size, bandwidth, layout and design — is a tricky process.

We’ve looked at a number of techniques for handling images in responsive designs, but one thing that we haven’t covered is where to put the actual breakpoints.

You’re probably familiar with breakpoints in responsive design, they’re the screen widths in your media queries. For responsive images the idea is the same; a “breakpoint” is the screen size at which you swap in and out different size images.

At first glance it might seem logical to use the same breakpoints for your images that you’re using in your CSS media queries, but as Cloud Four’s Jason Grigsby writes, that’s not always the ideal solution. Grigsby tackles the potentially tangled question of breakpoints in a new post, So How do You Pick Responsive Images Breakpoints?

To simplify what can be a very complex question, Grigsby ignores some scenarios, including the so-called “art direction” use case where images are optimized (cropped differently, for example) for different screens. Instead Grigsby focuses on the question of how to best select “different image files with different resolutions of the same image based on the screen size.”

The simplest solution is to just make your image and responsive design breakpoints the same, but that’s rarely going to be ideal for your site’s visitors. Here’s what Grigsby has to say about using the same breakpoints for both image and media queries:

If we were talking about the art direction use case, then it is likely that the breakpoints would be the same because changes in the layout might also indicate an edit to the image.

But in the case where we’re simply changing files to provide different resolutions and a faster download, the image breakpoints should be determined based on when the browser is downloading an unnecessarily large file.

The main problem is, what constitutes an “unnecessarily large file”? But even if you answer that, as Grigsby writes, you still haven’t answered every question:

How do you determine what is an unnecessarily large file? Is that 1 byte? 10k? 20k?

And if you can answer that question, how do you determine the resolutions at which those file size jumps are going to occur? Depending on the content of an image and the compression algorithm used, images are likely to have different resolutions at which they experience significant changes in file size.

In the end Grigsby doesn’t yet have an answer to the question of how to handle responsive image breakpoints. And neither do I. There just isn’t an easy answer that will work for every project. My thinking, and what I’ve done on my own site, runs pretty much along the same lines of Eric Portis’s comment on Grigsby’s post. If you’ve got ideas, head over to Cloud Four and let them know how you’re doing it.

So many screens, so few images (testing responsive sites with Adobe Shadow). Photo: Adobe

The web design community continues to debate the merits and drawbacks of the WHATWG’s proposed adaptive images solution.

As we reported last week, a new srcset attribute has been added to the <img> element in the WHATWG’s HTML specification. The new attribute will allow developers to specify different sized images based on the user’s screen size.

The idea is to find a way to serve smaller images to devices that don’t need large images — saving precious bandwidth — while serving high-resolution images to screens that warrant them. And the WHATWG’s srcset attribute does solve some of the problems surrounding adaptive images, but it’s far from ideal.

Now developer Jason Grigsby argues that not only will the srcset solution not completely solve the problem, but the goal of adaptive images is fundamentally at odds with how web browsers currently handle images. In other words, there currently is no way to solve the problem.

Adaptive images are at odds with how browsers handle images thanks to what’s known as a “lookahead pre-parser.” Browsers use lookahead pre-parsers to start downloading images as soon as possible (to speed page-load times), which means images are parsed and downloads started before the browser has determined the full page layout.

However, a truly useful adaptive image solution needs the browser to first determine the page layout and then determine which images to use.

Grigsby rightly calls it a chicken and egg dilemma. “How do we reconcile a pre-parser that wants to know which size image to download ahead of time with an image technique that wants to respond to its environment once the page layout has been calculated?”

Grigsby argues that the smart thing to do might be for browsers to eliminate pre-fetching:

For existing web content, the lookahead pre-parser is undoubtedly the fastest way to render the page. But if web development moves towards responsive images as standard practice, then delaying the download of images until the proper size of the image in the layout can be determined may actually be faster than using the lookahead pre-parser. The difference in size between a retina image for iPad and an image used on a low resolution mobile phone is significant.

That’s going to be a tough sell to browser makers right now. Browser makers are understandably loath to do anything that might slow down page-load times — even if that slow-down is temporary.

Other possible solutions Grigsby covers include progressive image formats (which suffer from similar chicken-and-egg dilemmas) and of course the <picture> element. The whole article is well worth a read since it gets into more details about why all of these solutions are ultimately less than ideal.

So many screens, so few images (testing responsive sites with Adobe Shadow). Photo: Adobe.

The web needs a more intelligent way to serve images.

No one wants to waste bandwidth sending large images over limited mobile pipes, but everyone wants images to look good on the myriad screens connecting to today’s web. Currently web authors use a variety of hacks to (incompletely) work around this problem, but to really solve it the web likely needs new tools.

Unfortunately, thanks to miscommunication between standards bodies, web developers and browser makers, instead of a solution to the image problem what developers got this week feels more like a slap in the face. Eventually an adaptive image solution will likely emerge, but the real lesson for many developers will be about how the standards process works and how they fit into it, if at all.

Webmonkey has previously looked at some proposed solutions to the adaptive image problem. Some very smart web developers came up with the idea of a <picture> element that works much like the current HTML <video> element. These developers thought they had the attention of the Web Hypertext Application Technology Working Group, better known as the WHATWG. Then, earlier this week, Edward O’Connor, Apple’s WHATWG representative, proposed another method of solving the problem, using a new srcset attribute on the <img> element. See our earlier coverage of the srcset attribute for a more detailed look at how it works and compares to the <picture> proposal.

What has web developers up in arms is that Ian Hickson, editor of the WHATWG spec (and better known as Hixie) has already added the srcset attribute to the WHATWG’s HTML draft spec, seemingly ignoring the months of effort that went into <picture>. Worse, members of the WHATWG apparently weren’t even aware that developers were putting forth the effort to come up with a solution via the Responsive Images community group. Nor were concerns about the srcset syntax given much consideration. Hickson does address some objections to srcset in his message to the WHATWG, but ends up dismissing most of them.

That doesn’t match up with how most people envision the web standards process. But as web developer and standards advocate Jeremy Keith writes, “this is exactly how the WHATWG is supposed to work. Use-cases are evaluated and whatever Hixie thinks is the best solution gets put in the spec, regardless of how popular or unpopular it is.”

In fact, think of the WHATWG as the source for initial, rapid development of new features. The group was started by browser makers because the W3C’s HTML Working Group (HTMLWG) moved too slowly. But if the WHATWG is the source of rapid development, the W3C is an effective check on that speed, ensuring that even those of us who don’t make web browsers still have a voice in the future of HTML. (see our earlier overview for more on the history and differences between the HTML WG and the WHATWG.)

While the HTML WG is also chaired by Hickson (a position he will soon step down from), it offers a much more democratic (and consequently slower) process and has overridden the WHATWG’s rash decisions in the past. For example the W3C added the time element back after Hickson removed it from the WHATWG spec.

Confused yet? It gets worse. The WHATWG is working on an ever-evolving standard, what it calls a “living standard,” which is different from — and may well diverge from — the snapshot-based standards issued by the W3C, like HTML5. In a comment on longtime web standards champion Jeffery Zeldman’s post on the matter, Jeremy Keith writes, “I don’t mind if the srcset attribute is in the WHATWG HTML spec but not in the W3C HTML5 spec. If it works, it’ll end up in a future W3C version number.”

Implicit in Keith’s statement is that if the srcset attribute doesn’t end up working out it won’t be in HTML5.x and would likely just fade away like the blink tag, the applet tag and other HTML ideas tried and later discarded.

Which is another way of saying developers need not panic. Perhaps web developers don’t have a voice in the WHATWG simply because we’ve been using the wrong channels (W3C community groups don’t seem to be an effective means of communicating with standards bodies, in fact they seem more like this.). If you’ve got ideas and would like a voice in the future of the web join the WHATWG mailing list and login to the IRC channel. Introduce yourself, learn the rules and contribute.

Flickr: now with bigger images and a (mostly) responsive design.

The larger images in Flickr’s revamped photo pages put the emphasis where it belongs — on your photos. Peripheral information, like comments, maps, tags, set info and so on are still there, they’re just now (rightly) dwarfed by the actual image.

The result is a much more photo-centric site that does a nice job of differentiating itself from the current trend of low-res, filter-heavy photo0sharing services.

Web developers, take note: Flickr’s new layout isn’t just eye-catching, it’s also somewhat responsively designed — adjusting to the myriad screens on the web today and displaying the best photo possible without clogging your tubes with huge photo downloads. Flickr does stop short of scaling pages down to phone-size screens — for which there is a separate mobile website — but it resizes nicely to handle tablets.

That’s right, Flickr is the latest (and perhaps the largest) website to embrace not just a mostly responsive design with a liquid layout and media queries, but also a responsive approach to images.

We’ve looked at dozens of ways to handle images in a responsive design, but Flickr has opted for a custom setup that uses a bit of server-side PHP and some JavaScript to serve images based on screen size. Flickr is also using a custom algorithm that takes the width and height of the screen into account and “will display content at a width that will best showcase the most common photo ratio, the 4:3.”

For more details on how Flickr is handling the responsive aspects of the new design, check out the Flickr code blog.

Developers working with the Flickr API should note that the new photo sizes are now available through the Flickr API if your app or website would also like to display larger images.

The high-resolution web is coming. Photo: Ariel Zambelich/Wired

Given enough time, all simple, already solved problems of the web eventually rear their ugly heads again.

Remember when limited bandwidth was a huge problem? Then bandwidth was infinite. Now it’s a problem again. And that means serving up images is once again a complex problem with no elegant solution. Its seems simple. Websites should serve the right image to the right screen, high-resolution images to high-resolution devices and low res to the rest. But of course it’s not that simple. Factors like bandwidth as well as screen size and orientation complicate the matter considerably.

Arguably the best solution right now is to send low-res images to every device. Sure, your images might look terrible on high-res screens, but at least you aren’t wasting people’s time or worse, costing them money.

While that’s the safest solution for now, the web doesn’t get better if no one takes any risks. Fortunately, until some standard or best practice emerges, we’ll likely continue to see developers pushing the boundaries and discovering new ways to handle the seemingly simple task of serving the appropriate image to the appropriate device.

The latest image cleverness we’ve seen is Adam Bradley’s Foresight.js. Foresight.js is designed to make it easy to serve up high-resolution images to devices like the new iPad, but what sets foresight.js apart from half a dozen other solutions that do the same thing is that it not only checks for a hi-res screen, but also checks to see if the device currently has a fast enough network connection for larger images. If, and only if, your visitor has both a device capable of displaying high-res images and a network connection fast enough to handle the larger file size, are larger images served.

Part of what makes Foresight.js appealing is its use of the proposed CSS image-set() function, one possible solution to the problem of serving up the right image at the right time. The image-set() function, which works in WebKit nightly builds and is under consideration by the W3C, looks like this:

myselector {
    background: image-set(url(foo-lowres.png) 1x, url(foo-highres.png) 2x) center;
}

Foresight.js takes the image-set() proposal and uses an ingenious hack to make it work in other browsers: the font-family property. Yes, it sounds crazy. But it works and remains technically valid CSS because font-family allows for arbitrary strings (to handle font names). That means browsers have no problem with a rule like this:

myselector {
    font-family: ' image-set( url(/images/foo.png), url(/images/foo_2x.png) 2x high-bandwidth ) ';
}

It’s a hack to be sure, but it’s our favorite kind of hack: clever and functional. Because browsers successfully parse the font-family rule (even if they can’t apply it) the value is added to the DOM and JavaScript has no problem accessing it, which is exactly what foresight.js does.

For more on foresight.js, head over to the GitHub page which as links to plenty of examples uses and copious documentation on the script’s many tricks. Also be sure to read through Bradley’s Challenges for High Resolution Images, which offers some background on foresight.js and the design decisions he made.

The high-resolution retina display iPad has one downside — normal resolution images look worse than on lower resolution displays. On the web that means that text looks just fine, as does any CSS-based art, but photographs look worse, sometimes even when they’re actually high-resolution images.

Pro photographer Duncan Davidson was experimenting with serving high-resolution images to the iPad 3 when he ran up against what seemed to be a limit to the resolution of JPG images in WebKit. Serving small high-resolution images — in the sub-2000px range — works great, but replacing 1000px wide photographs with 2000px wide photos actually looks worse due to downsampling.

The solution (turns out) is to go back to something you probably haven’t used in quite a while — progressive JPGs. It’s a clever solution to a little quirk in Mobile Safari’s resource limitations. Read Davidson’s follow-up post for more details, and be sure to look at the example image if you’ve got a new iPad because more than just a clever solution, this is what the future of images on web will look like.

As Davidson says:

For the first time, I’m looking at a photograph I’ve made on a screen that has the same sort of visceral appeal as a print. Or maybe a transparency laying on a lightbox. Ok, maybe not quite that good, but it’s pretty incredible. In fact, I really shouldn’t be comparing it to a print or a transparency at all. Really, it’s its own very unique experience.

To show off the sample on his site Davidson uses a bit of JavaScript to toggle the high- and low-res images, highlighting the difference.

But how could you go about serving the higher res image to just those screens with high enough resolution and fast enough connections to warrant it?

You can’t.

So what’s a web developer with high-res images to show off supposed to do? Well, right now you’re going to have to decide between all or nothing. Or you can use a hack like one of the less-than-ideal responsive image solutions we’ve covered before.

Right now visitors with the new iPad are probably a minority for most websites, so not that many people will be affected by low-res or poorly rendered high-res images. But Microsoft is already prepping Windows 8 for high-res retina-style screens and Apple is getting ready to bring the same concept to laptops.

The high-res future is coming fast and the web needs to evolve just as fast.

In the long run that means the web is going to need a real responsive image solution; something that’s part of HTML itself. An new HTML element like the proposed <picture> tag is one possible solution. The picture element would work much like the video tag, with code that looks something like this:

The browser uses this code to choose which image to load based on the current screen width.

The picture element would solve one part of the larger problem, namely serving the appropriate image to the appropriate screen resolution. But screen size isn’t the only consideration; we also need a way to measure the bandwidth available.

At home on my Wi-Fi connection I’d love to get Davidson’s high-res images on my iPad. When I’m out and about using a 3G connection it would be better to skip that extra overhead in favor of faster page load times.

Ideally browsers would send more information about the user’s environment along with each HTTP request. Think screen size, pixel density and network connection speed. Developers could then use that information to make a better-informed guess about which images it to serve. Unfortunately, it seems unlikely we’ll get such tools standardized and widely supported before the high-res world overtakes the web. With any server-side solution to the bandwidth problem still far off on the horizon, navigator.connection will become even more valuable in the mean time.

Further complicating the problem are two additional factors, data caps on mobile connections and technologies like Apple’s AirPlay. The former means that even if I have a fast LTE connection and a high-resolution screen I still might not want to use my limited data allotment to download high-res images.

AirPlay means I can browse to a site with my phone — which would likely trigger smaller images and videos since it’s a smaller screen — but then project the result on a huge HD TV screen. This is not even a hypothetical problem, you can experience it today with PBS’s iPhone app and AirPlay.

Want to help figure out how the web needs to evolve and what new tools we’re going to need? Keep an eye on the W3C’s Responsive Images community group, join the mailing list and don’t be shy about contributing. Post your experiments on the web and document your findings like Davidson and countless others are already doing.

It’s not going to happen overnight, but eventually the standards bodies and the browser makers are going to start implementing solutions and the more test cases that are out there, the more experimenting web developers have done, the better those solutions will be. It’s your web after all, so make it better.

Photo: Ariel Zambelich/Wired

The high-res future is coming

The sharp, crystal-clear screens are awesome news for new iPad owners, but they create some new dilemmas for web developers who’d like to offer a better experience for high-resolution screens. Sure, increased pixel density means you can serve up sharper, better looking graphics, but there is a cost as well — bigger images mean more bandwidth and longer page loads.

This isn’t a new problem by any means and Webmonkey has looked at a variety of solutions in the past, including techniques like adaptive images and responsive images.

The problem is simple: you need to send smaller images to small screens and larger images to larger screens. Sending a huge iPad-optimized image to a device with a max resolution of 320×480 just doesn’t make sense. At the same time, when bandwidth isn’t an issue, most sites will want to serve high-resolution content to displays that can handle it.

The ideal solution would be to detect both the resolution of the screen and the available bandwidth. Then, based on the combination of those two factors, the server could offer up the appropriate image. Currently that’s not possible, though there are already proposals to extend HTML to handle that scenario.

The Responsive Image Working Group is a W3C community group hoping to solve some of these problems. The group is proposing a new HTML element, <picture>, which will take into account factors like network speed, device dimensions, screen pixel density and browser cache to figure out which image to serve up. Think of it as a much smarter version of the old lowsrc property. So far though it’s all hypothetical

In the mean time if you’d like to serve up high resolution images to your third-generation iPad visitors look no further than Apple.com for one (not necessarily ideal) way to do it. An Apple Insider reader noticed that Apple is already prepping its site to deliver double-resolution images to new iPads. Cloud Four’s Jason Grigsby, whose responsive image research we’ve covered before, has a great breakdown of what Apple is doing.

Essentially Apple is serving up lower resolution images by default, then using JavaScript to send larger images on to iPads. That works, but it will definitely mean increased download times for new iPads since they have to download two files for every graphic. Apple’s approach will also up the number of HTTP requests, which will also slow down the page.

The slower page loads seem to be an acceptable trade off for Apple since the company no doubt wants to showcase the new iPad’s high resolution display with high resolution images. For other sites the bandwidth trade off may not be worth the gain in image resolution.

Still, screens are only going to continue getting better with ever-increasing pixel density. Now is the time, if you haven’t already, to start embracing CSS 3 (avoid images altogether with gradients, shadows and rounded corners in CSS 3), Scalable Vector Graphics (SVG) for resolution independent graphics and of course @media queries to serve high-res background images.

For more on detecting and developing for high resolution displays, check out these posts from around the web:

• Media Query & Asset Downloading Tests — Want to know how you can avoid the double image load tax Apple is paying? Tim Kadlec has the tests and results for a variety of methods.
• Optimising for High Pixel Density Displays. — Menacing Cloud’s take on optimizing for the iPhone 4 retina display.
• Mobile Web in High Resolution — Brad Birdsall’s take on bringing half pixel borders to high resolution devices
• Resolution Independence With SVG — David Bushell tackles SVG over at Smashing Magazine.
• Notes on Adaptive Images (yet again!) — Opera’s Bruce Lawson rounds up problems and solutions facing anyone trying to serve up different images based on screen size.