All posts tagged ‘responsive images’

Forget JavaScript, It’s Time for Browsers to Speed Up Images

The average webpage is now 1.2 megabytes and around 60 percent of that rather large payload comes from images. That’s a lot of data, whether you’re handling images responsively or just trying to speed up a desktop site.

You might think, if images are the bulk of what your browser is downloading, that browsers would be working hard to speed up the image downloads, perhaps trying alternate, space-saving image formats, but you’d be wrong.

You might also think that, as Google’s Ilya Grigorik writes, “innovating on better image formats would be a top agenda item” for the web. But again you’d be wrong. The web is still using the same image formats it’s been using virtually since the first images appeared online.

Grigorik thinks it’s high time that changed and we agree.

In a recent post looking at what it would take to deploy new image formats on the web, he writes, “if we really want to make an impact on web performance, then image formats is the place to do it… there is absolutely no reason why we shouldn’t have dozens of specialized formats, each tailored for a specific case and type of image.”

Of course no web developer wants to deal with dozens of specialized image formats. Nor should they need to — that’s a job for servers. “In a world with dozens of image formats,” continues Grigorik, “the human solution does not scale — read, markup does not scale… whereas computers are fantastic at doing exactly the kind of optimization work required to solve the problem.”

Grigorik isn’t alone in calling for new image formats, nor is he the first to suggest handing these tasks off to the server. Developer and responsive images proponent Matt Wilcox has argued for a similar solution, as have others.

The basic premise of these arguments is that deciding which image to serve up to which device and browser should be a server-side problem. And in fact there’s already a way to solve this problem with HTTP headers, namely the Accepts header, which tells the server which image formats the browser supports. Based on that information the server could then “re-encode, recompress, resize, strip unnecessary metadata and deliver the optimal format.”

The problem is that web browsers (with the exception of Opera) don’t actually send useful information in the Accepts header.

Thus, the first step in creating a server-side solution for smaller images is to get other browsers to send useful Accepts headers.

The Accepts header isn’t a magic bullet by any means, but it’s a problem that’s not hard to solve provided browser makers prioritize it. But to really get server side image solutions working the web would also need new server tools (fortunately, several already exist). There are other stumbling blocks as well. Grigorik addresses half a dozen potential problems and objections that you can read through in his post.

Even if browser makers come around to the idea and do start improving Accepts headers, bringing better image formats to the web is going to be an uphill battle. But Grigorik is determined to chase the idea. “Some uphill battles are worth fighting,” he writes in a comment, “I think this a good one. Wish me luck.”

W3C Publishes First Draft of HTML Responsive Images Extension

The web needs a more intelligent way to serve images; one-size-fits-all doesn’t work in an increasingly mobile world.

Web developers need a way to specify images based on screen size, bandwidth and other factors. To help them do that the W3C, the group that oversees the HTML standard, has published the first editor’s draft of the HTML Responsive Images Extension. The proposal is just a draft, but it points to one possible solution for the responsive image conundrum.

The problem is this: No one wants to waste bandwidth sending large images over limited mobile pipes, but everyone wants images to look good on the myriad screens connecting to today’s web. Developers need to send small images to small screens and large images to large ones. Currently, web authors use a variety of hacks to (incompletely) work around the problem, but to really solve it the web needs new tools.

The new proposed Responsive Images Extension is hoping to be that new tool. The proposal is one of the first community-created specs, coming from the work of the Responsive Images Community Group. The spec is edited by developer Mat Marquis, who represents the Community Group, and Microsoft W3C member Adrian Bateman.

The draft seeks to find a middle ground between the WHATWG’s proposed srcset attribute for the img tag and what many web developers want — a new <picture> element.

The goal for developers is to have a simple, but backward-compatible way to serve images based on screen pixel dimensions, density, user zooming and bandwidth. For the latter image choice would be left up to the browser.

In addition to those basic requirements, the Responsive Image Extension proposal has other goals:

  • Fallback gracefully on older user agents
  • Can be polyfilled effectively
  • Retains, at a minimum, the same level of accessibility as current img element
  • Preserves separation of content markup and styling
  • Provides a purely client-side solution that can include JavaScript, but doesn’t require it
  • Supports use cases where authors need to explicitly define different image versions as opposed to simply different resolutions of the same image
  • Provides a consistent and predictable pattern for delivering alternate media sources based on client context
  • Supports succinct but understandable mark-up

Put it all together and clearly the HTML Responsive Images Extension has its work cut out for it. Here’s some sample code showing how the proposed picture element might handle all that:

<picture alt="">
    <source media="(min-width: 45em)" srcset="large-1.jpg 1x, large-2.jpg 2x">
    <source media="(min-width: 18em)" srcset="med-1.jpg 1x, med-2.jpg 2x">
    <source srcset="small-1.jpg 1x, small-2.jpg 2x"> 
    <img src="small-1.jpg"> 

The first line contains a media query aimed at large screens and the uses the srcset attribute to specify two images, one for standard resolution displays and one for high-DPI screens. The second line handles medium size screens, while the third handles smaller screens and serves as the fallback for low-bandwidth situations. The last line uses the standard img tag for older browsers.

It’s a mouthful of markup, but it’s similar to how the HTML5 <audio> and <video> tags work. Unfortunately it’s sufficiently different enough to be confusing. As Philip Jagenstadt of Opera Software notes, “personally, I don’t think it’s a good idea to reuse the syntax of <video> + <source>, since it invites people to think that they work the same when they in fact are very different.” He goes on to add that “this is neither a “yay” nor “nay” from Opera Software, just my POV from working on HTMLMediaElement.”

And therein lies one big problem for the HTML Responsive Images Extension — browsers aren’t on board with it just yet.

For now that’s fine, the draft is, at this stage, just a proposal in search of feedback. If all goes well browser makers will now step into the fray to enumerate their concerns and any potential pitfalls on the implementation side. And there are pitfalls, including potential conflicts with browsers’ “look-ahead” parsers, but hopefully over time those will be worked out.

If you’ve got ideas on how to improve <picture>, head over to the Responsive Images Community Group and let them know what you think.

How Do You Use Responsive Images?

No one wants to waste bandwidth sending large images over limited mobile pipes, but everyone wants their images to look good on the multitude of screens connecting to today’s web. Finding the balance between the myriad factors — screen resolution, screen size, bandwidth, layout and design — is a tricky process.

We’ve looked at a number of techniques for handling images in responsive designs, but one thing that we haven’t covered is where to put the actual breakpoints.

You’re probably familiar with breakpoints in responsive design, they’re the screen widths in your media queries. For responsive images the idea is the same; a “breakpoint” is the screen size at which you swap in and out different size images.

At first glance it might seem logical to use the same breakpoints for your images that you’re using in your CSS media queries, but as Cloud Four’s Jason Grigsby writes, that’s not always the ideal solution. Grigsby tackles the potentially tangled question of breakpoints in a new post, So How do You Pick Responsive Images Breakpoints?

To simplify what can be a very complex question, Grigsby ignores some scenarios, including the so-called “art direction” use case where images are optimized (cropped differently, for example) for different screens. Instead Grigsby focuses on the question of how to best select “different image files with different resolutions of the same image based on the screen size.”

The simplest solution is to just make your image and responsive design breakpoints the same, but that’s rarely going to be ideal for your site’s visitors. Here’s what Grigsby has to say about using the same breakpoints for both image and media queries:

If we were talking about the art direction use case, then it is likely that the breakpoints would be the same because changes in the layout might also indicate an edit to the image.

But in the case where we’re simply changing files to provide different resolutions and a faster download, the image breakpoints should be determined based on when the browser is downloading an unnecessarily large file.

The main problem is, what constitutes an “unnecessarily large file”? But even if you answer that, as Grigsby writes, you still haven’t answered every question:

How do you determine what is an unnecessarily large file? Is that 1 byte? 10k? 20k?

And if you can answer that question, how do you determine the resolutions at which those file size jumps are going to occur? Depending on the content of an image and the compression algorithm used, images are likely to have different resolutions at which they experience significant changes in file size.

In the end Grigsby doesn’t yet have an answer to the question of how to handle responsive image breakpoints. And neither do I. There just isn’t an easy answer that will work for every project. My thinking, and what I’ve done on my own site, runs pretty much along the same lines of Eric Portis’s comment on Grigsby’s post. If you’ve got ideas, head over to Cloud Four and let them know how you’re doing it.

File Under: HTML, HTML5, Web Standards

Browsers at Odds With Web Developers Over ‘Adaptive Images’

The web design community continues to debate the merits and drawbacks of the WHATWG’s proposed adaptive images solution.

As we reported last week, a new srcset attribute has been added to the <img> element in the WHATWG’s HTML specification. The new attribute will allow developers to specify different sized images based on the user’s screen size.

The idea is to find a way to serve smaller images to devices that don’t need large images — saving precious bandwidth — while serving high-resolution images to screens that warrant them. And the WHATWG’s srcset attribute does solve some of the problems surrounding adaptive images, but it’s far from ideal.

Now developer Jason Grigsby argues that not only will the srcset solution not completely solve the problem, but the goal of adaptive images is fundamentally at odds with how web browsers currently handle images. In other words, there currently is no way to solve the problem.

Adaptive images are at odds with how browsers handle images thanks to what’s known as a “lookahead pre-parser.” Browsers use lookahead pre-parsers to start downloading images as soon as possible (to speed page-load times), which means images are parsed and downloads started before the browser has determined the full page layout.

However, a truly useful adaptive image solution needs the browser to first determine the page layout and then determine which images to use.

Grigsby rightly calls it a chicken and egg dilemma. “How do we reconcile a pre-parser that wants to know which size image to download ahead of time with an image technique that wants to respond to its environment once the page layout has been calculated?”

Grigsby argues that the smart thing to do might be for browsers to eliminate pre-fetching:

For existing web content, the lookahead pre-parser is undoubtedly the fastest way to render the page. But if web development moves towards responsive images as standard practice, then delaying the download of images until the proper size of the image in the layout can be determined may actually be faster than using the lookahead pre-parser. The difference in size between a retina image for iPad and an image used on a low resolution mobile phone is significant.

That’s going to be a tough sell to browser makers right now. Browser makers are understandably loath to do anything that might slow down page-load times — even if that slow-down is temporary.

Other possible solutions Grigsby covers include progressive image formats (which suffer from similar chicken-and-egg dilemmas) and of course the <picture> element. The whole article is well worth a read since it gets into more details about why all of these solutions are ultimately less than ideal.

File Under: HTML, HTML5, Web Standards

Ready or Not, Adaptive-Image Solution Is Now Part of HTML

The web needs a more intelligent way to serve images.

No one wants to waste bandwidth sending large images over limited mobile pipes, but everyone wants images to look good on the myriad screens connecting to today’s web. Currently web authors use a variety of hacks to (incompletely) work around this problem, but to really solve it the web likely needs new tools.

Unfortunately, thanks to miscommunication between standards bodies, web developers and browser makers, instead of a solution to the image problem what developers got this week feels more like a slap in the face. Eventually an adaptive image solution will likely emerge, but the real lesson for many developers will be about how the standards process works and how they fit into it, if at all.

Webmonkey has previously looked at some proposed solutions to the adaptive image problem. Some very smart web developers came up with the idea of a <picture> element that works much like the current HTML <video> element. These developers thought they had the attention of the Web Hypertext Application Technology Working Group, better known as the WHATWG. Then, earlier this week, Edward O’Connor, Apple’s WHATWG representative, proposed another method of solving the problem, using a new srcset attribute on the <img> element. See our earlier coverage of the srcset attribute for a more detailed look at how it works and compares to the <picture> proposal.

What has web developers up in arms is that Ian Hickson, editor of the WHATWG spec (and better known as Hixie) has already added the srcset attribute to the WHATWG’s HTML draft spec, seemingly ignoring the months of effort that went into <picture>. Worse, members of the WHATWG apparently weren’t even aware that developers were putting forth the effort to come up with a solution via the Responsive Images community group. Nor were concerns about the srcset syntax given much consideration. Hickson does address some objections to srcset in his message to the WHATWG, but ends up dismissing most of them.

That doesn’t match up with how most people envision the web standards process. But as web developer and standards advocate Jeremy Keith writes, “this is exactly how the WHATWG is supposed to work. Use-cases are evaluated and whatever Hixie thinks is the best solution gets put in the spec, regardless of how popular or unpopular it is.”

In fact, think of the WHATWG as the source for initial, rapid development of new features. The group was started by browser makers because the W3C’s HTML Working Group (HTMLWG) moved too slowly. But if the WHATWG is the source of rapid development, the W3C is an effective check on that speed, ensuring that even those of us who don’t make web browsers still have a voice in the future of HTML. (see our earlier overview for more on the history and differences between the HTML WG and the WHATWG.)

While the HTML WG is also chaired by Hickson (a position he will soon step down from), it offers a much more democratic (and consequently slower) process and has overridden the WHATWG’s rash decisions in the past. For example the W3C added the time element back after Hickson removed it from the WHATWG spec.

Confused yet? It gets worse. The WHATWG is working on an ever-evolving standard, what it calls a “living standard,” which is different from — and may well diverge from — the snapshot-based standards issued by the W3C, like HTML5. In a comment on longtime web standards champion Jeffery Zeldman’s post on the matter, Jeremy Keith writes, “I don’t mind if the srcset attribute is in the WHATWG HTML spec but not in the W3C HTML5 spec. If it works, it’ll end up in a future W3C version number.”

Implicit in Keith’s statement is that if the srcset attribute doesn’t end up working out it won’t be in HTML5.x and would likely just fade away like the blink tag, the applet tag and other HTML ideas tried and later discarded.

Which is another way of saying developers need not panic. Perhaps web developers don’t have a voice in the WHATWG simply because we’ve been using the wrong channels (W3C community groups don’t seem to be an effective means of communicating with standards bodies, in fact they seem more like this.). If you’ve got ideas and would like a voice in the future of the web join the WHATWG mailing list and login to the IRC channel. Introduce yourself, learn the rules and contribute.