The HTML5 video element promised to be a game-changer for internet media publishing. It provided a vendor-neutral standards-based mechanism for conveying video content on the web without the need for proprietary plugins while offering a path for tighter integration of video content on the web and broader platform support than has historically been available through plugins.

But the HTML5 video element has yet to live up to its full potential, because a dispute over video encoding has prevented the standard from being implemented consistently across all web browsers. Mozilla, which has long resisted adoption of H.264 on ideological grounds, is now preparing to support it on mobile devices where the codec is supplied by the platform or implemented in hardware.

The popular H.264 format is widely viewed as the best technical choice for encoding Internet video, but its underlying compression technologies are covered by a wide range of patents. This has raised the question of whether its appropriate for a standards-based web technology to rely on a patent-encumbered video format that requires publishers and software implementors to pay licensing fees.

The ubiquity of the web and its strength as a platform for innovation are partly due to the royalty-free licensing model that the W3C mandated for web standards. As Mozilla and other parties have argued over the past few years, the use of a patent-encumbered video format is antithetical to the principles of the open web. Critics of the H.264 licensing model have advocated the use of other video codecs, causing a split in the browser landscape.

Apple and Microsoft both support H.264 while Mozilla and Opera oppose the use of patented codecs. Google previously favored H.264, but shifted its position after opening VP8, a codec that the search giant has put forth as a viable alternative to H.264 for Internet video. Google vowed to remove H.264 support from its Chrome web browser at some undisclosed future date, but has not yet done so.

The lack of universal support for a single codec has proved problematic because it compels content creators to either encode their video in multiple formats or fail to support large segments of their audience. Building consensus around a single codec would remove one of the biggest remaining impediments to widespread adoption of the HTML5 video element.

A Change in Course

Mozilla’s strong commitment to the open web made it seem as though the organization’s position was intractable. Mozilla’s resolve on the matter appears to have cracked, however, as the organization confronts the challenge of bolstering its credibility as a mobile platform provider.

Andreas Gal, Mozilla’s director of research, announced on a public mailing list today that he wants to proceed with a plan that would enable H.264 decoding on Mozilla’s Boot2Gecko (B2G) mobile operating system. The proposed change would allow the video element in Mozilla’s HTML rendering engine to rely on codecs that are supplied by the underlying operating system or dedicated video hardware.

In addition to enabling H.264 playback in B2G, the proposed patch would also enable it in the Android version of mobile Firefox. Gal further expressed support for eventually taking similar measures in the desktop version of Firefox, with the stipulation that it would only be practical if the implementation ensured support for virtually all users.

Modern versions of the Windows operating system expose an H.264 codec to third-party software, but Windows XP does not. Gal said that he’d favor supporting H.264 in Firefox on the desktop if a means could be identified for ensuring that XP users (which represent a very significant portion of Firefox’s audience) aren’t left out. This is a radical change of policy for Mozilla, one that could have significant ramifications for the future of video on the web.

Despite the pragmatic concession, Gal says that Mozilla’s ideological position in favor of open codecs remains unchanged. The organization is still hopeful that an unencumbered codec will eventually prevail.

“We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats,” he wrote. “I don’t think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.”

The option of using system-provided codecs is an obvious solution that would allow Firefox to play H.264 video without having to ship the code itself. We’ve discussed (and endorsed) this approach in some of our previous coverage, but Mozilla has historically rejected it on ideological grounds. In the past, Mozilla’s position was that it didn’t want to take any steps that would legitimize or encourage the use of a patent-encumbered codec. The organization is no longer maintaining that argument.

Google’s major investment in advancing its unencumbered VP8 codec gave open web advocates hope that H.264 could still be displaced, but it hasn’t happened. The lack of follow-through from Google on its promise to remove H.264 from Chrome has eroded faith in the search giant’s ability to popularize VP8. Gal says that it’s no longer feasible to wait for the open codec to gain additional traction.

“Google pledged many things they didn’t follow through with and our users and our project are paying the price,” he wrote. “H.264 wont go away. Holding out just a little longer buys us exactly nothing.”

The proposal to support H.264 in mobile Firefox has generated a tremendous amount of controversy among Mozilla developers. The critics include Mozilla employees and independent contributors. Mozilla’s Joe Drew characterized the proposal as “capitulating on Free codecs” and expressed concern that the mobile-centric rationalization amounts to pushing an ideological compromise through the back door.

Firefox developer Justin Dolske also expressed some concerns. He pointed out that the possibility of enabling support for system codecs was discussed once before in relation to Fennec on the Nokia tablet devices and that it was rejected at the time for ideological reasons. He asked that the issue receive further discussion, specifically some clarification about what circumstances have changed that necessitate a reversal of the previous policy.

“The state of HTML5 video started off from a bad place, and to be fair still isn’t in a good place. So reassessing Mozilla’s stance is not unreasonable. But I think if Mozilla is going to do an about-face on open video standards (and it is an about-face), then there should be some serious discussion about it. Certainly more than than a few terse words saying it’s hopeless and obvious,” he wrote. “We spent a lot of time and made a lot of blog posts about why H.264 was bad for the web. Leaving those who advocated for us suddenly high-and-dry doesn’t feel like the right thing to do.”

The debate has continued on the mailing list. There is also some preliminary discussion from certain participants in the debate about whether it would make sense at this point to simply license the codecs and ship them directly in the browser. Such a move, which would be a step further than merely supporting external codecs where available, would ensure support for Windows XP users but would detrimentally impact downstream distributors of Firefox code.

The outcome of the debate is unclear, but it currently appears probable that the plan to support system-provided codecs will be upheld and carried out. There are already some patches that have been hashed out, which means it can be practically implemented without much difficulty. The questions about how to proceed on the desktop and whether to license and ship the codecs are more tentative in nature and will likely take more time to be resolved.

This article originally appeared on Ars Technica, Wired’s sister site for in-depth technology news.

MPEG LA, the one-stop shop for motion video patent licenses, yesterday announced a call for patents essential to the VP8 video compression algorithm — the algorithm that is fundamental to Google’s WebM video format. MPEG LA is asking organizations that hold patents believed to cover integral, unavoidable parts of the VP8 algorithm to come forward and submit those patents to the licensing company. The patents will in turn by analyzed by MPEG LA, and those deemed to be relevant will be pooled together. The pooled patents will then be available to license as a single convenient bundle.

In its promotion of WebM and VP8, Google has insisted that all the relevant patents were developed by codec company On2, which Google purchased last year. The patents can be licensed from Google without payment of any royalties or any restrictions on usage. Google has been heavily promoting WebM for use with the HTML5 <video> tag, which allows plugin-free video to be embedded in webpages, and the royalty freedom is a key part of WebM’s value proposition.

Competitive codecs such as the open and industry standard H.264 require royalties to be paid by software and hardware developers. Companies like Opera and Mozilla, as well as the W3C group that is developing the HTML5 specification, deem these royalties be an unacceptable impediment to their usage. They have no such qualms about the royalty-free WebM.

If MPEG LA is successful in assembling a patent pool, that royalty freedom could come to an end. The company is soliciting patent submissions until March 18th. Once the submissions have been made, it will determine which patents are essential to VP8; only those patents that are unavoidable can form part of the patent pool. The owners of those selected patents will then decide on the license conditions they wish to impose, and these conditions could include royalty payments.

Whether this will happen, of course, is the big question. MPEG LA might fail to form a patent pool altogether: it may receive no relevant patent submissions, in which case the patent pool process will likely end. Such an outcome still won’t mean that WebM is in the clear — a company may feel that it’s more lucrative to avoid a patent pool and allow WebM usage to become more widespread before asserting claims — but it would probably imply that there aren’t dozens of potential claimants just waiting to come forward.

This sort of outcome might well see Microsoft’s current neutral stance towards WebM (it will work in Internet Explorer 9, just as long as a suitable third-party codec is installed) become more overtly positive. Redmond might start shipping a WebM codec of its own, for example.

If MPEG LA does form a patent pool, the license terms will be critical. MPEG LA exists to monetize patents, however, so it’s unlikely that any patent pool would permit the kind of indiscriminate royalty-free license that Google currently offers. More likely, they would choose terms similar in kind to those of H.264; Web video may be free, but decoders still incur a royalty. This would put WebM implementors in a difficult position — either drop WebM support, pay up, or risk going to court to fight a patent infringement suit.

An infringement suit is an unappealing prospect: even if you win, the drain on your financial resources can mean that ultimately, you lose. This is especially problematic for organizations like Mozilla, since Google offers no indemnification for users of WebM — if Mozilla gets sued, Google won’t step in to help. As such, the safest, most conservative option for Opera and Mozilla would be to drop support. Google has deeper pockets and can better sustain a legal attack, but even there, the company has to weigh its options carefully. A lost court case could cost tens of millions of dollars. Paying up just to avoid the problem may very well be the better option.

But paying up is problematic too. VP8 is, for most purposes, inferior in quality to H.264. H.264 is much more widespread in software tools, hardware accelerators, and so on: it’s enormously widespread already. If VP8 loses its key feature — royalty freedom — implementers may very well decide that, since they have to pay anyway, they’d be better off paying for the superior, more widely used H.264 license, and abandoning WebM entirely.

Whatever happens — and it will probably be many months before we find out — this is bad news for WebM. The formation of a patent pool directly undermines Google’s claims about the codec — and yet, even if MPEG LA fails to create a pool, question marks surrounding the codec will remain.

This article originally appeared on Ars Technica, Wired’s sister site for in-depth technology news.

See Also:

• Microsoft Puts H.264 Video Back in Google Chrome, Considers WebM for IE
• Google Dropping H.264 Codec from Chrome Browser [Updated]
• MPEG LA Extends Web Video Licensing Moratorium Until the End of Time
• Major Browser Vendors Launch WebM Free Open Video Project

Microsoft has announced a plug-in for Google’s Chrome web browser that allows Chrome on Windows to play H.264 web video through the HTML5 <video> tag. The new plug-in comes on the heels of Google’s decision to remove H.264 support from Chrome and focus on the company’s competing WebM video codec.

You can grab the new Chrome plugin from Microsoft. Microsoft previously released a similar H.264 plugin for Firefox, which also only supports WebM video.

The video move is the latest sign of a collision between the two tech giants, who now compete directly in search, courtesy of Microsoft’s Bing initiative and mobile, where Google’s Android is taking market share and the new Windows Phone 7 is struggling for a foothold. Google has also launched various cloud-based applications that take aim at Office. This week, the two threw punches over search, with Google claiming Microsoft copies its results, and Microsoft complaining the Google perpetrated a sting worthy of a spy novel.

Now the two are sparring over web video. Google has thrown its weight behind the WebM codec, which the company owns, while Microsoft supports H.264. However, Microsoft says that, provided Google makes some changes, it may be willing to support the WebM codec as well.

While HTML5′s video tag promises a native way to watch video in your browser, video codec support among browsers is divided. Firefox, Opera and Chrome support the WebM codec while Apple’s Safari and Microsoft’s IE9 support H.264. As it stands there is no “it just works” solution, which means most websites still use Flash video players.

Microsoft’s H.264 plug-ins for Firefox and Chrome are part of the company’s attempt to be pragmatic — since Windows includes native support for H.264, users should be able to watch H.264 video even if the browser doesn’t support it. On the other side of the coin, Internet Explorer 9 will be able to play WebM video through a similar third-party plug-in.

However, while Microsoft isn’t including native support for WebM in the next version of IE, it doesn’t appear to totally rule out the idea. As part of the plugin announcement, Dean Hachamovitch, corporate vice president for Internet Explorer, outlines some of Microsoft’s problems with the WebM codec. The main problem is that Microsoft is concerned about WebM’s potential patent risks.

Google insists that it owns all of the patents covering WebM and the VP8 video codec. But the company offers no indemnification for costs incurred should a patent lawsuit arise. That means that anyone distributing WebM/VP8 could be on the hook for any patent-related fees that might come up.

Some have dismissed Microsoft’s patent worries as an example of Microsoft spreading “fear, uncertainty and doubt” about WebM, but Microsoft does have history on its side in this case. As Hachamovitch points out, such patent lawsuits often don’t arise until a technology is in widespread use. So just because no one is suing over WebM now, doesn’t mean they won’t in the future. Hachamovitch cites the JPEG photo compression format, pointing out that JPEG was around for ten years before the first patent lawsuits appeared. Eventually the patents in question were ruled invalid, but not before millions of dollars were spent defending and licensing JPEG.

Of course the same patent threats potentially hang over H.264, but the MPEG-LA consortium — the governing body that oversees the patents surrounding H.264 — provides a kind of legal buffer between H.264 licensees and any lawsuit.

Surprisingly, Hachamovitch says that, if Google is willing to indemnify WebM users against patent lawsuits, “Microsoft is willing to commit that we will never assert any patents on VP8.” Of course that doesn’t mean other companies won’t, but it would be a huge step forward for WebM if Microsoft jumped on the bandwagon. Google did not respond to a request to comment in time for this story.

For now at least Microsoft has chosen a pragmatic approach — plugins. There will be a WebM plugin for Internet Explorer and H.264 plugins for Firefox and Chrome. In the end, Windows users will be able to watch just about any video on the web regardless of which browser they’re using. It might not be an ideal solution, but it is one that, from the user’s point of view, just works.

See Also:

• Microsoft Adds H.264 Video Support to Firefox
• Google Dropping H.264 Codec from Chrome Browser [Updated]
• Major Browser Vendors Launch WebM Free Open Video Project
• Microsoft Says Web Video in IE9 Is All About H.264

Microsoft’s next browser will support native playback of videos using HTML5, but it will only support H.264, and not its more open alternatives.

In a post on the official IEBlog Thursday, Microsoft’s general manager of Internet Explorer Dean Hachamovitch outlined his company’s position in the ongoing Flash vs. HTML5 video debate. He says that when it comes to playing web videos without plug-ins, Microsoft will support H.264-encoded videos in its browser. He makes no mention of those encoded with Theora or any other codecs, and nobody is expecting Microsoft to support anything other than H.264 — Hachamovitch first mentioned singular support for H.264 in IE9 last month when he showed off an early version of the browser.

The argument over which web video playback technology to support has been a point of major tension among browser makers ever since last year, when the Worldwide Web Consortium (W3C) bowed out of the debate, declining to recommend any single video technology for HTML5. The result, so far, is a stalemate — Microsoft and Apple are supporting H.264, Mozilla and Opera are supporting Ogg Theora and Google, for the time being, is supporting both.

As we’ve said before, H.264 is a dangerous path for web video to go down, mostly because there are patents and licensing issues associated with it that keep it from being freely used. It should be noted that both Microsoft and Apple — the two main proponents of native H.264 playback in their browsers — hold patents in the H.264 patent pool.

Other technologies, such as Ogg Theora and VP8, appear to be a much safer alternative for video on the web to remain free and open, which is why the browser makers who have no stake in H.264 (Mozilla and Opera) are pushing for Theora.

Google Chrome’s support varies based on platform, and there’s a rumor the company will release the VP8 video technology it now owns under an open source license soon.

Curiously, there’s no mention of Silverlight in Hachamovitch’s post. But he doesn’t tie Flash to the whipping post like so many others have been quick to do. His words on Flash are quite tempered. Diplomatic, even:

Today, video on the web is predominantly Flash-based. While video may be available in other formats, the ease of accessing video using just a browser on a particular website without using Flash is a challenge for typical consumers. Flash does have some issues, particularly around reliability, security, and performance. We work closely with engineers at Adobe, sharing information about the issues we know of in ongoing technical discussions. Despite these issues, Flash remains an important part of delivering a good consumer experience on today’s web.

See Also:

• Can Google Save Free, Open Web Video With VP8?
• Firefox Quarantines Video Plug-ins to Stop Browser Crashes
• Developer Gives Internet Explorer the Gift of HTML5 Video

One the surface it sounds like a good thing — at least until 2016, you’re free to post H.264 videos on your web site without paying royalties to MPEG-LA, the controlling body. But after 2016, MPEG-LA could charge you whatever it wants — even an Austin Powers-style one million dollars per second of video.

MPEG-LA’s latest move seems ripped straight from a crack dealer’s marketing guide — “Here kid, the first hit’s free.” Then, once the web is even more heavily invested in H.264 than it is now, MPEG-LA can set its royalty fees at whatever rate it wants, sit back and reap the profits.

This news comes at a time when the web is in a heated debate over how to best display videos in the browser. The vast majority of content providers rely on Flash (which can decode H.264) to show videos. The certainty of Flash’s longevity on the web was thrown into question by the recent arrival of the iPad, which, like the iPhone, iPod Touch and other mobile devices, doesn’t support the Flash Player software. Some sites are experimenting with using HTML5 to display videos in either H.264 or Ogg Theora file formats. But different browser makers have chosen to support different file formats because of the licensing complexities — Mozilla, Apple, Opera and Google are all picking different sides.

It’s important to understand that the royalty fees being deferred by MPEG LA are in addition to the licensing fees the group already has in place (at over US$50,000 per year). Proponents of H.264, along with many unaware users, often argue that the licensing fees are irrelevant because web users like you and I remain unaffected by them.

But that doesn’t mean that the licensing fees won’t affect the web. Sure, the fees are no big deal for Apple, YouTube and other established players, but what if you want to build a web video encoding service to compete with YouTube and Vimeo? Well, if you want to serve your video to iPhone/iPod Touch/iPad users you’re going to need to come with $50,000+ in licensing fees.

Even without the royalty fees arriving in 2016, the licensing costs alone put start ups at a disadvantage, meaning that an H.264-encumbered web might well miss out on the next big leap in web video sharing.

Then there’s the decoding side of the equation.

At least part of the reason Mozilla and Opera refuse to support H.264 is the licensing fee necessary for software that decodes H.264. While both companies can likely afford it, smaller players can’t. For example, if you want to distribute your own version of Firefox, or simply create something totally new — some next-generation web browser or add-on based on Mozilla code — again, get ready to pony up the licensing fees if you plan to support H.264.

Even using Flash to decode H.264 doesn’t protect you from the licensing fees. As the Adobe H.264 page notes: “commercial use of the Flash Player to decode H.264 video may require a separate license.”

We’re not saying there’s anything wrong with H.264 or MPEG-LA’s desire to make money off it, but let’s not delude ourselves — H.264 isn’t a viable solution for the web’s open video woes.

See Also:

• YouTube Embraces HTML5, But Stops Short of Open Web Video
• Why Flash Isn’t Going Anywhere, iPad Be Damned
• W3C Drops Audio and Video Codec Requirements From HTML 5