Google's WebM v H.264: who wins and loses in the video codec wars?

Google announced last week that it is axing support for the H.264 video codec from its Chrome browser. (Only the one it distributes for desktops, at the moment; but it's not clear whether the Android browser includes an H.264 codec. We'll come to it.)

The ripples from this are still spreading around the web, and their echoes coming back. The questions are twofold: why, precisely, did Google do this? And who is going to win and lose from it?

Let's set the scene.
Adobe's Flash is the dominant method by which video is viewed in desktop/laptop browsers.
Google's Chrome browser is raising its share steadily, through advertising around the web and physically. (It's now ahead of Apple's Safari, for example.)
The patent-encumbered H.264 video codec is the dominant method at present by which mobile video is viewed, because Apple's iPhone, iPod Touch and iPad dominate mobile viewing, and they don't do Flash (because of stability, security, CPU usage and control of platform direction). Because of this Google's YouTube has re-encoded pretty much everything into H.264, in parallel with the Flash version for desktops and for Android and other phones (the latter a growing share of mobile browsing).

And now Google is going big on WebM. The move had been expected ever since Google announced it was going to open source the VP8 video codec after it bought On2. That was last May - described here then as "Google leads WebM fightback against H.264 video".

The idea then was that

"Google is combining with Mozilla (Firefox) and Opera in the WebM project to create a Matroska file format using VP8 and the Ogg Vorbis open source audio codec. It should enable any HTML5 web browser and any video player to play video."

Mozilla (aka Firefox) and Opera didn't like the H.264 codec, which is licensed via a patent pool in which the last item doesn't expire until 2038. (Some have already expired, but the pool lives on until the last one goes.)

Using H.264 involves paying money - for Google, a growing amount of money. The licensing cost can be raised by 10% every year (on agreements that are renewed every five years), and the amount payable is due annually.

Might Google's objection to H.264 be because of the cost? Unlikely. Florian Mueller (who is absolutely no fan of software patents) noted last June that MPEG-LA (which looks after the patents) already has a $6.5m cap on licensing (from 2011-2015), which will next be reviewed in 2016. That covers everything - encoders and decoders. (Read the terms [PDF]).

So Google has a motive there to go open source, even if WebM (as we'll see) isn't as efficient or fast as H.264. But more importantly, don't forget that Google is run by its engineers. It's driven by their decisions. Certainly someone higher up will have approved the decision, but this will fundamentally have been a decision that was done because a Google engineering team thought it was the right one in principle. Not because they thought it would screw up Apple. Nor because they thought it would strengthen Adobe. Nor even necessarily because they're against H.264 and software patents (though those would be things they don't like).

Announcing the move on the Chromium blog at the start of last week, Mike Jazayeri, the Chrome product manager, said that open-sourcing VP8 has meant performance improvements, "broad adoption by browser, tools and hardware vendors" and "independent (yet compatible) implementations".

But, he continues, at Google they

"are focusing our investments in those technologies that are developed and licensed based on open web principles. To that end, we are changing Chrome's HTML5 <video> support to make it consistent with the codecs already supported by the open Chromium project. Specifically, we are supporting the WebM (VP8) and Theora video codecs, and will consider adding support for other high-quality open codecs in the future. Though H.264 plays an important role in video, as our goal is to enable open innovation, support for the codec will be removed and our resources directed towards completely open codec technologies."

In other words, Google is taking H.264 for a walk out the back and only one of them is returning. Support will be dropped in forthcoming (at some point) versions of Chrome.

Before we move on, let's just ask what those performance improvements Jazayeri refers to are. The linked page in his post says that compared to the May 2010 release, it's 20-40% faster at decoding, and 7% improvement in best-quality encoding. Real-time encoding speed: not great, it seems, or not good enough yet for videoconferencing, going by the comments.

Performance is not something that has been mentioned much before over WebM. A test from last summer (before those performance improvements) of H.264 and VP8 found that VP8 often did well - sometimes even better - than H.264.

But there's a catch: "When comparing VP8 and x264, VP8 also shows 5-25 times lower encoding speed with 20-30% lower quality at average. For example x264 High-Speed preset is faster and has higher quality than any of VP8 presets at average."

Which means there's some way to go before VP8 is as good as H.264, even with those performance improvements mentioned.

Which means Google's decision is about the open source thing.

In a followup post on the Chromium blog on Friday, Jazayeri points out that Firefox (the second-biggest browser) and Chrome and Opera support WebM (because it's open) and Internet Explorer (the biggest browser) and Safari (in all its forms, including - and this is important - mobile) support H.264. Microsoft has said it won't have native support for WebM in Internet Explorer 9.

Jazayeri:

"We acknowledge that H.264 has broader support in the publisher, developer, and hardware community today (though support across the ecosystem for WebM is growing rapidly). However, as stated above, there will not be agreement to make it the baseline in the HTML video standard due to its licensing requirements."

He argues that to Google, it's not a lot of money, but "to the next great video startup and those in emerging markets these fees stifle innovation."

Next, the killer question: Won't this decision force publishers to create multiple copies of their videos? Which is a very big question indeed. But the key thing about this question is that it's a very big question for HTML5 video - not for what you might call the HTML4 generation, which is most likely the device you're using now: a desktop or laptop.

Here's where Jazayeri gets rather sneaky. "Firefox and Opera have never supported H.264 due to its licensing requirements, they both support WebM and Ogg Theora." True. "Therefore, unless publishers and developers using the HTML <video> tag don't plan to support the large portion of the desktop and mobile web that use these browsers, they will have to support a format other than H.264 anyway".

Ummm. That's tricky. Opera is very widely used, it's true: it's on lots of Nokia phones. But they are not used for the majority of mobile video viewing. That's done by MobileSafari - Apple's browser - which dominates for HTML5 video viewing. (Android phones don't use the <video> content: they use Adobe Flash in a plugin. For the purposes of this evaluation, they simply don't have an effect - yet.)

So we come to Jazayeri's closing remarks:

"Bottom line, we are at an impasse in the evolution of HTML video. Having no baseline codec in the HTML specification is far from ideal. This is why we're joining others in the community to invest in WebM and encouraging every browser vendor to adopt it for the emerging HTML video platform (the WebM Project team will soon release plugins that enable WebM support in Safari and IE9 via the HTML standard <video> tag)."

Which is the point that nobody who wants this to be a riotous catfight seems to have noticed: if Apple wants, it can write a WebM plugin for MobileSafari. That's if H.264 availability starts to fall off.

Which, if you think about it, it might well do. Google owns YouTube, the world's biggest video sharing site. It wouldn't be the most enormous shock to see a similar announcement come out of YouTube - that it's going to be encoding content in Flash (perhaps H.264 in a wrapper) and WebM.

That could theoretically create a problem for Apple - except it's a company which has written the occasional bit of software, including stuff using open source - MobileSafari is based on WebKit which is open source.

How about Microsoft? It might have the most to lose from not including a WebM plugin in Internet Explorer 9 (if you assume that IE9 won't ramp up in use as quickly as others have, and that IE generally will continue to lose share). Indeed, a sarcastic blogpost from Tim Sneath, a Windows and web evangelist at Microsoft, suggests the WebM move makes as much sense as declaring that everyone should now speak Esperanto (the "universal" language that barely anyone speaks): "Though English plays an important role in speech today, as our goal is to enable open innovation, its further use as a form of communication in this country will be prohibited and our resources directed towards languages that are untainted by real-world usage".

Sneath may simply be making fun, though. Google shifting the video goalposts like this, despite the small share of Chrome, matters - because Firefox 4, the next version of the most popular browser, will have support for WebM. And of course Firefox is the second-largest browser. It won't support H.264. It will support Flash (and Apple's Quicktime and Microsoft's Silverlight) plugins. There, it's not about the open source thing. It's about the money thing.

We're nearly at the end, but WebM isn't quite out of the woods yet. Expect that the lingering threats of patent lawsuits - Steve Jobs hinted at them last May - will be brought to bear.

More importantly, the difficult trick will be getting WebM's performance improved. Video codec programming is a really specialist niche, and Google's apparently slow progress so far in improving WebM suggests that it's going to have to put some effort in - or bring in some specialists - to push things along. Otherwise, those video producers aren't going to be pleased at the idea that they've been forced to give up video quality so that Google can promote an open web.

So let's sum up.

Google: winner. It gets to decide direction of HTML5 video codecs. If WebM somehow bizarrely fails, it can easily afford to pay for H.264. Expect YouTube to make an announcement, if/when WebM encoding improves enough, that it's going to be using WebM for HTML5 video in future.

Adobe: winner now, possible long-term loser. It's still ensconced on the desktop, and on Android phones - though might Google dump it for WebM on its Android phones in the future because (a) WebM is its own (b) Flash might have security issues? (And you should expect that Android phones will begin to make up a very large share of smartphones in the next few years.) The question for Adobe is whether people will keep making content in Flash if WebM catches on as the HTML <video> codec. But there's nothing to stop it being able to use WebM, just as it already uses H.264.

Big-on-the-desktop Firefox (and teeny-on-the-desktop Opera): winners. They will have WebM plugins, so sites could move to WebM rather than Flash encoding and get the benefit of one-time encoding, and these browsers won't frustrate users.

Apple: short-term inconvenience, not necessarily a loser. Having to shift to WebM would be a pain, and Steve Jobs will mightily curse Google and all its doings, but this is software, and it's video. Apple pretty much invented that back in the day with Quicktime. It's a software update if YouTube shifts to WebM.

Notice how different this is from Apple's relationship with Flash: Adobe, not Apple, writes the Flash plugin, which is what frustrates Apple. But here Apple would choose to write the WebM plugin (assuming Google is going to open source it, which it says it is. The cat will be among the pigeons if this is an Android - apparently open but actually with some secrets hidden inside the Googleplex.)

Meanwhile Apple will continue using H.264 all over the place, such as for iChat video, device video (it gives developers an encoding system), video output encoding... and Steve Jobs may try to corral content producers to lobby Google/YouTube on the basis that WebM is a big hassle, and the quality isn't as good. It's hard to know how that will play at Google's engineer-driven base though.

You and me: slightly confused neither-winner-nor-loser. Neither of us is paying H.264 licensing fees. Neither of us greatly cares whether our smartphone gets served WebM or H.264 video - unless the quality and/or bandwidth required is noticeably worse/greater with WebM. In that latter case, we become losers - viewing on a mobile will use more bandwidth/look worse, meaning we'll have been sacrificed on Google's engineers' altar of it seems like a good idea in principle.

In fact, this is the sort of thing that in the end doesn't matter to users as much as the noise around it makes it feel. Yes, WebM is open source, and that's a good thing generally. Its performance isn't great, and that's a bad thing. The question is whether Google can tie the two together and improve it without getting slammed by patent lawsuits that could cost it a hell of a lot more than just licensing H.264 until 2038 would ever have done.

But then, some things are best done on principle.