Google’s Stefano Cazzulani, Chrome Product Manager, writes on the Chromium blog that, despite a plethora of benchmarks on the web, Google wanted a new suite with “new benchmarks created from full, unaltered, well-known web applications and libraries.” The result, says the company, is a test suite that better reflects performance in “real web applications.”
Of course what constitutes “well-known web applications and libraries” is left to Google’s Chrome team to decide, and, perhaps not coincidentally, Chrome scores quite well on Octane’s hand-picked suite. That’s not to say that Chrome isn’t actually quite fast, but it does highlight the main problem with browser-maker benchmarks — the browser vendor creating them almost always seems to score the highest on them.
I ran the latest version of each of the major web browsers through Octane on a 2008 MacBook Pro (average of five runs each):
- Safari 6: 6007
- Chrome 21: 8517
- Firefox 14: 5351
- Opera 12: 3330
- Internet Explorer 9: (tested in VMWare, but IE9 didn’t render the page.)
- Mobile Safari (iPad 3): 553 (incomplete test, typed arrays aren’t supported in Mobile Safari).
Naturally the results will vary depending on your hardware, particularly your graphics card, but in all my tests Chrome won by a large margin.
To see the actual tests — which include a 2D physics engine, a 3D rendering engine culled from translated C++ along wit PDF.js and other libraries — head on over to Google Code where you’ll find the source for the entire suite. Also be sure to read through the FAQ for more info about the thinking behind Octane.