Browsers test can turn incredibly complex shockingly simple, depending on what you are looking for and why. At CNET, we prefer a holistic approach to benchmarking browser, looking at a combination of reference tests general behavior of the browser, as well as several "real world" tests that examine the browser's performance in common scenarios.
Note on testing mobile: We are finalizing our standards for testing the mobile browser and will update this post as soon as they are ready. For now, the following procedures apply only to desktop browsers.
Is your favorite browser on our test list?
Unless your favorite browser is a remixed version of Netscape obscure, chances are we to test it. However, testing browser is further complicated it would otherwise be by the fact that two of the five major browsers, Firefox and Chrome update on a release cycle of six weeks. Sometimes these updates bring dramatic changes, but often they do not. Unfortunately due to the limitations of your human editors humble, CNET will not test all browsers simultaneously.
Instead, we will perform quarterly tests for the most popular browsers used and best known, and half for testing a larger number of competitors. In addition, tests are graded according to the platform, so browsers are not tested Windows simultaneously with Mac, Android, iOS or browsers.
Desktop browsers tested both Windows and Mac, unless otherwise indicated:
Chromium
Firefox
Internet Explorer 9 (Windows 7 only)
Internet Explorer 10 (Windows 8 only)
Opera
Safari (Mac only)
Desktop browsers tested every two years will include:
Maxthon
Before (Windows only)
How we test desktop browsers
We manage each of the three times, and then restart the computer before each test, so that the browser is being "cold." We also wait 30 seconds after starting the browser to ensure that all background processes have been completed. Then, we calculate the average of three trials. Performance criteria
The test Acid3 Web Standards Project's browser verifies compliance with recognized standards. Slightly outdated because it does not look HTML5, which has been finalized, it is a good way to establish a baseline. Browsers that do not hit 100 out of 100 on the Acid3 are late to a fundamental right, so crucial.
Octane Google, the successor of reference test Google V8, focuses on JavaScript performance testing areas such as code optimization, encryption and decryption, emulation and manipulation of tables, and assign each sub-test a number. Over the final score is, the better.
Mozilla Kraken is also a test of JavaScript performance, which focuses specifically on rendering times for audio, imaging, AI, JSON, and encryption. A smaller number is preferable for the final score.
The HTML5 test awards points for each feature that HTML5 browser supports, a total of 500. This is a crude way to assess how the future before the browser.
JSGameBench, GUImark3 (game test text test) and look FISHIE Microsoft reservoir performance HTML5 canvas in various environments such as games. Canvas is an important part of the HTML5 test because it creates all the nifty 2D images and shapes that can move across your screen. Each of these three tests using a different standard. FISHIE Tank, for example, allows the tester to determine the number of fish on the screen. The test will then show you how many frames per second, it can make them in.
Microsoft Chalkboard performs a series of timed tests on HTML5 panning, zooming and scaling. Fast, the better.
Sort setImmediate Microsoft looks at the intersection of consumption HTML5 and power. Currently only supports IE 10 setImmediate API, but HTML5 test subtest gives an idea of how your browser is energy.
Facebook checks Ringmark support HTML5 features, adapted to the needs of the mobile browser. However, it works well on desktop computers and offers a good and rare comparison between the desktop and laptop.
Real world tests
We also carry four "real world" testing to see how the browser behaves under conditions of use. These tests look at specific behaviors of browser you are likely to encounter: start cold start, the memory used when it is open, stop time, and waking up from sleep.
As the reference test, each test is performed three times and then averaged. Contrary to these tests, which are performed only with the tab open test run, our real world tests are performed twice with five tabs open, and with 50 tabs. It is to reproduce the real scenario to keep an eye open simultaneously many, something that many people (even if you do not.)
