Testing with JsUnit

JsUnit Resources

JsUnit belongs to the xUnit family of testing tools, the most popular of which is JUnit for Java. Unit tests are a means of ensuring that small "units" of code function they way that they should by performing automated tests on their interfaces.

JsUnit tests run inside test pages, which are normal web pages with some extra JavaScript code. The test pages are loaded into a "Test Runner", which controls the tests and keeps track of the results. The test runner calls test functions, which do the real work. The test functions call "assertion" functions of all sorts, such as assertTrue() and assertEquals(), which check to make sure that the target code is doing what it should be doing. The test runner displays a green bar, which gets longer as tests are completed. If even one test fails then the bar turns red, and a message about the failed test is displayed.

Testing manually is slow and mind-numbing work. Something will always be missed, and developers will avoid doing it. Automated testing is fast and accurate. When all the tests are "green" you feel good about it.

If we have a comprehensive test suite, and all the tests succeed, then we can be fairly confident that the code is working the way it should. The more good tests, the more confident we feel. More importantly, if we make a change to the code and the test suite is still green, then we can be fairly certain that we haven't introduced a bug. We run the test suite before committing new code to CVS, and if it isn't green we stop and find the problem. This way the code you get from CVS will always be "green", if the test suite fails you can be pretty sure that the changes you made broke something.

Important

If the latest code in CVS fails the HTML test suite in a supported browser, let us know. (XHTML and XML tests are less important at the moment since we don't have all the required features in place.)

Test-driven development is the idea that you write the tests before you write the code, and then you write the code to satisfy the test. It's not a bad way of doing things. Programming methodologies like agile software development and extreme programming strongly encourage the use of tests. It's hard to argue that tests aren't a good idea. The only reason to skip writing tests is to save time in the short term, but it'll cost you more time later if you miss an important bug for lack of a test. Unit tests also help use port Mozile to new browsers by isolating the problematic differences between them. We aren't going to force you to adopt any particular programming methodology, but strongly encourage you to write tests for any code you change or add. We're much more likely to accept changes into Mozile if they come with good tests. And we won't accept any changes that cause the test suite to fail.

Mozile is currently using a slightly modified version of JsUnit 2.2 alpha. The JsUnit files are stored in www/0.8/jsunit/, while Mozile's test files are stored in www/0.8/tests/. The tests/index.xml file provides an index of all the tests, with buttons which trigger test runs, individually and in groups, inside HTML, XHTML, and XML test pages. The tests/shared directory contains the test pages, which are shared by all the tests. The other subdirectories of the tests directory contain the JavaScript test files, organized by module. If a module doesn't have it's own subdirectory, the tests will be stored in tests/core/.

The steps we take to get from index.xml to a test are somewhat complicated. Say we want to run the mozile.dom tests, which are stored in the tests/dom/dom.js file. We point the browser to index.xml, which uses index.xsl and index.js to give us a nice dynamic web page. Then find the table row marked "mozile.dom.* :: DOM" and press the corresponding "XHTML" button. A new window opens with the URL file://../jsunit/testRunner.html?autoRun=true& testpage=../tests/shared/testpage.html&test=[unit:dom/dom] which is the Test Runner page with a bunch of additional arguments. The arguments say that the tests/shared/testpage.html test page will be used, and the [unit:dom/dom] test will be run. The test runner page loads, and it loads the tests/shared/testpage.html test page inside a frame. The tests/dom/dom.js test file is then loaded into the tests/shared/testpage.html test page. Then the test runner does the work of running the tests.

It's also possible to run test suites (groups of tests), and tests for HTML, XHTML, and XML documents. Mozile's test system is more complicated than the minimum necessary for JsUnit testing, but it provides a lot of flexibility. All of the Mozile tests can be run, for all of the document types, with a single button press.

Although the process used to load the tests is fairly complicated, writing the tests is straight-forward. Here is an example from the tests/dom/dom.js test file.

Example 3.5. Test example from tests/dom/dom.js

mozile.require("mozile.dom");
mozile.require("mozile.xml");

var name = "mozile.dom.*";

/**
 * Expose the functions named in this array to JsUnit.
 * Make sure to keep it up to date!
 */
function exposeTestFunctionNames() {
  return ["testIsHTML", "testGetters", "testNamespace", "testCreate", 
    "testRemove", "testIgnore", "testIsWhitespace"];
}

/**
 *
 */
function testIsHTML() {
  // HTML Case
  if(document.documentElement.nodeName.toLowerCase() == "html") {
    assertTrue("This is HTML", mozile.dom.isHTML(document));
  }
  // XML Case
  else {
    assertFalse("This is not HTML", mozile.dom.isHTML(document));
  }
}
...

The mozile.require("mozile.dom"); lines are just the same as in any module. The var name = "mozile.dom.*"; line helps identify the test file in failure messages. The exposeTestFunctionNames() function returns an array with the names of all the test functions in this file.

Important

Make sure the exposeTestFunctionNames() function is up-to-date, or your tests might not run.

The testIsHTML() test function is simple. It includes a conditional, which determines which test should be run, and then assertTrue() and assertFalse() statements which contain a brief message and tell the test runner what the result of the mozile.dom.isHTML(document) call should be. If the result does not match what is asserted, the test will fail and the test status bar will turn red.

Most test functions are more complicated that this one, but they all come down to a series of assertion functions. See the JsUnit documentation for a full list of assertion functions.

There are a few things to keep in mind when running the tests. File paths can be tricky, because the test page is inside a frame of the test runner page, and those files are in different locations. Also, the test functions may be executed in a different page than the one they were loaded into; the file paths can be different inside test functions than they are in the global namespace of the test file.

The HTML, XHTML, and XML test pages in tests/shared are structurally very similar, but there are differences in MIME types and namespaces which may cause browsers to fail on one and not the others. This is why we test all three kinds.

You can make use of the util.js files in different places inside the tests test directory to store code which is shared between test files.

Important

Please do your best to update and improve the test suite. Make sure the test suite runs "green" before sending us the changes to your code. We can't add your changes to Mozile until the test suite runs.