The JavaScript team maintains a library of tests that are based on the ECMA-262 standard for web scripting languages and other documentation for JavaScript.

If you find a bug in the core JavaScript engine, you can now contribute a test to our library. Your tests will be used to reproduce bugs, verify bug fixes, prevent regressions, and improve the stability of the JavaScript engine.

If you are contributing code to the JavaScript engine, you can use the tests to verify that the engine still works correctly before checking in.

What's covered by the JavaScript tests?

The JavaScript tests cover the functionality of the core JavaScript engine. This includes the following JavaScript objects:

Arrays

Booleans

Dates

Functions

Global Object

Math

Numbers

Objects

Regular Expressions

Strings

Additionally, the JavaScript tests cover parsing, lexical conventions, expressions, statements, type conversion, and exception handling. See the current standard, or the current working draft of the revised standard if you have questions about what is part of core JavaScript.

We also have tests for LiveConnect. We will post more information on those tests soon.

What's not covered by the JavaScript tests?

Tests must not refer to any objects that are not part of core JavaScript. This is because the tests are run not only against the JavaScript engine in the browser, but also against the stand-alone JavaScript engines.

Because the tests are not always run in the browser, it is important that you not refer to any objects that are part of the browser or DOM. For example, you must not refer to the following objects, or methods or properties of the following objects:

navigator
window
document
layer
form elements

If you would like to write tests that cover DOM objects, talk to the people at netscape.public.mozilla.qa.general, or see the quality assurance group's page. You should also check out the NGLayout project's page on how you can help.

Which areas should I write tests for?

If you would like to contribute tests to the library, please contact the coordinator.

We would really appreciate it if you wrote tests that demonstrate any bugs you find in the JavaScript engine. Providing a test will make sure that we can reproduce the bug, and will make sure that there won't be regressions.

Additionally, one area in which our test coverage somewhat lags behind implementation is functionality in ECMA-262 Version 2 that is new or changed since Version 1. Version 2 document status shows what has changed or is new since version 1.

It would probably be better if you write tests for areas whose content has already been accepted by the working group. Otherwise, the test will have to be updated if the content changes.

Currently, the new, accepted functionality includes regular expressions and closure objects.

Where are the JavaScript tests?

The tests that are currently available on mozilla.org are checked into mozilla/js/tests. You must check them out explicitly -- they are not automatically checked out when you check out the mozilla source code.

See the instructions on checking out source code for how to set up your CVSROOT and log in. To check out the tests, the command is

cvs -z3 checkout mozilla/js/tests

How do I write a JavaScript test?

Follow the format of the template. Basically, you need to comment the test so that we know what you're testing; set some variables so that the test driver can figure out what you're testing; and then add some test cases.

Each test case has three parts: a string representation of what you are testing, the expected result, and the actual result. You can write functions to generate the test cases, or you can hard-code all the test data. The structure of the test is up to you. Check out some of the existing tests for examples.

How do I run the JavaScript tests?

To run an individual test in the JavaScript shell:

Check out the JavaScript source, and build the JavaScript shell.
Start the JavaScript shell.
% js js>
Load the shared functions file:
js> load("d:/mozilla/js/tests/ecma/shell.js")
Load the test file. The test should execute and display results.
js> load("d:/mozilla/js/tests/ecma/Expressions/11.1.1.js")

There's also a perl script called runtests.pl that will run all the tests for you. To use runtests.pl, you must have Perl 5 installed on your system.

Check out the JavaScript source, and build the JavaScript shell.
Set your MOZ_SRC environment variable to the directory where you've checked out and built the JavaScript shell.
Run runtests.pl.
For command line options, at the command prompt, type
% runtests.pl --h

To run the tests in the browser:

Execute mkhtml.pl, a perl script that generates an HTML file for each test file.
Build apprunner.exe.
View the generated HTML pages in apprunner. The tests should execute and display results.

We do have a real test driver for the JavaScript tests; currently, we are not able to post this on mozilla.org. I hope this will change in the future.

Resources

Christine Begle