JavaScript Tests (testharness.js)

JavaScript tests are the correct type of test to write in any situation where you are not specifically interested in the rendering of a page, and where human interaction isn’t required; these tests are written in JavaScript using a framework called testharness.js.

A high-level overview is provided below and more information can be found here:

See also the general guidelines for all test types.

Window tests

Without HTML boilerplate (.window.js)

Create a JavaScript file whose filename ends in .window.js to have the necessary HTML boilerplate generated for you at .window.html. I.e., for test.window.js the server will ensure test.window.html is available.

In this JavaScript file you can place one or more tests, as follows:

test(() => {
  // Place assertions and logic here
  assert_equals(document.characterSet, "UTF-8");
}, "Ensure HTML boilerplate uses UTF-8"); // This is the title of the test

If you only need to test a single thing, you could also use:

// META: title=Ensure HTML boilerplate uses UTF-8
setup({ single_test: true });
assert_equals(document.characterSet, "UTF-8");
done();

See asynchronous (async_test()) and promise tests (promise_test()) for more involved setups.

With HTML boilerplate

You need to be a bit more explicit and include the testharness.js framework directly as well as an additional file used by implementations:

<!doctype html>
<meta charset=utf-8>
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<body>
  <script>
    test(() => {
      assert_equals(document.characterSet, "UTF-8");
    }, "Ensure UTF-8 declaration is observed");
  </script>

Here too you could avoid the wrapper test() function:

<!doctype html>
<meta charset=utf-8>
<title>Ensure UTF-8 declaration is observed</title>
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<body>
  <script>
    setup({ single_test: true });
    assert_equals(document.characterSet, "UTF-8");
    done();
  </script>

In this case the test title is taken from the title element.

Dedicated worker tests (.worker.js)

Create a JavaScript file that imports testharness.js and whose filename ends in .worker.js to have the necessary HTML boilerplate generated for you at .worker.html.

For example, one could write a test for the FileReaderSync API by creating a FileAPI/FileReaderSync.worker.js as follows:

importScripts("/resources/testharness.js");
test(function () {
    const blob = new Blob(["Hello"]);
    const fr = new FileReaderSync();
    assert_equals(fr.readAsText(blob), "Hello");
}, "FileReaderSync#readAsText.");
done();

This test could then be run from FileAPI/FileReaderSync.worker.html.

(Removing the need for importScripts() and done() is tracked in issue #11529.)

Tests for other or multiple globals (.any.js)

Tests for features that exist in multiple global scopes can be written in a way that they are automatically run in several scopes. In this case, the test is a JavaScript file with extension .any.js, which can use all the usual APIs.

By default, the test runs in a window scope and a dedicated worker scope.

For example, one could write a test for the Blob constructor by creating a FileAPI/Blob-constructor.any.js as follows:

test(function () {
    const blob = new Blob();
    assert_equals(blob.size, 0);
    assert_equals(blob.type, "");
    assert_false(blob.isClosed);
}, "The Blob constructor.");

This test could then be run from FileAPI/Blob-constructor.any.worker.html as well as FileAPI/Blob-constructor.any.html.

It is possible to customize the set of scopes with a metadata comment, such as

// META: global=sharedworker
//       ==> would run in the shared worker scope
// META: global=window,serviceworker
//       ==> would only run in the window and service worker scope
// META: global=dedicatedworker
//       ==> would run in the default dedicated worker scope
// META: global=dedicatedworker-module
//       ==> would run in the dedicated worker scope as a module
// META: global=worker
//       ==> would run in the dedicated, shared, and service worker scopes

For a test file x.any.js, the available scope keywords are:

  • window (default): to be run at x.any.html

  • dedicatedworker (default): to be run at x.any.worker.html

  • dedicatedworker-module to be run at x.any.worker-module.html

  • serviceworker: to be run at x.any.serviceworker.html (.https is implied)

  • serviceworker-module: to be run at x.any.serviceworker-module.html (.https is implied)

  • sharedworker: to be run at x.any.sharedworker.html

  • sharedworker-module: to be run at x.any.sharedworker-module.html

  • jsshell: to be run in a JavaScript shell, without access to the DOM (currently only supported in SpiderMonkey, and skipped in wptrunner)

  • worker: shorthand for the dedicated, shared, and service worker scopes

  • shadowrealm-in-window: runs the test code in a ShadowRealm context hosted in an ordinary Window context; to be run at x.any.shadowrealm-in-window.html

  • shadowrealm-in-shadowrealm: runs the test code in a ShadowRealm context hosted in another ShadowRealm context; to be run at x.any.shadowrealm-in-shadowrealm.html

  • shadowrealm-in-dedicatedworker: runs the test code in a ShadowRealm context hosted in a dedicated worker; to be run at x.any.shadowrealm-in-dedicatedworker.html

  • shadowrealm-in-sharedworker: runs the test code in a ShadowRealm context hosted in a shared worker; to be run at x.any.shadowrealm-in-sharedworker.html

  • shadowrealm-in-serviceworker: runs the test code in a ShadowRealm context hosted in a service worker; to be run at x.https.any.shadowrealm-in-serviceworker.html

  • shadowrealm-in-audioworklet: runs the test code in a ShadowRealm context hosted in an AudioWorklet processor; to be run at x.https.any.shadowrealm-in-audioworklet.html

  • shadowrealm: shorthand for all of the ShadowRealm scopes

To check what scope your test is run from, you can use the following methods that will be made available by the framework:

self.GLOBAL.isWindow()
self.GLOBAL.isWorker()
self.GLOBAL.isShadowRealm()

Although the global done() function must be explicitly invoked for most dedicated worker tests and shared worker tests, it is automatically invoked for tests defined using the “multi-global” pattern.

Extension tests (.extension.js)

Create a JavaScript file whose name ends in .extension.js to have the necessary HTML boilerplate generated for you at .extension.html.

Extension tests leverage the browser.test API rather than interacting with the testharness.js framework directly.

For example, one could write a test for browser.runtime.getURL() by creating a web-extensions/browser.runtime.extension.js file as follows:

runTestsWithWebExtension("/resources/runtime/")
// ==> this method assumes that the extension resources (manifest, scripts, etc.) exist at the path

And by creating a web-extensions/resources/runtime/background.js file as follows:

browser.test.runTests([
  function getURLWithNoParameter() {
    browser.test.assertThrows(() => browser.runtime.getURL())
  }
])

This test could then be run from web-extensions/browser.runtime.extension.html.

Other features of .window.js, .worker.js, .any.js and .extension.js

Specifying a test title

Use // META: title=This is the title of the test at the beginning of the resource.

Including other JavaScript files

Use // META: script=link/to/resource.js at the beginning of the resource. For example,

// META: script=/common/utils.js
// META: script=resources/utils.js

can be used to include both the global and a local utils.js in a test.

In window environments, the script will be included using a classic <script> tag. In classic worker environments, the script will be imported using importScripts(). In module worker environments, the script will be imported using a static import.

wptserve generates markup with /resources/testharness.js and /resources/testharnessreport.js included automatically, so there’s no need to include those scripts from the .js test file.

Specifying a timeout of long

Use // META: timeout=long at the beginning of the resource.

Specifying test variants

Use // META: variant=url-suffix at the beginning of the resource. For example,

// META: variant=?default
// META: variant=?wss

Variants

A test file can have multiple variants by including meta elements, for example:

<meta name="variant" content="?default">
<meta name="variant" content="?wss">

Test runners will execute the test for each variant specified, appending the corresponding content attribute value to the URL of the test as they do so.

/common/subset-tests.js and /common/subset-tests-by-key.js are two utility scripts that work well together with variants, allowing a test to be split up into subtests in cases when there are otherwise too many tests to complete inside the timeout. For example:

<!doctype html>
<title>Testing variants</title>
<meta name="variant" content="?1-1000">
<meta name="variant" content="?1001-2000">
<meta name="variant" content="?2001-last">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="/common/subset-tests.js"></script>
<script>
 const tests = [
                 { fn: t => { ... }, name: "..." },
                 ... lots of tests ...
               ];
 for (const test of tests) {
   subsetTest(async_test, test.fn, test.name);
 }
</script>

With subsetTestByKey, the key is given as the first argument, and the query string can include or exclude a key (which will be matched as a regular expression).

<!doctype html>
<title>Testing variants by key</title>
<meta name="variant" content="?include=Foo">
<meta name="variant" content="?include=Bar">
<meta name="variant" content="?exclude=(Foo|Bar)">
<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<script src="/common/subset-tests-by-key.js"></script>
<script>
   subsetTestByKey("Foo", async_test, () => { ... }, "Testing foo");
   ...
</script>

Table of Contents