IDL Tests (idlharness.js)

Introduction

idlharness.js generates tests for Web IDL fragments, using the JavaScript Tests (testharness.js) infrastructure. You typically want to use .any.js or .window.js for this to avoid having to write unnessary boilerplate.

Adding IDL fragments

Web IDL is automatically scraped from specifications and added to the /interfaces/ directory. See the README there for details.

Testing IDL fragments

For example, the Fetch API’s IDL is tested in /fetch/api/idlharness.any.js:

// META: global=window,worker
// META: script=/resources/WebIDLParser.js
// META: script=/resources/idlharness.js
// META: timeout=long

idl_test(
  ['fetch'],
  ['referrer-policy', 'html', 'dom'],
  idl_array => {
    idl_array.add_objects({
      Headers: ["new Headers()"],
      Request: ["new Request('about:blank')"],
      Response: ["new Response()"],
    });
    if (self.GLOBAL.isWindow()) {
      idl_array.add_objects({ Window: ['window'] });
    } else if (self.GLOBAL.isWorker()) {
      idl_array.add_objects({ WorkerGlobalScope: ['self'] });
    }
  }
);

Note how it includes /resources/WebIDLParser.js and /resources/idlharness.js in addition to testharness.js and testharnessreport.js (automatically included due to usage of .any.js). These are needed to make the idl_test function work.

The idl_test function takes three arguments:

• srcs: a list of specifications whose IDL you want to test. The names here need to match the filenames (excluding the extension) in /interfaces/.

• deps: a list of specifications the IDL listed in srcs depends upon. Be careful to list them in the order that the dependencies are revealed.

• setup_func: a function or async function that takes care of creating the various objects that you want to test.

Methods of IdlArray

IdlArray objects can be obtained through the setup_func argument of idl_test. Anything not documented in this section should be considered an implementation detail, and outside callers should not use it.

add_objects(dict)

dict should be an object whose keys are the names of interfaces or exceptions, and whose values are arrays of strings. When an interface or exception is tested, every string registered for it with add_objects() will be evaluated, and tests will be run on the result to verify that it correctly implements that interface or exception. This is the only way to test anything about [LegacyNoInterfaceObject] interfaces, and there are many tests that can’t be run on any interface without an object to fiddle with.

The interface has to be the primary interface of all the objects provided. For example, don’t pass {Node: ["document"]}, but rather {Document: ["document"]}. Assuming the Document interface was declared to inherit from Node, this will automatically test that document implements the Node interface too.

Warning: methods will be called on any provided objects, in a manner that WebIDL requires be safe. For instance, if a method has mandatory arguments, the test suite will try calling it with too few arguments to see if it throws an exception. If an implementation incorrectly runs the function instead of throwing, this might have side effects, possibly even preventing the test suite from running correctly.

prevent_multiple_testing(name)

This is a niche method for use in case you’re testing many objects that implement the same interfaces, and don’t want to retest the same interfaces every single time. For instance, HTML defines many interfaces that all inherit from HTMLElement, so the HTML test suite has something like

.add_objects({
  HTMLHtmlElement: ['document.documentElement'],
  HTMLHeadElement: ['document.head'],
  HTMLBodyElement: ['document.body'],
  ...
})

and so on for dozens of element types. This would mean that it would retest that each and every one of those elements implements HTMLElement, Element, and Node, which would be thousands of basically redundant tests. The test suite therefore calls prevent_multiple_testing("HTMLElement"). This means that once one object has been tested to implement HTMLElement and its ancestors, no other object will be. Thus in the example code above, the harness would test that document.documentElement correctly implements HTMLHtmlElement, HTMLElement, Element, and Node; but document.head would only be tested for HTMLHeadElement, and so on for further objects.

set_test_filter(filter)

Sets a predicate controlling which interface and member tests are generated. The predicate receives an interface name and, for member tests, a member name. It must return true to include the tests or false to exclude them.

Interface-level and member-level filtering are independent: an interface’s members can be tested even when its interface-level checks are excluded. Constructors are part of the interface-level checks. IDL validation, dependency definitions, and structural checks for partial interfaces and mixins are unaffected. Already-untested definitions and members are not enabled by a filter.

Call this method in the setup callback passed to idl_test(). Filtering is applied after merging partial interfaces and mixins, so mixin members are filtered using the name of the interface that includes them.

For example, to exclude one operation while keeping the rest of an interface:

idl_array.set_test_filter((name, memberName) =>
  !(name === 'Example' && memberName === 'excludedOperation'));