Lint Tool

We have a lint tool for catching common mistakes in test files. You can run it manually by running the wpt lint command from the root of your local web-platform-tests working directory like this:

./wpt lint

The lint tool is also run automatically for every submitted pull request, and reviewers will not merge branches with tests that have lint errors, so you must either fix all lint errors, or you must add an exception to suppress the errors.

Fixing lint errors

You must fix any errors the lint tool reports, unless an error is for something essential to a certain test or that for some other exceptional reason shouldn’t prevent the test from being merged; in those cases you can add an exception to suppress the errors. In all other cases, follow the instructions below to fix all errors reported.

ABSOLUTE-URL-REF

Reference test with a reference file specified via an absolute URL: ‘%s’

AHEM COPY

Don’t add extra copies of Ahem, use /fonts/Ahem.ttf

AHEM SYSTEM FONT

Don’t use Ahem as a system font, use /fonts/ahem.css

ASSERT-PRECONDITION

Test-file line has an assert_precondition(...) call

To fix:

Replace with assert_implements or assert_implements_optional

ASSERT_THROWS

Test-file line has an assert_throws(...) call

To fix:

Replace with assert_throws_dom or assert_throws_js or assert_throws_exactly

BROKEN-GLOBAL-METADATA

Invalid global metadata: %s

BROKEN-METADATA

Metadata comment is not formatted correctly

CONSOLE

Test-file line has a console.*(...) call

To fix:

remove the console.*(...) call (and in some cases, consider adding an assert_* of some kind in place of it)

CONTENT-MANUAL

Manual test whose filename doesn’t end in ‘-manual’

CONTENT-VISUAL

Visual test whose filename doesn’t end in ‘-visual’

CR AT EOL

Test-file line ends with CR (U+000D) character

To fix:

reformat file so each line just has LF (U+000A) line ending (standard, cross-platform “Unix” line endings instead of, e.g., DOS line endings).

DUPLICATE-BASENAME-PATH

File has identical basename path (path excluding extension) as other file(s) (found extensions: %s)

To fix:

rename files so they have unique basename paths

DUPLICATE-CASE-INSENSITIVE-PATH

Path differs from path %s only in case

To fix:

rename files so they are unique irrespective of case

EARLY-TESTDRIVER-VENDOR

Test file has an instance of <script src='/resources/testdriver-vendor.js'> prior to <script src='/resources/testdriver.js'>

To fix:

flip the order

EARLY-TESTHARNESSREPORT

Test file has an instance of <script src='/resources/testharnessreport.js'> prior to <script src='/resources/testharness.js'>

To fix:

flip the order

FILE TYPE

/%s is an unsupported file type (%s)

GENERATE_TESTS

Test file line has a generate_tests call

To fix:

remove the call and call test() a number of times instead

GITIGNORE

.gitignore found outside the root

HTML INVALID SYNTAX

Test-file line has a non-void HTML tag with /> syntax

To fix:

Replace with start tag and end tag

IGNORED PATH

%s matches an ignore filter in .gitignore - please add a .gitignore exception

INDENT TABS

Test-file line starts with one or more tab characters

To fix:

use spaces to replace any tab characters at beginning of lines

INDENTED-METADATA

Metadata comments should start the line

INVALID-META-FILE

The META.yml is not a YAML file with the expected structure

INVALID-TIMEOUT

Test file with <meta name='timeout'...> element that has a content attribute whose value is not long: %s

To fix:

replace the value of the content attribute with long

INVALID-WEB-FEATURES-FILE

The WEB_FEATURES.yml file contains an invalid structure

LATE-TIMEOUT

Test file with <meta name='timeout'...> element after <script src='/resources/testharnessreport.js'> element

To fix:

move the <meta name="timeout"...> element to precede the script element.

LAYOUTTESTS APIS

eventSender/testRunner/internals used; these are LayoutTests-specific APIs (WebKit/Blink)

MALFORMED-VARIANT

%s must be a non empty string and start with ‘?’ or ‘#’

MISSING DEPENDENCY

Chromium-specific content referenced

To fix:

Reimplement the test to use well-documented testing interfaces

MISSING-LINK

Testcase file must have a link to a spec

To fix:

Ensure that there is a <link rel="help" href="[url]"> for the spec. MISSING-LINK is designed to ensure that the CSS build tool can find the tests. Note that the CSS build system is primarily used by test.csswg.org/, which doesn’t use wptserve, so *.any.js and similar tests won’t work there; stick with the .html equivalent.

MISSING-REFTESTWAIT

Missing class=reftest-wait

To fix:

ensure tests that include reftest-wait.js also use class=reftest-wait on the root element.

MISSING-TESTDRIVER-VENDOR

Missing <script src='/resources/testdriver-vendor.js'>

MISSING-TESTHARNESSREPORT

Missing <script src='/resources/testharnessreport.js'>

To fix:

ensure each test file contains <script src='/resources/testharnessreport.js'>

MISSING-WEB-FEATURES-FILE

The WEB_FEATURES.yml file references a test that does not exist: ‘%s’

MOJOM-JS

Don’t check *.mojom.js files into WPT

To fix:

Check if the file is already included in mojojs.zip: https://source.chromium.org/chromium/chromium/src/+/master:chrome/tools/build/linux/FILES.cfg If yes, use loadMojoResources from resources/test-only-api.js to load it; if not, contact ecosystem-infra@chromium.org for adding new files to mojojs.zip.

MULTIPLE-TESTDRIVER

More than one <script src='/resources/testdriver.js'>

MULTIPLE-TESTDRIVER-VENDOR

More than one <script src='/resources/testdriver-vendor.js'>

MULTIPLE-TESTHARNESS

More than one <script src='/resources/testharness.js'>

To fix:

Ensure each test has only one <script src='/resources/testharness.js'> instance. For .js tests, remove // META: script=/resources/testharness.js, which wptserve already adds to the boilerplate markup.

MULTIPLE-TESTHARNESSREPORT

More than one <script src='/resources/testharnessreport.js'>

To fix:

Ensure each test has only one <script src='/resources/testharnessreport.js'> instance. For .js tests, remove // META: script=/resources/testharnessreport.js, which wptserve already adds to the boilerplate markup.

MULTIPLE-TIMEOUT

More than one meta name=’timeout’

To fix:

ensure each test file has only one instance of a <meta name="timeout"...> element

NON-EXISTENT-REF

Reference test with a non-existent ‘%s’ relationship reference: ‘%s’

OPEN-NO-MODE

File opened without providing an explicit mode (note: binary files must be read with ‘b’ in the mode flags)

PARSE-FAILED

Unable to parse file

To fix:

examine the file to find the causes of any parse errors, and fix them.

PATH LENGTH

/%s longer than maximum path length (%d > 150)

To fix:

use shorter filename to rename the test file

PRINT STATEMENT

A server-side python support file contains a print statement

To fix:

remove the print statement or replace it with something else that achieves the intended effect (e.g., a logging call)

PROMISE_REJECTS

Test-file line has a promise_rejects(...) call

To fix:

Replace with promise_rejects_dom or promise_rejects_js or promise_rejects_exactly

REFERENCE-IN-OTHER-TYPE

Reference link included in a %s test

SAME-FILE-REF

Reference test which points at itself as a reference

SET TIMEOUT

setTimeout used

To fix:

replace all setTimeout(...) calls with step_timeout(...) calls

SPECIALPOWERS API

SpecialPowers used; this is gecko-specific and not supported in wpt

STRAY-METADATA

Metadata comments should start the file

TENTATIVE-DIRECTORY-NAME

Directories for tentative tests must be named exactly ‘tentative’

To fix:

rename directory to be called ‘tentative’

TEST DRIVER INTERNAL

Test-file uses test_driver_internal API

To fix:

Only use test_driver public API

TESTDRIVER-IN-UNSUPPORTED-TYPE

testdriver.js included in a %s test, which doesn’t support testdriver.js

TESTDRIVER-PATH

testdriver.js script seen with incorrect path

TESTDRIVER-UNSUPPORTED-QUERY-PARAMETER

testdriver.js script seen with unsupported query parameters

TESTDRIVER-VENDOR-PATH

testdriver-vendor.js script seen with incorrect path

TESTHARNESS-IN-OTHER-TYPE

testharness.js included in a %s test

TESTHARNESS-PATH

testharness.js script seen with incorrect path

TESTHARNESSREPORT-PATH

testharnessreport.js script seen with incorrect path

TRAILING WHITESPACE

Whitespace at EOL

To fix:

Remove trailing whitespace from all lines in the file.

UNKNOWN-GLOBAL-METADATA

Unexpected value for global metadata

UNKNOWN-METADATA

Unexpected kind of metadata

UNKNOWN-TIMEOUT-METADATA

Unexpected value for timeout metadata

VARIANT-MISSING

Test file with a <meta name='variant'...> element that’s missing a content attribute

To fix:

add a content attribute with an appropriate value to the <meta name='variant'...> element

W3C-TEST.ORG

Test-file line has the string w3c-test.org

To fix:

either replace the w3c-test.org string with the expression {{host}}:{{ports[http][0]}} or a generic hostname like example.org

WEB-PLATFORM.TEST

Internal web-platform.test domain used

To fix:

use server-side substitution, along with the .sub filename-flag, to replace web-platform.test with {{domains[]}}

WEBIDL2.JS

Legacy webidl2.js script used

WORKER COLLISION

path ends with %s which collides with generated tests from %s files

Updating the ignored files

Normally you must fix all lint errors. But in the unusual case of error reports for things essential to certain tests or that for other exceptional reasons shouldn’t prevent a merge of a test, you can update and commit the lint.ignore file in the web-platform-tests root directory to suppress errors the lint tool would report for a test file.

To add a test file or directory to the list, use the following format:

ERROR TYPE:file/name/pattern

For example, to ignore all TRAILING WHITESPACE errors in the file example/file.html, add the following line to the lint.ignore file:

TRAILING WHITESPACE:example/file.html

To ignore errors for an entire directory rather than just one file, use the * wildcard. For example, to ignore all TRAILING WHITESPACE errors in the example directory, add the following line to the lint.ignore file:

TRAILING WHITESPACE:example/*

Similarly, you can also use shell-style wildcards to express other filename patterns or directory-name patterns.

Finally, to ignore just one line in a file, use the following format:

ERROR TYPE:file/name/pattern:line_number

For example, to ignore the TRAILING WHITESPACE error for just line 128 of the file example/file.html, add the following to the lint.ignore file:

TRAILING WHITESPACE:example/file.html:128