Lint Tool

We have a lint tool for catching common mistakes in test files. You can run it manually by running the wpt lint command from the root of your local web-platform-tests working directory like this:

./wpt lint

The lint tool is also run automatically for every submitted pull request, and reviewers will not merge branches with tests that have lint errors, so you must either fix all lint errors, or you must whitelist test files to suppress the errors.

Fixing lint errors

You must fix any errors the lint tool reports, unless an error is for something essential to a certain test or that for some other exceptional reason shouldn’t prevent the test from being merged; in those cases you can whitelist test files to suppress the errors. In all other cases, follow the instructions below to fix all errors reported.

ABSOLUTE-URL-REF
Reference test with a reference file specified via an absolute URL: ‘%s’
AHEM COPY
Don’t add extra copies of Ahem, use /fonts/Ahem.ttf
AHEM SYSTEM FONT
Don’t use Ahem as a system font, use /fonts/ahem.css
ASSERT_THROWS

Test-file line has an assert_throws(...) call

To fix:

Replace with assert_throws_dom or assert_throws_js or assert_throws_exactly

BROKEN-GLOBAL-METADATA
Invalid global metadata: %s
BROKEN-METADATA
Metadata comment is not formatted correctly
CONSOLE

Test-file line has a console.*(...) call

To fix:

remove the console.*(...) call (and in some cases, consider adding an assert_* of some kind in place of it)

CONTENT-MANUAL
Manual test whose filename doesn’t end in ‘-manual’
CONTENT-VISUAL
Visual test whose filename doesn’t end in ‘-visual’
CR AT EOL

Test-file line ends with CR (U+000D) character

To fix:

reformat file so each line just has LF (U+000A) line ending (standard, cross-platform “Unix” line endings instead of, e.g., DOS line endings).

CSS-COLLIDING-REF-NAME
The filename %s is shared by: %s
CSS-COLLIDING-SUPPORT-NAME
The filename %s is shared by: %s
CSS-COLLIDING-TEST-NAME
The filename %s in the %s testsuite is shared by: %s
EARLY-TESTHARNESSREPORT

Test file has an instance of <script src='/resources/testharnessreport.js'> prior to <script src='/resources/testharness.js'>

To fix:

flip the order

FILE TYPE
/%s is an unsupported file type (%s)
GENERATE_TESTS

Test file line has a generate_tests call

To fix:

remove the call and call test() a number of times instead

GITIGNORE
.gitignore found outside the root
IGNORED PATH
%s matches an ignore filter in .gitignore - please add a .gitignore exception
INDENT TABS

Test-file line starts with one or more tab characters

To fix:

use spaces to replace any tab characters at beginning of lines

INDENTED-METADATA
Metadata comments should start the line
INVALID-TIMEOUT

Test file with <meta name='timeout'...> element that has a content attribute whose value is not long: %s

To fix:

replace the value of the content attribute with long

LATE-TIMEOUT

Test file with <meta name='timeout'...> element after <script src='/resources/testharnessreport.js'> element

To fix:

move the <meta name="timeout"...> element to precede the script element.

LAYOUTTESTS APIS
eventSender/testRunner/internals used; these are LayoutTests-specific APIs (WebKit/Blink)
MALFORMED-VARIANT
%s <meta name=variant> ‘content’ attribute must be the empty string or start with ‘?’ or ‘#’
MISSING DEPENDENCY

Chromium-specific content referenced

To fix:

Reimplement the test to use well-documented testing interfaces

MISSING-LINK

Testcase file must have a link to a spec

To fix:

Ensure that there is a <link rel="help" href="[url]"> for the spec. MISSING-LINK is designed to ensure that the CSS build tool can find the tests. Note that the CSS build system is primarily used by test.csswg.org/, which doesn’t use wptserve, so *.any.js and similar tests won’t work there; stick with the .html equivalent.

MISSING-TESTDRIVER-VENDOR
Missing <script src='/resources/testdriver-vendor.js'>
MISSING-TESTHARNESSREPORT

Missing <script src='/resources/testharnessreport.js'>

To fix:

ensure each test file contains <script src='/resources/testharnessreport.js'>

MULTIPLE-TESTDRIVER
More than one <script src='/resources/testdriver.js'>
MULTIPLE-TESTDRIVER-VENDOR
More than one <script src='/resources/testdriver-vendor.js'>
MULTIPLE-TESTHARNESS

More than one <script src='/resources/testharness.js'>

To fix:

ensure each test has only one <script src='/resources/testharnessreport.js'> instance

MULTIPLE-TESTHARNESSREPORT
More than one <script src='/resources/testharnessreport.js'>
MULTIPLE-TIMEOUT

More than one meta name=’timeout’

To fix:

ensure each test file has only one instance of a <meta name="timeout"...> element

NON-EXISTENT-REF
Reference test with a non-existent ‘%s’ relationship reference: ‘%s’
OPEN-NO-MODE
File opened without providing an explicit mode (note: binary files must be read with ‘b’ in the mode flags)
PARSE-FAILED

Unable to parse file

To fix:

examine the file to find the causes of any parse errors, and fix them.

PATH LENGTH

/%s longer than maximum path length (%d > 150)

To fix:

use shorter filename to rename the test file

PRINT STATEMENT

A server-side python support file contains a print statement

To fix:

remove the print statement or replace it with something else that achieves the intended effect (e.g., a logging call)

PROMISE_REJECTS

Test-file line has a promise_rejects(...) call

To fix:

Replace with promise_rejects_dom or promise_rejects_js or promise_rejects_exactly

SAME-FILE-REF
Reference test which points at itself as a reference
SET TIMEOUT

setTimeout used

To fix:

replace all setTimeout(...) calls with step_timeout(...) calls

SPECIALPOWERS API
SpecialPowers used; this is gecko-specific and not supported in wpt
STRAY-METADATA
Metadata comments should start the file
SUPPORT-WRONG-DIR
Support file not in support directory
TESTDRIVER-PATH
testdriver.js script seen with incorrect path
TESTDRIVER-VENDOR-PATH
testdriver-vendor.js script seen with incorrect path
TESTHARNESS-IN-OTHER-TYPE
testharness.js included in a %s test
TESTHARNESS-PATH
testharness.js script seen with incorrect path
TESTHARNESSREPORT-PATH
testharnessreport.js script seen with incorrect path
TRAILING WHITESPACE

Whitespace at EOL

To fix:

Remove trailing whitespace from all lines in the file.

UNKNOWN-GLOBAL-METADATA
Unexpected value for global metadata
UNKNOWN-METADATA
Unexpected kind of metadata
UNKNOWN-TIMEOUT-METADATA
Unexpected value for timeout metadata
VARIANT-MISSING

Test file with a <meta name='variant'...> element that’s missing a content attribute

To fix:

add a content attribute with an appropriate value to the <meta name='variant'...> element

W3C-TEST.ORG

Test-file line has the string w3c-test.org

To fix:

either replace the w3c-test.org string with the expression {{host}}:{{ports[http][0]}} or a generic hostname like example.org

WEB-PLATFORM.TEST
Internal web-platform.test domain used
WEBIDL2.JS
Legacy webidl2.js script used
WORKER COLLISION
path ends with %s which collides with generated tests from %s files

Updating the whitelist

Normally you must fix all lint errors. But in the unusual case of error reports for things essential to certain tests or that for other exceptional reasons shouldn’t prevent a merge of a test, you can update and commit the lint.whitelist file in the web-platform-tests root directory to suppress errors the lint tool would report for a test file.

To add a test file or directory to the whitelist, use the following format:

ERROR TYPE:file/name/pattern

For example, to whitelist the file example/file.html such that all TRAILING WHITESPACE errors the lint tool would report for it are suppressed, add the following line to the lint.whitelist file:

TRAILING WHITESPACE:example/file.html

To whitelist an entire directory rather than just one file, use the * wildcard. For example, to whitelist the example directory such that all TRAILING WHITESPACE errors the lint tool would report for any files in it are suppressed, add the following line to the lint.whitelist file:

TRAILING WHITESPACE:example/*

Similarly, you can also use shell-style wildcards to express other filename patterns or directory-name patterns.

Finally, to whitelist just one line in a file, use the following format:

ERROR TYPE:file/name/pattern:line_number

For example, to whitelist just line 128 of the file example/file.html such that any TRAILING WHITESPACE error the lint tool would report for that line is suppressed, add the following to the lint.whitelist file:

TRAILING WHITESPACE:example/file.html:128