diff options
Diffstat (limited to 'tests/README.md')
| -rw-r--r-- | tests/README.md | 68 |
1 files changed, 43 insertions, 25 deletions
diff --git a/tests/README.md b/tests/README.md index 4f5a3169..e625581a 100644 --- a/tests/README.md +++ b/tests/README.md @@ -6,8 +6,8 @@ Top level directory structure: - `suite`: Input files. Mostly organized in parallel to the code. Each file can contain multiple tests, each of which is a section of Typst code following `--- {name} ---`. -- `ref`: Reference images which the output is compared with to determine whether - a test passed or failed. +- `ref`: References which the output is compared with to determine whether a + test passed or failed. - `store`: Store for PNG, PDF, and SVG output files produced by the tests. ## Running the tests @@ -54,18 +54,29 @@ You may find more options in the help message: testit --help ``` -To make the integration tests go faster they don't generate PDFs by default. -Pass the `--pdf` flag to generate those. Mind that PDFs are not tested -automatically at the moment, so you should always check the output manually when -making changes. +To make the integration tests go faster they don't generate PDFs or SVGs by +default. Pass the `--pdf` or `--svg` flag to generate those. Mind that PDFs and +SVGs are **not** tested automatically at the moment, so you should always check +the output manually when making changes. ```bash testit --pdf ``` ## Writing tests -The syntax for an individual test is `--- {name} ---` followed by some Typst -code that should be tested. The name must be globally unique in the test suite, -so that tests can be easily migrated across files. +The syntax for an individual test is `--- {name} {attr}* ---` followed by some +Typst code that should be tested. The name must be globally unique in the test +suite, so that tests can be easily migrated across files. A test name can be +followed by space-separated attributes. For instance, `--- my-test html ---` +adds the `html` modifier to `my-test`, instructing the test runner to also +test HTML output. The following attributes are currently defined: + +- `render`: Tests paged output against a reference image (the default, only + needs to be specified when `html` is also specified to enable both at the + same) +- `html`: Tests HTML output against a reference HTML file. Disables the `render` + default. +- `large`: Permits a reference image size exceeding 20 KiB. Should be used + sparingly. There are, broadly speaking, three kinds of tests: @@ -80,35 +91,42 @@ There are, broadly speaking, three kinds of tests: below. If the code span is in a line further below, you can write ranges like `3:2-3:7` to indicate the 2-7 column in the 3rd non-comment line. -- Tests that ensure certain visual output is produced: Those render the result - of the test with the `typst-render` crate and compare against a reference - image stored in the repository. The test runner automatically detects whether - a test has visual output and requires a reference image in this case. +- Tests that ensure certain output is produced: - To prevent bloat, it is important that the test images are kept as small as - possible. To that effect, the test runner enforces a maximum size of 20 KiB. - If you're updating a test and hit `reference image size exceeds`, see the - section on "Updating reference images" below. If truly necessary, the size - limit can be lifted by adding `// LARGE` as the first line of a test, but this - should be the case very rarely. + - Visual output: By default, the compiler produces paged output, renders it + with the `typst-render` crate, and compares it against a reference image + stored in the repository. The test runner automatically detects whether a + test has visual output and requires a reference image in this case. + + To prevent bloat, it is important that the test images are kept as small as + possible. To that effect, the test runner enforces a maximum size of 20 KiB. + If you're updating a test and hit `reference output size exceeds`, see the + section on "Updating reference images" below. If truly necessary, the size + limit can be lifted by adding a `large` attribute after the test name, but + this should be the case very rarely. + + - HTML output: When a test has the `html` attribute, the compiler produces + HTML output and compares it against a reference file stored in the + repository. By default, this enables testing of paged output, but you can + test both at once by passing both `render` and `html` as attributes. If you have the choice between writing a test using assertions or using reference images, prefer assertions. This makes the test easier to understand in isolation and prevents bloat due to images. ## Updating reference images -If you created a new test or fixed a bug in an existing test, you need to update -the reference image used for comparison. For this, you can use the `--update` -flag: +If you created a new test or fixed a bug in an existing test, you may need to +update the reference output used for comparison. For this, you can use the +`--update` flag: ```bash testit --exact my-test-name --update ``` -This will generally generate compressed reference images (to remain within the -above size limit). +For visual tests, this will generally generate compressed reference images (to +remain within the size limit). If you use the VS Code test helper extension (see the `tools` folder), you can -alternatively use the save button to update the reference image. +alternatively use the save button to update the reference output. ## Making an alias If you want to have a quicker way to run the tests, consider adding a shortcut |