diff options
Diffstat (limited to 'tests/README.md')
| -rw-r--r-- | tests/README.md | 54 |
1 files changed, 38 insertions, 16 deletions
diff --git a/tests/README.md b/tests/README.md index 68b72f2d..6cc79217 100644 --- a/tests/README.md +++ b/tests/README.md @@ -3,13 +3,10 @@ ## Directory structure Top level directory structure: - `src`: Testing code. -- `typ`: Input files. The tests in `compiler` specifically test the compiler - while the others test the standard library (but also the compiler - indirectly). +- `suite`: Input files. Mostly organize in parallel to the code. - `ref`: Reference images which the output is compared with to determine whether a test passed or failed. -- `png`: PNG files produced by tests. -- `pdf`: PDF files produced by tests. +- `store`: Store for PNG, PDF, and SVG output files produced by the tests. ## Running the tests Running all tests (including unit tests): @@ -37,11 +34,6 @@ Running a test with the exact filename `page.typ`. testit --exact page.typ ``` -Debug-printing the layout trees for all executed tests. -```bash -testit --debug empty.typ -``` - To make the integration tests go faster they don't generate PDFs by default. Pass the `--pdf` flag to generate those. Mind that PDFs are not tested automatically at the moment, so you should always check the output manually when @@ -50,18 +42,48 @@ making changes. testit --pdf ``` -## Update expected images +## Writing tests +The syntax for an individual test is `--- {name} ---` followed by some Typst +code that should be tested. The name must be globally unique in the test suite, +so that tests can be easily migrated across files. + +There are, broadly speaking, three kinds of tests: + +- Tests that just ensure that the code runs successfully: Those typically make + use of `test` or `assert.eq` (both are very similar, `test` is just shorter) + to ensure certain properties hold when executing the Typst code. + +- Tests that ensure the code fails with a particular error: Those have inline + annotations like `// Error: 2-7 thing was wrong`. An annotation can be + either an "Error", a "Warning", or a "Hint". The range designates where + in the next non-comment line the error is and after it follows the message. + If you the error is in a line further below, you can also write ranges like + `3:2-3:7` to indicate the 2-7 column in the 3rd non-comment line. + +- Tests that ensure certain visual output is produced: Those render the result + of the test with the `typst-render` crate and compare against a reference + image stored in the repository. The test runner automatically detects whether + a test has visual output and requires a reference image in this case. + + To prevent bloat, it is important that the test images are kept as small as + possible. To that effect, the test runner enforces a maximum size of 20 KiB. + If truly necessary, this limit can however be lifted by adding `// LARGE` as + the first line of a test. + +If you have the choice between writing a test using assertions or using +reference images, prefer assertions. This makes the test easier to understand +in isolation and prevents bloat due to images. + +## Updating reference images If you created a new test or fixed a bug in an existing test, you need to update -the reference image used for comparison. For this, you can use the -`UPDATE_EXPECT` environment variable or the `--update` flag: +the reference image used for comparison. For this, you can use the `--update` +flag: ```bash testit mytest --update ``` If you use the VS Code test helper extension (see the `tools` folder), you can -alternatively use the checkmark button to update the reference image. In that -case you should also install `oxipng` on your system so that the test helper -can optimize the reference images. +alternatively use the save button to update the reference image. ## Making an alias If you want to have a quicker way to run the tests, consider adding a shortcut |