Conformance

The suite every Tórculo implementation runs. A case is a small JSON file with an input and the exact expected output; an implementation conforms to a kind of case when it passes every non-pending case of that kind. Normative text: spec/SPEC.md. Suite version and case count: manifest.json. Changes: CHANGELOG.md.

Every pliego of the suite carries music and sound design (BANDA, EFECTOS, MEZCLA, MÚSICA, AMBIENTE, ESPACIO and sonido: effects). The sounds are synthesized by tools/sonido.py with the Python standard library and the canon's randomness: the same bytes on every machine, dedicated to the public domain (CC0 1.0). They are not committed; the generator and the runner make them (their SHA-256 are in the manifest).

§Layout

§The oracles

Independent readers written from the specification, Python standard library only:

§Regenerating and running

python3 conformance/tools/generar.py             # everything except the fingerprints
python3 conformance/tools/generar.py --huellas   # also renders the fingerprints (heavy)
python3 conformance/tools/ejecutar.py            # the quick classes (about a minute)
python3 conformance/tools/ejecutar.py --pesado   # everything: fingerprints, soundtrack, Rust láminas, package round trip

Requirements: Python 3.11 or later; Node 22 or later (TypeScript canon and the WebAssembly pen); rustc (Rust canon); ffmpeg (fingerprints); the Python with which manim-saorín runs (MESA_PY, by default the Homebrew Manim one) and its folder (MESA_IMPL); the pen's Python (TORCULO_PLUMA_PY). Heavy work goes through tools/pesado.

§How expectations are made

So that nobody has to trust a single implementation:

• Diagnostics are written by hand in manual/lee.json. The generator refuses to write a case if the reference disagrees, unless the case is marked pending with its reason.
• Vocabularies are extracted from the parser (spec/vocabularios/extrae.py): the specification is that table, and the case checks the parser still says it.
• IRs are the reference's, reviewed and frozen. Other compilers are held to the normative projection; the reference, to the whole IR.
• Closure and mirror are properties: the runner computes both sides.
• Canon vectors are computed in generar.py with integer arithmetic written again, not with the generated canon.
• Láminas: their hashes come from the WebAssembly build of the pen (pluma/pruebas/referencia/_laminas.json); Python and Rust must give the same bytes.
• Takes, packages, shaders: one broken file per rule, with the code the oracle must give; the generator stops if the oracle disagrees.
• Fingerprints and soundtrack are the reference interpreter's, on the reference machine; they are compared with tolerance (§13.4) and the soundtrack against the targets of §8.2 (−14 LUFS ± 1 LU, true peak at most −1 dBTP, the cues and counts).

§Case format

{"id": "lee-errores-afecto", "kind": "lee", "input": {"pliego": "pliegos/errores/afecto.mesa"},
 "expect": {"valido": false, "errores": ["E010"], "avisos": ["W113"]}}

id equals the file name; paths are relative to conformance/. A case MAY carry "pendiente": "reason": a known deviation of the reference implementation (SPEC Appendix D), reported apart.

§Adapters

The runner talks to an implementation through an adapter (tools/adaptadores/). An adapter for another compiler offers the same functions as manim_saorin.py: ir(pliego), lee(pliego) returning codes, escribe, parte, funde, taller, and, for an interpreter, toca_gx(pliego, out) writing the video, the master and the soundtrack report.

§The report

ejecutar.py prints a one-line summary and writes informe.json: the suite version and cases_sha256, the date, the implementations, the totals (pasa, falla, pendiente, omitido), the totals per kind, and one result per case with its detail when it did not pass.