# Conformance

> The Tórculo conformance suite: the cases an implementation must pass to call itself conforming, and the latest report.

URL: https://torculo.joseluissaorin.com/en/conformance · Reviewed: 2026-10-15 · Español: https://torculo.joseluissaorin.com/conformidad.md

The suite every Tórculo implementation runs. A case is a small JSON file with an input and
the exact expected output; an implementation conforms to a kind of case when it passes
every non-pending case of that kind. Normative text: [`spec/SPEC.md`](../spec/SPEC.md).
Suite version and case count: [`manifest.json`](manifest.json). Changes:
[`CHANGELOG.md`](CHANGELOG.md).

**Every pliego of the suite carries music and sound design** (`BANDA`, `EFECTOS`, `MEZCLA`,
`MÚSICA`, `AMBIENTE`, `ESPACIO` and `sonido:` effects). The sounds are synthesized by
[`tools/sonido.py`](tools/sonido.py) with the Python standard library and the canon's
randomness: the same bytes on every machine, dedicated to the public domain (CC0 1.0). They
are not committed; the generator and the runner make them (their SHA-256 are in the
manifest).

## Layout

| Path | What | Made by |
|---|---|---|
| `pliegos/*.mesa` | the pliegos: one per area of the language and per profile | hand |
| `pliegos/errores/*.mesa` | one broken rule per pliego, named after what it breaks | hand |
| `pliegos/sonido/` | music cues (`banda/`) and effects and ambiences (`efectos/`) | `tools/sonido.py` |
| `manual/lee.json` | the expected diagnostics of every pliego, **written by hand** | hand |
| `expected/ir/*.ir.json` | the frozen IR of the reference implementation | `tools/generar.py` |
| `expected/huella/*.json` | the fingerprints (§13.4) and soundtrack reports of the reference interpreter | `tools/generar.py --huellas` |
| `canon/vectores.json` | the canon's test vectors, computed with independent integer arithmetic | `tools/generar.py` |
| `laminas/*.dibujo.json` | the drawings whose láminas every producer must reproduce byte for byte | copied from `pluma/pruebas/referencia` |
| `tomas/*.take`, `tomas/fuentes/*.py` | takes recorded by the engine fork, and the scenes that made them | the engine |
| `tomas/invalidas/*.take` | one broken rule per take | `tools/generar.py` |
| `mesapack/*.mesapack` | a minimal valid package and one broken rule per package | `tools/generar.py` |
| `materias/*.wgsl` | shaders that break the pass convention | `tools/generar.py` |
| `cases/*.json` | one case per file | `tools/generar.py` |
| `informe.json` | the last report of the runner against the reference implementations | `tools/ejecutar.py` |

### The oracles

Independent readers written from the specification, Python standard library only:

| Tool | Checks |
|---|---|
| `tools/jcs.py` | RFC 8785 canonical JSON (self-test: `python3 tools/jcs.py`) |
| `tools/ir.py` | the normative projection of an IR (§10.6) |
| `tools/toma.py` | takes (§19); codes T001 to T007 |
| `tools/lamina.py` | láminas (§17); codes L001 to L011 |
| `tools/mesapack.py` | packages (§14); codes P001 to P009 |
| `tools/materias.py` | the pass convention (§18); codes M001 to M006 |
| `tools/huella.py` | fingerprints of a film (§13.4) and their tolerant comparison |
| `tools/adaptadores/manim_saorin.py` | the adapter to the reference implementation of the language |

## Regenerating and running

```sh
python3 conformance/tools/generar.py             # everything except the fingerprints
python3 conformance/tools/generar.py --huellas   # also renders the fingerprints (heavy)
python3 conformance/tools/ejecutar.py            # the quick classes (about a minute)
python3 conformance/tools/ejecutar.py --pesado   # everything: fingerprints, soundtrack, Rust láminas, package round trip
```

Requirements: Python 3.11 or later; Node 22 or later (TypeScript canon and the WebAssembly
pen); `rustc` (Rust canon); `ffmpeg` (fingerprints); the Python with which manim-saorín
runs (`MESA_PY`, by default the Homebrew Manim one) and its folder (`MESA_IMPL`); the
pen's Python (`TORCULO_PLUMA_PY`). Heavy work goes through `tools/pesado`.

## How expectations are made

So that nobody has to trust a single implementation:

- **Diagnostics** are written by hand in `manual/lee.json`. The generator refuses to write
  a case if the reference disagrees, unless the case is marked **pending** with its reason.
- **Vocabularies** are extracted from the parser (`spec/vocabularios/extrae.py`): the
  specification is that table, and the case checks the parser still says it.
- **IRs** are the reference's, reviewed and frozen. Other compilers are held to the
  normative projection; the reference, to the whole IR.
- **Closure and mirror** are properties: the runner computes both sides.
- **Canon vectors** are computed in `generar.py` with integer arithmetic written again,
  not with the generated canon.
- **Láminas**: their hashes come from the WebAssembly build of the pen
  (`pluma/pruebas/referencia/_laminas.json`); Python and Rust must give the same bytes.
- **Takes, packages, shaders**: one broken file per rule, with the code the oracle must
  give; the generator stops if the oracle disagrees.
- **Fingerprints and soundtrack** are the reference interpreter's, on the reference
  machine; they are compared with tolerance (§13.4) and the soundtrack against the targets
  of §8.2 (−14 LUFS ± 1 LU, true peak at most −1 dBTP, the cues and counts).

## Case format

```json
{"id": "lee-errores-afecto", "kind": "lee", "input": {"pliego": "pliegos/errores/afecto.mesa"},
 "expect": {"valido": false, "errores": ["E010"], "avisos": ["W113"]}}
```

`id` equals the file name; paths are relative to `conformance/`. A case MAY carry
`"pendiente": "reason"`: a known deviation of the reference implementation (SPEC
Appendix D), reported apart.

| kind | input | expect | passes when |
|---|---|---|---|
| `vocabulario` | the vocabularies file | `iguales` | extracting from the parser gives the same file |
| `lee` | `pliego` | `valido`, `errores`, `avisos` | same validity and the same **sets** of codes |
| `ir` | `pliego` | `ir` (path), `sha256`, `proyeccion_sha256` | the normative projection's fingerprint matches; for the reference, the whole IR too |
| `cierre` | `pliego` | `iguales_modulo_procedencia`, `se_quita` | `canon(compile(p)) == canon(compile(escribe(p)))` |
| `espejo` | `pliego` | `efectivo_igual` | the effective workshop and hand of `p`, `parte(p)` and `funde(parte(p))` are equal |
| `canon_azar` | `vectores`, `lengua` | `iguales` | the language's canon gives every vector exactly |
| `canon_paleta` | `vectores`, `lengua` | `colores` | every colour of the generated file matches `canon.toml` |
| `lamina` | `dibujo`, `implementacion` | `sha256`, `valida` | the packed lámina has that hash and validates |
| `toma` | `toma` | `valida`, `resumen` or `codigo` | the reader gives that summary, or rejects with that code |
| `mesapack` | `mesapack` | `valido`, `errores` | the validator gives exactly those codes |
| `mesapack_referencia` | `pliego` | `valido`, `errores` | the packager's package of that pliego validates |
| `materia` | `wgsl`, `modulos` | `conforme`, `errores` | the checker gives exactly those codes |
| `huella` | `pliego`, `motor`, `calidad` | `huella`, `tolerancia` | the rendered film's fingerprint matches |
| `banda` | `pliego`, `motor`, `calidad` | loudness, true peak, cues, counts | the soundtrack report meets them |

## Adapters

The runner talks to an implementation through an adapter (`tools/adaptadores/`). An adapter
for another compiler offers the same functions as `manim_saorin.py`: `ir(pliego)`,
`lee(pliego)` returning codes, `escribe`, `parte`, `funde`, `taller`, and, for an
interpreter, `toca_gx(pliego, out)` writing the video, the master and the soundtrack report.

## The report

`ejecutar.py` prints a one-line summary and writes `informe.json`: the suite version and
`cases_sha256`, the date, the implementations, the totals (`pasa`, `falla`, `pendiente`,
`omitido`), the totals per kind, and one result per case with its detail when it did not
pass.


## The latest report

passes: 89 · fails: 0 · pending: 5 · skipped: 8

| Case | Class | Status | Detail |
| --- | --- | --- | --- |
| banda-estampas | banda | skipped | clase pesada: con --pesado |
| banda-oficio | banda | skipped | clase pesada: con --pesado |
| canon-azar-python | canon_azar | passes |  |
| canon-azar-rust | canon_azar | passes |  |
| canon-azar-typescript | canon_azar | passes |  |
| canon-paleta-css | canon_paleta | passes |  |
| canon-paleta-python | canon_paleta | passes |  |
| canon-paleta-rust | canon_paleta | passes |  |
| canon-paleta-typescript | canon_paleta | passes |  |
| canon-paleta-wgsl | canon_paleta | passes |  |
| cierre-dirigido | cierre | passes |  |
| cierre-escena | cierre | passes |  |
| cierre-servilleta | cierre | pending | La referencia rompe el cierre en una servilleta con banda: `mesa escribe` reescribe el preámbulo y pierde BANDA, EFECTOS, MEZCLA, MÚSICA y AMBIENTE (la IR escrita ya no suena). [/banda: sonido/banda ≠ None; /beats[0]/aspecto/ambiente: amb sala bucle ≠ None; /beats[0]/aspecto/musica: 1M1 ≠ None; /beats[1]/aspecto/ambiente: amb sala bucle ≠ None; /beats[1]/aspecto/musica: 1M1 ≠ None; /beats[2]/aspecto/ambiente: amb sala bucle ≠ None; /beats[2]/aspecto/musica: 1M1 ≠ None; /beats[3]/aspecto/ambiente: amb sala bucle ≠ None] |
| cierre-taller | cierre | passes |  |
| espejo-taller | espejo | pending | La referencia rompe la ley del espejo con la MANO de este pliego: importada desde `manos/firme.mano`, `toca la truca: sí` se lee como `no`; escrita en línea, `precisión alta` queda como una clave sin normalizar al lado de `precision alta`; y `parte` pierde el nombre de la sección (`#TALLER collage` → `#TALLER`). [MANO efectivo distinto: toca la truca: no; toca la truca: si] |
| huella-estampas | huella | skipped | clase pesada: con --pesado |
| huella-oficio | huella | skipped | clase pesada: con --pesado |
| ir-aspecto | ir | passes |  |
| ir-banda | ir | passes |  |
| ir-camara | ir | passes |  |
| ir-cartel | ir | passes |  |
| ir-dirigido | ir | passes |  |
| ir-escena | ir | passes |  |
| ir-estampas | ir | passes |  |
| ir-oficio | ir | passes |  |
| ir-oido | ir | passes |  |
| ir-raccord | ir | passes |  |
| ir-servilleta | ir | passes |  |
| ir-taller | ir | passes |  |
| lamina-manicula-python | lamina | passes |  |
| lamina-manicula-rust | lamina | skipped | Rust: con --pesado |
| lamina-manicula-wasm | lamina | passes |  |
| lamina-nota-aqui-python | lamina | passes |  |
| lamina-nota-aqui-rust | lamina | skipped | Rust: con --pesado |
| lamina-nota-aqui-wasm | lamina | passes |  |
| lamina-tintero-python | lamina | passes |  |
| lamina-tintero-rust | lamina | skipped | Rust: con --pesado |
| lamina-tintero-wasm | lamina | passes |  |
| lee-aspecto | lee | passes |  |
| lee-banda | lee | passes |  |
| lee-camara | lee | passes |  |
| lee-cartel | lee | passes |  |
| lee-dirigido | lee | passes |  |
| lee-errores-accion | lee | passes |  |
| lee-errores-afecto | lee | passes |  |
| lee-errores-ambiente | lee | passes |  |
| lee-errores-ambiguedad | lee | passes |  |
| lee-errores-ancla | lee | passes |  |
| lee-errores-bestiario-ausente | lee | passes |  |
| lee-errores-collage | lee | passes |  |
| lee-errores-color-voz | lee | passes |  |
| lee-errores-cue | lee | passes |  |
| lee-errores-dibujo | lee | passes |  |
| lee-errores-efecto | lee | passes |  |
| lee-errores-epoca | lee | passes |  |
| lee-errores-espacio | lee | passes |  |
| lee-errores-evento | lee | passes |  |
| lee-errores-funcion | lee | passes |  |
| lee-errores-gritos | lee | passes |  |
| lee-errores-letra-preambulo | lee | pending | La referencia acepta `LETRA` (una línea de clip) en el preámbulo sin error y la descarta en silencio: el signo no llega a la IR. Por la ley de las tres sedes (§3) una línea fuera de su sitio es un error (E090), nunca una línea que desaparece. [esperado {'avisos': ['W113'], 'errores': ['E090'], 'valido': False}, da {'valido': True, 'errores': [], 'avisos': ['W113', 'W124']}; mensajes: ['sin «silencio final» declarado (se aplica 3)', 'línea 12 · página vacía · b01: el beat corta a un cuadro nuevo y no po']] |
| lee-errores-letrero | lee | passes |  |
| lee-errores-movimiento | lee | passes |  |
| lee-errores-pantalla | lee | passes |  |
| lee-errores-pieza-sin-reparto | lee | passes |  |
| lee-errores-reservada | lee | pending | La referencia acepta `REPARTO nota: …` sin error y luego trata `nota:` como directiva (avisa «clip vacío»). §5.5 y §6.4 exigen E021 al declarar un hablante con una palabra reservada. [esperado {'avisos': ['W113'], 'errores': ['E021'], 'valido': False}, da {'valido': True, 'errores': [], 'avisos': ['W113', 'W116']}; mensajes: ['línea 12 · b01: clip vacío', 'sin «silencio final» declarado (se aplica 3)']] |
| lee-escena | lee | passes |  |
| lee-estampas | lee | pending | La referencia avisa «clip vacío» (W116) en un clip cuyo único contenido es un signo (`LETRA`). Según §8.3 un signo es una pieza y el clip no está vacío. [esperado {'avisos': ['W112'], 'errores': [], 'valido': True}, da {'valido': True, 'errores': [], 'avisos': ['W112', 'W116']}; mensajes: ['porcentaje pobre 82 % (sano: ≥85 %). 7 líneas con cifras; el pliego di', 'línea 39 · b03: clip vacío']] |
| lee-oficio | lee | passes |  |
| lee-oido | lee | passes |  |
| lee-raccord | lee | passes |  |
| lee-servilleta | lee | passes |  |
| lee-taller | lee | passes |  |
| materia-acuarela | materia | passes |  |
| materia-difuminado | materia | passes |  |
| materia-papel | materia | passes |  |
| materia-pincel | materia | passes |  |
| materia-risografia | materia | passes |  |
| materia-rota-m001 | materia | passes |  |
| materia-rota-m004 | materia | passes |  |
| materia-rota-m005 | materia | passes |  |
| materia-tejido | materia | passes |  |
| materia-transicion | materia | passes |  |
| mesapack-formato-futuro | mesapack | passes |  |
| mesapack-hash-roto | mesapack | passes |  |
| mesapack-ida-y-vuelta-dirigido | mesapack_referencia | skipped | clase pesada: con --pesado |
| mesapack-minimo | mesapack | passes |  |
| mesapack-no-zip | mesapack | passes |  |
| mesapack-ruta-peligrosa | mesapack | passes |  |
| mesapack-sidecar-absoluto | mesapack | passes |  |
| mesapack-sin-declarar | mesapack | passes |  |
| mesapack-sin-manifiesto | mesapack | passes |  |
| mesapack-sin-pliego | mesapack | passes |  |
| mesapack-sin-versiones | mesapack | passes |  |
| toma-bano | toma | passes |  |
| toma-estampa | toma | passes |  |
| toma-invalida-codigo-desconocido | toma | passes |  |
| toma-invalida-cortada | toma | passes |  |
| toma-invalida-post-fuera-de-sitio | toma | passes |  |
| toma-invalida-sin-start | toma | passes |  |
| toma-invalida-tras-end | toma | passes |  |
| toma-invalida-version-upstream | toma | passes |  |
| vocabulario-parser | vocabulario | passes |  |

JSON: https://torculo.joseluissaorin.com/conformance/informe.json