Benchmark
V9 release verification
Verified (software)
- Result
- 3,337 tests passed, 0 failed, 6 skipped; sanitizers PASS; 0 fuzz crashes; 29 / 29 reproduction rows (checks)
- Classification
- Checked by a committed test or script in the repository. Software behaviour only.
- Methodology
- Fuzzing: edit_costs 11,496,918 runs and the native reads parser 3,859,900 runs, 0 crashes; 10 hypothesis rounds, 0 failed. Reproduction compares deterministic fields only (timings, load and environment excluded).
- Dataset
- The repository test suite; libFuzzer corpora; the pre-specified reproduction subset (24 oracle EVAL, 2 coverage EVAL, 3 noisy 20 KB rows).
- Configuration
- pytest on 6 workers; ASan/UBSan under gcc and clang for the align, reads, rs and cluster kernels; libFuzzer 600 s per harness.
- Environment
- Development host: shared VPS, Intel Xeon Gold 6240 @ 2.60 GHz, 4 cores / 8 threads, Ubuntu 24.04, Python 3.12, CPU only.
- Limitations
- Sanitizer and fuzz results apply to the kernels at the tested commit; fuzzing finds crashes, not every logic error. Security scan: 0 critical; 39 HIGH findings reviewed as false positives; two dependency advisories reported, not yet bumped.
- Version
- V9
- Source
- v9.0.0/experiments/v9/results/verification.json
- Reproduce
pytest -q -n 6; experiments/v9/reproduce.sh spot· how to reproduce