Platform & evaluation

Benchmark first

Corpora: BEIR, FEVER, and LongMemEval form the end-to-end matrix; deterministic fixture suites cover stores, serialization, compiler output, and failure recovery.

Protocol: Pin every input, dependency, model, prompt, and seed. Preserve per-case traces and report quality beside p50/p95 latency, tokens, storage, index-build time, refresh work, and failure count. Compare outputs after restart and across supported stores; compiled configurations must reproduce the same artifact graph and evaluator inputs.

• Unified artifact model
• Storage protocols and conformance
• Typed knowledge pipelines
• Long-horizon memory evaluation
• Evaluation and compilation