Supported

Verification portfolios¶

Contract¶

Mechanism	Protects against	Contract boundary
Best-of-N	A single poor sampled candidate	Correctness when every scorer is wrong
Verdict consensus	One unstable judgment	Independence between judges
Grounding score	Unsupported required ideas	Truth beyond the supplied evidence
Selection trace	Opaque winner choice	Calibration of injected scores

Example consensus example

Candidate	Evidence weight	Verdict
A	0.70	Supported
B	0.20	Contradicted
C	0.10	Uncertain

The selected verdict is supported. Equal supported and contradicted weight produces abstention, leaving the tie explicit.

select_best

Scores existing candidates with stable tie-breaking.

verdict_consensus

Aggregates supported, contradicted, and uncertain assessments.

score_grounded

Evidence, coverage, completeness, corroboration, certainty.

harmonic_score

idea_completeness

Score meaning. Scores are deterministic quality signals for selection and abstention. Probability calibration belongs to a separate analysis.

Numerical evidence and conclusion policy¶

weighted_mean returns normalized weights and each observation’s numerical contribution. wilson_proportion returns a finite point estimate and interval for caller-labeled binary outcomes. The caller names outcomes, selects actions, and interprets uncertainty.

Preserve outcome uncertainty as ranking features¶

from mari_kit.knowledge import WeightedObservation, weighted_mean, wilson_proportion

estimate = weighted_mean([
    WeightedObservation(observation_id="study-a", value=0.20, weight=2.0),
    WeightedObservation(observation_id="study-b", value=-0.10, weight=1.0),
])

success_rate = wilson_proportion(successes=7, total=10)
assert success_rate.level == 0.95

Output	What it says	Boundary
Contribution `0.133`	Observation A’s weighted contribution	That A is independent or authoritative
Interval containing `0.70`	Sampling uncertainty under a binomial model	Next remediation choice

Wilson score interval Cochrane statistical methods

Research basis

Self-consistency improves chain-of-thought reasoning FEVER: evidence-based verification ALCE: citation quality evaluation

Mari exposes an auditable selection portfolio. Model sampling and benchmark metrics belong to the surrounding evaluation system.

Verification functions score parsed values and retain successful attempts and failures for audit.

How it works¶

best_of_n calls the producer up to n times and parses each output. Parse failures remain beside successful siblings. Valid candidates receive scores, followed by selection with stable first-wins ties. A configured threshold stops production early. verdict_consensus counts typed verdicts. Grounding scores combine declared deterministic components for ranking or abstention. Probability calibration requires separate work.

verify.py¶

from mari_kit.verification import best_of_n, score_grounded

result = best_of_n(
    lambda: model(question, documents),
    lambda raw: parse_answer(question, documents, raw),
    lambda answer: score_grounded(answer,
        required_ideas=("eligibility", "time limit")),
    attempts=3, threshold=0.90)

audit(result.selected, result.attempts, result.failures, result.stopped_early)

Retain selected candidates together with exact evidence and the scorer recipe. A scorer or required-idea change is a separate dependency from the source text and can invalidate a cached selection.