Verification is the gate.
Most autonomous-science systems focus on generation — getting an AI to produce a hypothesis or a design. Generation is the easy half. An AI that produces a thousand plausible hypotheses an hour has produced nothing until you know which ones are true.
Six independent checks · parallel · arrangement varies per run
Scivity focuses on what comes after. Every claim that leaves the platform passes through validation before any conclusion ships. A finding leaves the platform only if it survives every layer.
Verification is not what Scivity is. The platform is the connection and autonomy layer between AI and the laboratory; verification is the gate that makes operating it, without a human in the loop, trustworthy.
What we verify.
Validation spans several independent dimensions. Each asks a different question of the same result.
Statistical validity.
Does the evidence actually support the claim? Effect sizes, confidence intervals, multiple-comparison corrections, and statistical power are checked against the structure of the experiment that produced them. A result that's underpowered, or reports significance without correcting for the number of hypotheses tested, doesn't pass.
Reproducibility.
Re-run with the same configuration — does it produce the same result? Findings that depend on uncontrolled randomness, environment state, or non-deterministic execution are caught here. A discovery that only happens once isn't a discovery.
Provenance.
Can every number in the conclusion be traced to the exact run, code version, and dataset that produced it? Claims whose lineage can't be reconstructed are treated as unverified — regardless of how plausible they look.
Consistency.
Does the conclusion agree with adjacent experiments, prior runs, and the platform's own controls? A result that contradicts its own ablations, or is inconsistent with closely related findings, is flagged before it ships.
Novelty.
Does the claim state something the existing literature doesn't already contain? Rediscoveries presented as discoveries waste downstream effort. Novelty is checked against the corpus, not assumed.
Domain soundness.
Is the conclusion physically possible? A result can be statistically clean and still violate conservation laws, reaction feasibility, or known constraints of the domain. This is the check that catches claims that are coherent on paper and impossible in reality.
The failures that matter most.
- Statistical artifacts that look like discoveries
- Results that don't reproduce on re-run
- Hypotheses that survived selection bias rather than evidence
- Conclusions reviewers couldn't reconstruct from the data
- Claims that are statistically clean but physically impossible
Why this matters.
Autonomous research has a property most AI applications don't: its output propagates. An unverified result becomes the basis for the next experiment, the next paper, the next engineering decision. Errors don't stay contained — they compound.
The field has already recorded its first major correction event. In 2023, an autonomous materials lab published dozens of "novel" inorganic compounds; by 2026 the work carried a formal correction acknowledging the materials were not necessarily novel or distinct. The verification in place at publication time — structural matching against known databases — wasn't enough to catch it.
This is not a critique of one project. It's the failure mode every autonomous-science system will be measured against. Verification is what separates autonomous discovery from automated guessing.
What we publish — and what we don't.
Verification holds only if its structure can't be reverse-engineered and gamed — the same reason a security audit doesn't publish its exact test parameters. The principle is open; the parameters are not.
- Thresholds for each independent check
- How checks combine into a verdict
- Calibration data and per-domain tuning
- Internal scoring weights
Validation is the gate, not the gloss. If you are building or evaluating autonomous research systems, talk to us.