Scientific Discovery
The Prediction Contract: Make an Intuitive Hypothesis Risk Being Wrong
2026-09-10
A prediction contract is a short research specification, not a legal agreement. It states the target, data, comparison, expected result, decision threshold and conditions that make the test invalid. Freezing those details before the relevant analysis makes an intuitive idea genuinely vulnerable to an adverse result. Later changes remain possible, but they receive a new version and an exploratory label.
Replace a promising direction with an accountable statement
An intuition that a model is missing memory may be an interesting starting point. It becomes a testable claim only when memory has a defined mathematical representation and an observable consequence. Otherwise almost any later finding can be presented as confirmation.
Andrei's applied-psionics approach is exploratory: an impression nominates a candidate, while literature, data and computation determine its status. The prediction contract makes that handoff visible. It does not certify the origin of the impression or promise a positive result.
Nosek and colleagues frame preregistration around separating prediction from explanation after results are known. This proposed one-page contract applies that distinction to a specific consulting decision, while leaving room for explicitly labeled exploratory follow-up.
Sources: Nosek et al. (2018), The preregistration revolution.
A worked contract for an existing forecast archive
Consider a hypothetical archive of building-temperature forecasts. The candidate says adding one preselected lagged temperature feature will improve predictions on a fixed held-out month. The comparator is the existing model without that feature, trained on exactly the same earlier period.
The contract specifies mean absolute error, the eligible hours, treatment of missing records and a five-percent relative improvement threshold chosen for this example. It also states that improvement must not come from changing the evaluation month or discarding difficult days.
If baseline error is 2.0 units and the candidate error is 1.92, the relative improvement is four percent. That would miss the illustrative threshold even though the candidate is numerically better. The contract prevents a near miss from becoming an unqualified success.
| Field | Illustrative specification |
|---|---|
| Target | Hourly building-temperature forecast |
| Change | One fixed lagged feature |
| Comparator | Original model, same training interval |
| Primary metric | Held-out mean absolute error |
| Decision rule | At least 5% relative improvement |
| Invalidity condition | Outcome leakage or unusable timestamps |
Distinguish rejection from an invalid test
A hypothesis can fail its prediction even when the computation is correct. A test can also fail to evaluate the hypothesis because the data are corrupted or a required variable is absent. Give these outcomes separate labels.
Do not define invalidity so broadly that every adverse result qualifies. Missing metadata may invalidate a temporal analysis; an inconvenient effect size does not. State the technical conditions in advance and preserve the evidence used to invoke them.
The Center for Open Science permits conditional analysis plans and stresses distinguishing planned work from unplanned changes. Use explicit if-then rules for foreseeable data problems, rather than making up an alternative analysis only after seeing which result is attractive.
Sources: Center for Open Science, Preregistration.
Specify uncertainty and the scope of the win
Passing one threshold on one dataset does not establish a universal mechanism. Include an uncertainty assessment appropriate to the data structure and report sensitivity to defensible alternatives. A result resting on one unusual interval should be described differently from a stable pattern.
A prediction contract can also name an ordinary rival. In the forecast example, perhaps the lagged feature merely repairs a clock mismatch. A predictive gain would remain useful, but the scientific explanation would differ from a new memory mechanism.
Record what the result authorizes. It might justify a second dataset comparison, a more detailed model review or no further expenditure. It should not automatically authorize a broad scientific claim or an expensive implementation.
Use the contract to scope the next paid decision
For a first consulting discussion, identify the decision, the existing evidence and the smallest result that would change your next move. These inputs help separate a tractable computational question from an open-ended request to solve an entire field.
A Direction Preview can identify a bounded direction and the investigation needed to test it further. The detailed research scope, milestones and commercial terms are agreed separately. The prediction contract described here is a methodological aid, not an escrow mechanism or binding payment instruction.
The practical takeaway is to make the test readable before making it sophisticated. A short, unambiguous prediction that can fail is more useful than a long technical proposal that can accommodate every possible outcome.
Questions this raises
Is this a contract for automatic AI payment decisions?
No. It is a research specification. Payment terms and any proposed evaluation mechanism require a separate agreement.
Can a prediction be revised?
Yes. Preserve the original, explain the revision and evaluate the revised claim separately. Do not replace a failed prediction with its successor.
Sources and their limits
- Nosek et al. (2018), The preregistration revolution. Separating prediction from post-outcome explanation is the methodological context, not evidence for psionics.
- Center for Open Science, Preregistration. Existing-data plans should disclose prior exposure and distinguish planned analyses from exploratory changes.
Prepared with AI assistance. The linked sources support the specified technical points; they do not validate applied psionics as a whole or guarantee a result for a client.
Read the editorial and evidence standard.
Continue reading
- Why predictions should precede explanations
- What belongs in a Direction Preview
- Explore the topic library
Explore Scientific Oracle consultingfor a scoped review of an existing-data research decision. Start with a non-confidential outline of the question, available evidence and the decision it needs to inform.