Research

How to Audit Old Intuitive Predictions Without Rewriting the Past

2026-09-10

A retrospective intuition audit starts with the complete dated archive, not a shortlist of successes. Freeze which entries qualify, what each prediction meant and how outcomes will be scored before calculating performance. Separate genuinely pre-outcome records from reconstructed memories. The result can expose useful patterns and weaknesses, but it cannot retrospectively create blinding or preregistration that did not exist.

Start with the archive you actually have

Research notebooks are rarely designed for evaluation. They contain fragments, repeated thoughts, unsent drafts and predictions made after relevant information was already available. Treat those features as part of the evidence, rather than cleaning them away until the record looks convincing.

Create an inventory with a stable identifier, original timestamp, source file, known edits and the relevant outcome date. Preserve the originals. A later export can supply a convenient working copy, but it should not replace the record that establishes what existed when.

In Andrei's exploratory applied-psionics framework, this audit asks whether intuitive suggestions were operationally useful. It does not assume paranormal information access. Ordinary memory, expertise, exposure and chance remain explanations to examine.

Separate four kinds of historical entry

Imagine a hypothetical archive of 120 research notes. Thirty lack reliable timestamps, twenty describe outcomes already known, twenty-five are too ambiguous to score, and forty-five contain identifiable pre-outcome predictions. These numbers are illustrative, not a report of an audit performed here.

The forty-five entries may form a descriptive evaluation set under a published eligibility rule. The other seventy-five should remain in the inventory with exclusion reasons. Removing them from the archive itself would hide the difficulty of converting impressions into testable claims.

An old note that says the result will be unusual cannot become a prediction of a specific direction simply because that direction later occurred. If the meaning cannot be recovered without looking at the answer, mark it unscorable.

  • Dated and specific: potentially eligible for scoring.
  • Dated but ambiguous: preserve without a success label.
  • Post-outcome: useful history, not a prediction.
  • Missing provenance: report the uncertainty rather than inventing timing.

Freeze the audit before computing the headline

Write the selection rule, score, baseline and handling of missing outcomes in a short analysis plan. Disclose how much of the archive and its outcomes the auditor has already seen. A new plan can constrain the audit from this point forward; it cannot erase previous exposure.

The Center for Open Science specifically notes the importance of disclosing prior knowledge when preregistering analysis of existing data. Use that principle to label the audit honestly, rather than describing a retrospectively selected archive as a prospective test.

If classifications are disputed, retain both reasonable readings and show whether they change the conclusion. A disagreement over five entries may matter more than a sophisticated statistical model. Do not let software give false precision to an unstable coding decision.

Sources: Center for Open Science, Preregistration.

Ask what ordinary information could explain

A timestamp before an outcome does not prove the outcome was unpredictable. The note may have followed a conference abstract, an early chart, a public trend or a colleague's explanation. Add a field for information plausibly available when the prediction was made.

Kahneman and Klein discuss learnable regularities and feedback as conditions for intuitive expertise. That makes prior exposure a serious candidate explanation, not a nuisance to suppress. Recognizing a familiar pattern can be valuable without requiring an extraordinary account of how it happened.

For the hypothetical archive, compare predictions with an appropriate contemporary baseline. If nearly every project was already expected to finish late, predicting delay is a different achievement from identifying which apparently healthy project would fail. Keep those tasks separate.

Sources: Kahneman and Klein (2009), Conditions for Intuitive Expertise.

Turn the audit into an actionable research question

The strongest output may be a boundary: notes about timing were specific, while notes about mechanisms were mostly ambiguous. Another possible result is that confidence did not distinguish useful suggestions. Either finding can improve how the next computational question is formulated.

Before discussing a private archive, share only a non-confidential description of its size, dates, domains and outcome availability. The first decision is whether an honest audit is feasible. Detailed materials, access rules and the scope of further work can be agreed separately.

A credible summary reports eligible entries, exclusions, missing outcomes, the baseline and the limitations together. An impressive percentage without those fields is a story about selected memories, not a dependable measure of research performance.

Questions this raises

Can old notes establish predictive ability?

They may provide limited evidence if timing, meaning, selection and outcome scoring are credible. They cannot repair missing provenance or eliminate known exposure to answers.

Should ambiguous notes be deleted?

No. Preserve them and report why they cannot be scored. Their frequency is informative about the method's practical usability.

Sources and their limits

Prepared with AI assistance. The linked sources support the specified technical points; they do not validate applied psionics as a whole or guarantee a result for a client.

Read the editorial and evidence standard.

Continue reading

Explore Scientific Oracle consultingfor a scoped review of an existing-data research decision. Start with a non-confidential outline of the question, available evidence and the decision it needs to inform.