Scientific Discovery
When Intuition, AI and Data Disagree: Keep a Research Disagreement Log
2026-09-10
A disagreement log records the original intuitive candidate, the AI's objections, the data result and the reason for the final decision. Do not merge them into a consensus paragraph before resolving their differences. Turn each disagreement into a specific missing fact, alternative model or computational check. Some conflicts should remain unresolved when the available evidence cannot discriminate between them.
Agreement is not the only useful output
A model can produce a fluent explanation supporting almost any loosely framed idea. A researcher can also prefer interpretations that preserve an initial intuition. If the workflow rewards agreement, it may remove precisely the objections that would make the research useful.
In Andrei Ursachi's exploratory applied-psionics approach, intuition nominates hypotheses and AI helps expand and challenge them. Neither receives the authority to overrule inconvenient data by rhetorical force. A disagreement log preserves that division of responsibility.
Name the disputed claim narrowly. The intuition says there is a delayed relationship is not the same proposition as the model says this mechanism is established. A discussion can appear contradictory simply because the participants are answering different questions.
A hypothetical dispute about a lagged signal
Imagine a review of public environmental measurements. The intuitive candidate predicts that one variable leads another by a fixed interval. An AI review objects that both may follow the same daily cycle. The initial analysis shows a lagged association.
The association does not settle the dispute. It is compatible with the intuitive prediction and with the ordinary shared-cycle explanation. The next useful check is whether the nominated relationship remains informative relative to a preselected seasonal baseline on suitable held-out periods.
Record the candidate, the objection and the unresolved discriminator separately. If the shared-cycle baseline explains the result, keep that adverse conclusion. If metadata are insufficient to compare timing reliably, the right status is unresolved rather than consensus achieved.
| Voice or artifact | Specific contribution |
|---|---|
| Intuitive candidate | A fixed lead-lag prediction |
| AI challenge | A shared daily cycle may explain it |
| Initial computation | Association observed in the illustrative scenario |
| Required discriminator | Comparison with the defined seasonal baseline |
Classify the disagreement before adding more debate
Some disputes concern facts: a unit, a dataset version or a published result. Others concern model assumptions, decision costs or the meaning of success. Sending all of them into a longer model conversation wastes effort because they require different resolutions.
For factual disputes, retrieve and verify the original source. For analytical disputes, run the specified comparison. For commercial disputes, return to the agreed decision criterion. An AI preference cannot decide a client's risk tolerance, and a contract cannot make a scientific proposition true.
Silberzahn and colleagues showed that different defensible analytical choices can lead to varying answers from the same dataset. That is a reason to expose consequential choices rather than present one polished analysis as the only possible reading.
Sources: Silberzahn et al. (2018), Many Analysts, One Data Set.
Preserve the order in which minds changed
Save the initial ranking, the model output, the chosen computational test and the resulting revision. A final explanation alone cannot show whether the AI independently challenged the idea or merely restated an interpretation that had already changed.
When a new hypothesis emerges after seeing data, label it as exploratory. Nosek and colleagues distinguish this mode of explanation from a prediction specified earlier. The disagreement log can preserve both without pretending they carry the same evidential weight.
Multiple models agreeing does not create independent scientific replication. They may share source material, training patterns or prompt assumptions. The decisive question is whether their claims survive a relevant external check, not how many model names appear beside the conclusion.
Sources: Nosek et al. (2018), The preregistration revolution.
A stronger recommendation includes its strongest objection
A buyer should receive the recommended next step together with the main reason it might be wrong, the evidence already checked and the unresolved question that matters most. This is often more actionable than a report that sounds certain throughout.
For an initial consulting discussion, describe where your team currently disagrees and what existing evidence is available. A bounded review can convert that disagreement into a targeted computational question and a proposal for deeper investigation if the result warrants it.
The practical takeaway is to use disagreement as a map of the next check. The goal is not a room in which intuition, AI and data all sound harmonious. It is a decision whose remaining uncertainty is visible.
Questions this raises
Should the final memo include every AI message?
Not necessarily. Preserve the complete record privately and summarize the material objections, changes and unresolved points with traceable references.
Can three AI models settle a scientific disagreement?
Their analysis may help, but model agreement is not external validation. The disputed claim still needs relevant evidence and a defensible comparison.
Sources and their limits
- Silberzahn et al. (2018), Many Analysts, One Data Set. Different defensible analytical choices can produce varying answers to the same data question.
- Nosek et al. (2018), The preregistration revolution. Separating prediction from post-outcome explanation is the methodological context, not evidence for psionics.
Prepared with AI assistance. The linked sources support the specified technical points; they do not validate applied psionics as a whole or guarantee a result for a client.
Read the editorial and evidence standard.
Continue reading
- Assign intuition and AI different jobs
- What an independent hypothesis review should challenge
- Explore the topic library
Explore Scientific Oracle consultingfor a scoped review of an existing-data research decision. Start with a non-confidential outline of the question, available evidence and the decision it needs to inform.