Computational Science

Two Mechanisms, One Perfect Fit: The Identifiability Problem

2026-09-10

A good fit does not guarantee that the data identify the model's internal parameters. Different parameter combinations can generate the same observable trajectory. Before interpreting a fitted mechanism, test whether alternatives remain indistinguishable under the available measurements. Algebraic checks, synthetic recovery and parameter-profile analysis can expose this limit computationally, sometimes showing that the original question cannot be answered from the existing data.

A fitting algorithm must return something

An optimizer can produce precise-looking parameter values even when many alternatives fit almost equally well. Its output is a location found under an objective, initialization and constraints. It is not automatically a unique scientific explanation. An intuitive mechanism may guide the model form, but fitting that form does not remove the need to ask whether its components are distinguishable.

Raue and colleagues describe structural and practical identifiability and the use of profile likelihood to investigate them. The distinction matters commercially: some uncertainty reflects limited or noisy observations, while another part is built into how the model maps hidden parameters to observable outputs. More computation on the same indistinguishable outputs cannot necessarily resolve the latter.

Sources: Raue et al. (2009): Structural and practical identifiability.

A complete toy example without any noise

Consider the hypothetical model y(t) = (a + b) × exp(-k × t). Let k be fixed at 0.2. Parameter pairs a = 2, b = 3 and a = 4, b = 1 produce exactly the same trajectory because both sums equal five. Even perfectly measured values of y at arbitrarily many times cannot distinguish these pairs within this model.

The identifiable quantity in this example is the sum, not the two components individually. If a and b were given different mechanistic names, a visually perfect fit would still not tell us which named contribution dominates. This is an algebraic demonstration, not an observed scientific result. It shows why explanatory detail can exceed what the measurement equation actually reveals.

Match the audit to the available evidence

The inputs for a real review are the exact model equations, parameter bounds, observed variables, measurement times, uncertainty assumptions and existing data. Begin by asking whether a reparameterization exposes a redundant combination, as in the toy sum. Then test synthetic recovery from several known parameter settings before interpreting estimates from empirical observations.

For noisy or finite data, inspect parameter profiles and tradeoffs rather than relying only on one fitted value. SciPy's curve-fitting documentation notes that covariance estimates depend on a local approximation and that ill-conditioned covariance can indicate unreliable results. A clean optimizer status is therefore not a sufficient uncertainty assessment.

Sources: SciPy: curve_fit.

The smallest useful computational challenge

Fit from several well-separated starting values under the same frozen objective and compare both predictions and parameters. If predictions remain stable while parameters vary widely, investigate the resulting ridge. For each parameter of interest, hold it at a sequence of values and refit the others. A broad flat profile indicates that the available data tolerate substantial change in that parameter.

Do not turn arbitrary parameter bounds into evidence. If the uncertainty interval ends only because the search hits an imposed limit, say so. Inspect whether a prior or regularization term selects one solution among many and distinguish that assumption from information supplied by the observations. The model may be useful for prediction while its internal explanation remains unresolved.

Stop the mechanism claim, not necessarily the whole project

Stop reporting a unique mechanism when distinct parameter settings make indistinguishable predictions over the available observations. Return the identifiable combination or a set of compatible explanations. If practical uncertainty is the issue, inspect whether another lawful existing dataset contains a more discriminating observable. If not, mark the question not estimable within the present evidence boundary.

Passing an identifiability check only means the model's parameters can, under the stated conditions, be distinguished. It does not establish that the model is correct, that all alternatives were considered or that the parameters have the proposed physical interpretation. Model adequacy and identifiability are complementary checks, not substitutes for one another.

What a buyer gains before a large research commitment

A useful first engagement returns an identifiability map: which quantities the data support, which depend on assumptions and which cannot be separated. Include alternative parameter solutions and their overlapping predictions. A follow-on proposal can then target a genuinely discriminating comparison instead of spending months refining a parameter that the current observations cannot determine.

This is one of the clearest examples of computational work improving research direction. A creative mechanism can be taken seriously enough to formalize, then challenged honestly enough to reveal its limits. The outcome may be a smaller model and a better question. That is substantive progress even without a new mechanism or a dramatic discovery claim.

Questions this raises

Will more observations always fix identifiability?

No. In the toy model, additional measurements of the same output cannot separate parameters that appear only through their sum.

Can a non-identifiable model still predict well?

Yes. Stable predictions and interpretable internal parameters are different properties and should be evaluated separately.

Sources and their limits

Prepared with AI assistance. The linked sources support the specified technical points; they do not validate applied psionics as a whole or guarantee a result for a client.

Read the editorial and evidence standard.

Continue reading

Explore Scientific Oracle consultingfor a scoped review of an existing-data research decision. Start with a non-confidential outline of the question, available evidence and the decision it needs to inform.