1. A final answer hides the route to failure

A policy assistant gives an outdated approval rule. The response is fluent, relevant, and accompanied by a citation. The team can call it a retrieval failure, a reasoning failure, or a hallucination, but those labels do not establish what to change. Several different paths could produce the same visible answer.

The index might contain only an old document. Retrieval might find the current document while context assembly drops it. The current policy might apply to a different business unit. A cached summary could carry the old rule. The correct information might reach the model and still be misused. Each explanation implies a different inspection and a different repair.

Our claim is methodological: diagnosis should follow the failed task through relevant boundaries and seek evidence that distinguishes explanations. We use a stack as a map of inspection questions. It is not a required classification system, a model of hidden neural computation, or a promise that every incident has one isolated cause.

2. The system is the appropriate unit of investigation

The user experiences a configured product: interface, sources, model inputs, business logic, tools, state, people, and external services. Investigating the model alone can exclude the component that introduced the problem. It can also overlook a boundary where a practical intervention would contain the consequence without changing model behavior.

Sculley and colleagues describe system-level technical debt in ML, including data dependencies, entanglement, feedback, configuration, and undeclared consumers. Their analysis supports attention to interactions beyond model code. It does not establish that any one of these factors explains a particular language-model incident.

The investigation should retain only distinctions that affect what the team inspects or does. A simple assistant may have no durable memory or external action. A complex workflow may revisit sources several times. Follow the architecture that exists and the job that failed rather than forcing both into a complete theoretical ladder.

Supporting reading: [1] Sculley et al. (2015). Hidden Technical Debt in Machine Learning Systems.

3. Establish the task and the judgment basis

Start with the customer’s intended work, relevant scope and constraints, the actual outcome, and why the difference mattered. In the policy example, asking for today’s approval process and asking which process applied to a past purchase are different tasks. An old policy is not intrinsically the wrong source.

Identify the applicable expectation before declaring failure. Was the product required to use a governing policy, report uncertainty, obtain confirmation, or preserve an earlier correction? If ownership or intent is disputed, record that gap and resolve it with someone accountable for the outcome. Otherwise the analyst risks inventing a requirement after observing an inconvenient result.

Separate source fidelity from truth and applicability. A response can accurately summarize an obsolete source. It can also make a true statement without the evidence the product promises to provide. These differences matter when deciding whether the repair belongs in source governance, retrieval, response behavior, or the product promise.

4. Capture the artifacts needed to reconstruct the execution

Preserve the request and relevant interaction, selected object, initial state, available sources, retrieval results, exact assembled input, tool calls and responses, state changes, and final user-visible effects. Record the versions needed to interpret them. Capture only the detail relevant to the judgment while respecting access and retention constraints.

The exact model input is especially useful when a system claims that it supplied a source. A retrieval log proves that an item was found, not that the item survived filtering, compression, formatting, or truncation. A final acknowledgment proves that a correction was mentioned, not that the corrected value reached the action.

This method does not require access to private internal reasoning. Observable inputs, outputs, events, and state are the evidence. A model’s generated explanation can suggest a hypothesis, but it is not privileged testimony about the mechanism that produced its behavior. Missing artifacts should become an explicit evidence gap.

5. Use boundaries to locate a visible mismatch

Inspect the boundaries implicated by the case. Did the system attempt the right task? Did needed material reach the relevant operation? Was it governing and applicable? Did the output preserve quantities, qualifiers, and constraints? Did a tool execute the intended action? Did later state retain valid corrections? Did the customer complete the work?

Locate the earliest mismatch that the available evidence demonstrates, then follow its propagation. A wrong account selection can enter a draft, survive review, and produce a wrong posting. The later events matter as consequences, but counting each as an independent initiating failure can obscure the leverage point.

Earliest visible does not mean ultimate cause. The wrong selection could result from ambiguous input, stale state, an omitted record, or interpretation despite adequate evidence. Preserve those alternatives. Containment may be justified immediately when consequences are serious, while explanation proceeds through a separate investigation.

text
Observed condition -> first demonstrated mismatch -> propagation -> outcome

Record separately:
- applicable expectation
- supporting artifact locations
- explanation A and a plausible alternative
- observation that would distinguish them
- proposed intervention and its expected effect

6. Worked hypothetical case: the obsolete approval rule

Consider a fictional assistant answering an employee’s question about a current equipment purchase. The governing policy requires approval from a department owner. An older policy names a different role. The assistant names the old role, and the employee spends time rerouting the request. These details illustrate the reasoning method; they are not a measured customer case.

The trace shows both policy versions among retrieved candidates. That weakens the explanation that the index lacked the current document, but it does not establish what the answering operation received. Inspecting the input shows that a compression step retained the old rule and omitted the current exception and effective date.

The first demonstrated information loss is therefore at assembly. A focused repair could preserve the governing passage and its applicability metadata. However, selecting the newest document indiscriminately would fail historical questions. The intervention should express the actual requirement: choose material applicable to the requested situation and preserve the distinctions needed to interpret it.

7. Compare explanations with focused interventions

Replay the saved case with the verified applicable passage supplied directly. Compare it with the current assembly under controlled conditions, then examine historical questions, conflicting authority, missing sources, and cases requiring several documents. This can test whether the proposed assembly change improves the relevant behavior without eliminating needed evidence.

A change in output supports a claim about that intervention under those conditions. It does not reveal a unique internal mechanism. If the prompt, model, document set, and answer template all change, a successful run cannot identify which change mattered. Keep the claim proportional to the comparison.

Liu and colleagues found position-sensitive performance in the long-context tasks and models they studied. That motivates testing placement and context construction locally. It does not establish that placement caused this hypothetical incident, nor that every current model behaves the same way. Use research to generate a discriminating check rather than to skip it.

Supporting reading: [2] Liu et al. (2023). Lost in the Middle: How Language Models Use Long Contexts.

8. Develop patterns without losing individual evidence

Across incidents, record concrete observations such as “replaced the corrected delivery address with the earlier value” or “claimed completion after a rejected submission.” Compare similar successes, disputed judgments, and cases where artifacts are missing. A recurring category should connect behaviors that fail for a common product reason.

Keep boundaries and counterexamples with the category. “Losing a correction” differs from a legitimate update based on new information. “Unsupported claim” differs from a false claim. A broad label such as poor reasoning can group failures whose interventions are unrelated, making a dashboard tidy while making engineering work harder.

A discovery collection may deliberately contain many difficult examples. Use it to form hypotheses, refine coverage, and develop criteria. Measuring how often a pattern occurs in production requires a suitable sample definition. The observed frequency in an incident queue reflects its selection process as well as the product’s behavior.

9. A practical investigation and release method

Keep the durable case small enough to use during actual work. Link the expected outcome, relevant trace, concrete mismatch, leading alternatives, chosen check, and resulting decision. A schema is useful only when it helps someone recover the reasoning or choose a better next step.

After a candidate repair, protect the original failure and relevant contrasts with appropriate checks. Use an ordinary unit or integration test for an exact software defect. Use behavioral cases when semantic and contextual interaction matters. Then observe whether the live workflow improves, including review, delay, and recovery work.

  • Establish the task, consequence, and applicable expectation.
  • Verify that the execution can be reconstructed.
  • Locate the first demonstrated mismatch and its later effects.
  • State competing explanations and a check that could distinguish them.
  • Compare a focused change while preserving relevant contrasts.
  • Decide, record limits, and follow the effect after release.

10. Counterarguments and limits of causal language

One objection is that a focused prompt adjustment can fix an incident faster than a structured investigation. Sometimes it can. The method should scale with consequence and recurrence. A reversible local fix may need a saved example and a quick comparison. A costly failure crossing several systems merits stronger reconstruction and attribution.

Another objection is that nonlinear interactions make localization misleading. We agree that a boundary map is not proof of independence. Interventions can change other signals or expose a different failure. Preserve whole-task comparisons and inspect collateral effects. The aim is a useful next check, not a complete mechanistic explanation of every computation.

Finally, calling the map causal can overstate its authority. We reserve causal explanations for claims with supporting comparisons and assumptions. Traces ordinarily establish sequence and observed dependencies. A repair can be useful even while its deeper mechanism remains uncertain. This paper does not provide a formal causal-identification procedure.

11. What would justify keeping this method?

The diagnostic method itself should face evidence. Does it change the next inspection, expose an unsupported attribution, distinguish a source problem from an interpretation problem, or prevent an ineffective intervention? Does that value exceed the cost of capturing artifacts and maintaining records? Compare fresh investigations when evaluating the method; repeatedly reclassifying a known incident confounds usefulness with familiarity.

Our conclusion is a practical position: a failed output should open an investigation into the job and the system that produced it. Preserve what happened, what might explain it, what changed, and what improved as separate claims. The resulting discipline is worthwhile when it leads to better interventions and less unsupported certainty, at a cost the team can sustain.