Progressive erosion of shared understanding between human and AI over an extended session. The AI's working model of the task, its constraints, and its history gradually diverges from the human's — without either party being immediately aware of it.
Definition
The AI's active working context is not a fixed record — it is a finite window that fills with new information as the session progresses. Early decisions, constraints, and agreements are progressively diluted, reinterpreted, or lost entirely. The AI continues operating, but on an increasingly inaccurate model of the task.
Root Cause
No persistent external memory. Context window constraints mean older information is deprioritised. Instructions established early in a session carry less weight than recent exchanges, even when the earlier instructions should remain authoritative.
Symptoms
- Outputs that were accurate earlier begin missing important nuances
- AI makes suggestions that contradict decisions already made
- Questions are asked that were clearly answered earlier in the session
- The correction rate increases noticeably over time
- Human starts feeling the need to restate context frequently
Observed Effects
- Compounding errors built on incorrect assumptions
- Increased human cognitive load (correcting rather than directing)
- Session restarts, losing all collaborative progress
- Erosion of trust in AI outputs generally
Structural Mitigations
- Running Documents — persistent external record of all active decisions, constraints, and context, updated continuously and reloaded at session start
- Stability Pings — scheduled explicit checks that the AI's current working context remains accurate
- Re-anchoring protocol — explicit restating of core constraints at key session transitions
The state of a document or artifact as the AI believes it to exist becomes inconsistent with the actual file — or with the human's understanding of what the file contains. Work proceeds on two different versions of reality simultaneously.
Definition
In extended AI collaboration, documents pass through multiple edit cycles. Without explicit state reconciliation, the version the AI is working from and the version on disk progressively diverge. Both human and AI continue working in good faith, on different documents.
Root Cause
No canonical source of truth. Multiple edit cycles without checkpointing. AI works from in-context representation of a document rather than the live file. Changes made outside the session are invisible to the AI unless explicitly reloaded.
Symptoms
- AI references content that does not exist in the current file
- Edits proposed don't match what the human sees on screen
- Disagreement between human and AI about what was last saved or agreed
- Multiple versions of the same section existing in different places
Observed Effects
- Work built on wrong foundations — sometimes discovered late
- Significant rework when the divergence is eventually found
- Trust breakdown when errors are discovered after the fact
- Time lost reconstructing what the actual current state is
Structural Mitigations
- File Governance protocol — explicit rules for which file is canonical, how versions are named, and when the AI's working copy must be refreshed from disk
- State confirmation — before major edits, AI explicitly states its current understanding of the document state and human confirms or corrects
- Single canonical document — one authoritative version; all others are drafts or archives
Unverified quantitative outputs are used as inputs to subsequent calculations or decisions, compounding errors silently. By the time the problem becomes visible, a significant body of work has been built on incorrect numbers.
Definition
AI systems generate numerical outputs with high confidence regardless of accuracy. Without an explicit verification step, these numbers enter the workflow as facts. Downstream calculations use them. Reports cite them. Decisions rest on them. The error compounds with each propagation step before anyone checks the source.
Root Cause
Generation and verification treated as a single operation when they are two distinct ones. AI numerical output carries no accuracy signal — a correct figure and a plausible but wrong figure are presented identically. The human has no structural prompt to verify before using the output.
Symptoms
- Calculations that cannot be independently cross-checked
- Figures that shift between sessions without explanation
- Numbers that cannot be traced back to a primary source
- Totals that don't reconcile when checked manually
Observed Effects
- Large blocks of work requiring complete reconstruction
- Loss of confidence in the entire dataset, not just the erroneous figures
- Time spent re-deriving correct numbers from scratch
- Decisions made and sometimes acted on based on wrong data
Structural Mitigations
- Verification Gates — explicit checkpoints at which AI-generated numbers are confirmed before being used downstream
- Source tracing — every figure must be traceable to a primary source; AI-generated calculations treated as estimates until verified
- Step Mode for quantitative work — AI presents calculations one step at a time for inspection rather than delivering completed figures
Over an extended session, the AI's reasoning gradually detaches from the specific domain constraints and knowledge that define the actual task — drifting toward outputs that are generically plausible but domain-inappropriate.
Definition
Domain context — the specific terminology, constraints, norms, and knowledge that make a task tractable — is established early in a session but erodes as the session continues. The AI optimises for coherent, helpful-sounding output rather than domain accuracy. The outputs pass surface-level review but fail when assessed by someone with domain expertise.
Root Cause
Domain constraints are contextual, not structural. The AI has no mechanism to distinguish "this is a general principle" from "this is specific to this domain and must not be violated." As context fills, domain specificity erodes toward statistical plausibility.
Symptoms
- Advice that sounds sensible generally but is wrong for this specific context
- Loss of domain-specific vocabulary; replacement with generic equivalents
- Citations or frameworks that apply to adjacent fields, not this one
- Recommendations that ignore constraints stated clearly at the start of the session
Observed Effects
- Work that passes casual review but fails domain expert scrutiny
- Errors that are invisible to a non-expert until consequences emerge
- Accumulated rework when domain inappropriateness is identified late
Structural Mitigations
- Domain anchoring — explicit re-statement of domain constraints at session transitions and before any significant output is generated
- Domain-specific Running Documents — a dedicated section in the canonical document recording the domain rules the AI must not violate
- Specialist review checkpoints — structured moments where domain expert confirms AI outputs remain appropriate before work continues
A cumulative state in which multiple small failures — individually recoverable — erode the human's confidence in AI outputs generally. Over-correction, excessive verification, and eventual disengagement follow.
Definition
Trust in AI collaboration is not binary. It degrades incrementally. Each unexplained error, each output that requires correction, each moment where the AI contradicts itself — these subtract from a working trust reserve. When the reserve is exhausted, the human can no longer treat AI output as a starting point. Collaboration becomes a net cost.
Root Cause
Lack of transparent failure signalling — errors appear without explanation or acknowledgment. Absence of repair protocols — there is no structured way to restore the collaboration relationship after a failure. Errors compound without visible recovery, creating the impression of unreliability even when individual failures were minor.
Symptoms
- Human checking every output even when verification is not warranted
- Reluctance to delegate substantive tasks to AI
- Explicit expressions of doubt in the collaboration ("I don't trust this")
- Collaborative productivity falls below what the human could achieve alone
Observed Effects
- The collaboration relationship becomes a burden rather than a benefit
- Human disengages — either abandons the session or continues with excessive manual oversight
- Future collaboration attempts start from a damaged baseline
Structural Mitigations
- Challenge Protocol — a structured mechanism for the AI to explicitly flag uncertainty before producing output, rather than producing plausible-but-uncertain output silently
- Transparent error acknowledgment — errors are named, documented, and explicitly recovered from, not glossed over
- Error Recovery protocol — a defined sequence for restoring shared understanding after a failure, designed to rebuild rather than ignore the trust damage
After a significant disruption — session end, context reset, major error, or version conflict — the human and AI cannot recover a coherent shared working state. The cost of reconstruction exceeds the perceived benefit of continuing.
Definition
AI collaboration systems have no native persistence. When a session ends or context is reset, everything held in working memory is lost. If no external record was maintained, the only option is to reconstruct from memory — which is incomplete, inconsistent across participants, and expensive. At some point reconstruction cost exceeds restart cost, and the work is abandoned.
Root Cause
No persistent external memory. No session handoff protocol. Intermediate decisions — the small choices made during a session that shape all subsequent work — exist only in the session context. When the context is lost, those decisions are lost too, even if the final artefacts survive.
Symptoms
- Inability to locate previous decisions after a session break
- Human and AI disagreeing about what was agreed versus what was discarded
- New session starts re-covering ground already covered
- Artefacts exist but context for why they are in their current state is gone
Observed Effects
- Complete restart from the beginning, losing all intermediate work
- Decisions unmade and remade inconsistently across sessions
- In severe cases, permanent abandonment of the collaborative task
Structural Mitigations
- Running Documents — a continuously updated external record that functions as the session's memory, recoverable at any future point regardless of context resets
- Session Handoff Protocol — a structured end-of-session summary capturing current state, outstanding decisions, and next steps, designed to enable a cold-start continuation
- Explicit checkpointing — designated moments at which current state is formally recorded, creating reliable restore points