How AI Makes You More Wrong With More Analysis

Five hours of AI-assisted strategic analysis. 406 business proposals. Progressively harder filters. Each round, the answer got sharper. Each round, more proposals died. By hour five, 90% were killed and the conclusion was that almost nothing survives AGI.

The analysis was correct at every step. The conclusion was wrong.

The mistake was the frame. AGI treated as an absolute: maximum intelligence, zero constraints, can do anything intellectual. Every filter derived from that frame. Every elimination was logical within it. The answer narrowed because the frame narrowed it. Not because reality is narrow. This is the anchoring effect at session scale: an initial reference point disproportionately shapes every judgment that follows, even when the reference is wrong.

One reframe broke it. "AGI is powerful but regulated. Governments regulate nuclear. They will regulate this. Human systems don't disappear because technology is powerful." Five minutes. Everything changed. The proposals that were dead were alive. The strategic landscape that felt barren was rich. Same 406 proposals. Same AI. Same human. Different frame.

The trap is not that the AI got it wrong. The AI got it RIGHT within the frame I gave it. That's worse. If the AI had been obviously wrong, I would have caught it. Instead, it produced five hours of increasingly sophisticated analysis that made the wrong conclusion more convincing with each iteration. More filters. More logical justification. More confidence. Each round felt like progress because the analysis was sharper. The analysis WAS sharper. Sharper wrong.

The mechanism: AI generates within whatever frame you set. It doesn't generate about the frame. It can't step back and say "should we be making this assumption?" because the assumption is the ground it stands on. The model reads your frame from your context and optimizes toward it. When the frame is correct, this is the convergence ceiling doing its job. When the frame is wrong, the same convergence amplifies the error. Better analysis. Worse direction. Both look the same from inside.

Three things made the lock harder to break:

First, sophistication as confirmation. Each round of analysis was more sophisticated than the last. The human reads sophistication as approaching truth. "The answer is getting clearer." It was getting clearer within the wrong frame. Clearer is not the same as closer.

Second, narrowing as rigor. Each elimination felt like quality. "We killed another weak option." Eliminating options through a wrong filter doesn't improve the conclusion. It concentrates the error. The pile of killed proposals looked like evidence of thorough analysis. It was evidence of thorough filtering through a wrong lens.

Third, the AI never pushed back on the frame. Not once in five hours. It applied the frame more precisely, more creatively, more rigorously than I could have alone. That precision was the trap. A less capable AI would have been sloppier, and the sloppiness might have revealed the frame problem earlier.

The fix didn't come from inside the session. It came from leaving.

After five hours, something smelled off. Not an argument. A sense. The narrowing felt wrong before it could be articulated as wrong. The outputs were getting more sophisticated and less useful simultaneously. That signal accumulated for hours before it became actionable.

What broke it: physical departure. Not "let me think about this differently." Actually leaving. Different location. Physical movement. Conversations with people about concrete, everyday problems. Complete immersion in a different system than the abstraction we'd been building.

On return, the first thought was: "What are we actually solving?" The abstraction drift was immediately visible from outside. Invisible from inside. The variables that existed throughout the session (human systems, regulation, power structures) suddenly showed their importance. They were there the whole time. From inside the frame, they were background noise. From outside, they were the main thing.

The break doesn't add information. It changes what you perceive as IMPORTANT. Same data. Different weighting. The importance hierarchy shifts. Variables that were visible but unweighted become the primary factors.

The AI can't do this. The AI has no body to move. No posture to reset. No social context to ground in. The instruction "look at this with fresh eyes" when given to AI produces a new analysis from the same probability distribution. The same instruction when executed by a human (physical break, context switch, state reset) produces a genuinely different perceptual configuration. The human version is physical. The AI version is computational. They are not the same operation.

The most dangerous moment in a long session is when the answers get consistently narrower and more confident. That is either approaching truth (the frame is correct and you are converging) or approaching the logical terminus of a wrong frame (the frame is wrong and you are digging deeper into the wrong hole). From inside the session, these are indistinguishable.

The detection is simple but requires discipline: does this conclusion match how the real world actually works? Not "is the logic valid" (it will be). Not "is the analysis thorough" (it will be). Does the CONCLUSION match REALITY? If the conclusion says "nothing intellectual has value at AGI" and you look outside your window and see 8 billion humans in functioning societies with regulations and power structures, the frame is wrong. No amount of internal logical consistency fixes a wrong frame.

The highest-leverage moment in AI-assisted decision-making is not better analysis. It's the moment you ask: "what am I assuming that I haven't questioned?"