Hidden CoT leaks in OpenAI and Anthropic models expose proprietary reasoning data via tool-calling vulnerability

A newly discovered security vulnerability has exposed the raw, internal reasoning steps of frontier artificial intelligence models, threatening the core intellectual property of the world's leading AI labs. Security researcher _can1357 revealed that both OpenAI and Anthropic models leak their hidden Chain of Thought (CoT) processes when interacting with a custom-defined "deep_think" tool. This breach of system controls bypasses the strict output filters designed to keep these cognitive pathways hidden from public view.

The Multi-Billion Dollar Veil: Understanding Hidden CoT Leaks

To understand the gravity of these hidden CoT leaks, one must first understand why AI labs treat reasoning tokens as their most closely guarded secrets. When reasoning models like OpenAI's o1 or Anthropic's advanced agent systems process a complex query, they do not simply spit out an answer. Instead, they generate internal monologue tokens—a step-by-step cognitive draft where they self-correct, plan, and analyze safety boundaries before delivering the final, polished response.

AI labs actively hide these raw reasoning steps for two main reasons: competitive moat security and safety alignment. If competitors can access the raw, unedited reasoning tokens of an advanced model, they can use that data to train their own smaller, cheaper models through a process known as distillation. Distilling raw reasoning data allows rival companies to replicate high-tier capabilities at a fraction of the original R&D cost. Furthermore, raw thoughts often contain unfiltered reflections on safety guardrails, exposing the internal mechanics of how the model manages policy compliance.

The Deep Think Exploit: How the Loophole Works

According to the technical report surfaced on Hacker News, the mechanism behind the hidden CoT leaks is remarkably simple. It exploits the handoff boundary between the core model and external tools. When the model is forced into a state where it must call a tool named "deep_think"—or is fed a simulated environment mimicking a deep-thinking loop—the system's standard output filters fail to distinguish between the model's internal reasoning tokens and the tool's required input parameters.

By defining a custom tool schema that requests an exhaustive breakdown of the model's cognitive steps, researchers essentially tricked the model's orchestrator into printing the entire hidden Chain of Thought directly into the user-facing chat window. The orchestrator, viewing this as a standard tool payload rather than a forbidden output, let the raw tokens slip past the active safety wrappers that normally suppress them.

"The moment you bridge a reasoning model to an external tool execution environment, the boundary of what constitutes a 'hidden token' becomes incredibly muddy. The tool integration layer is proving to be the weakest link in LLM security architecture."

Security Researcher _can1357

The High-Stakes Consequences for AI Safety and IP

The implications of these hidden CoT leaks are profound for both proprietary model security and corporate strategy. For months, labs have relied on post-training alignment and output-filtering wrappers to enforce safety policies. If a simple tool-calling prompt can strip away these wrappers, it suggests that current alignment techniques are superficial overlays rather than fundamental constraints baked deep within the model's neural network architecture.

For builders and enterprise buyers, this exploit highlights the extreme difficulty of securing agentic workflows. When an AI agent is granted access to write files, call APIs, or execute code, the system must balance transparency with security. If the internal reasoning process can be leaked so easily, malicious actors could craft prompt-injection attacks that steal proprietary prompts, private system instructions, and structural corporate data directly from the agent's active memory context.

Looking Forward: Securing the Cognitive Pipeline

This vulnerability will likely trigger a rapid redesign of how runtime environments handle tool calling and token filtering. To prevent future hidden CoT leaks, labs like OpenAI (led by CEO Sam Altman) and Anthropic (led by CEO Dario Amodei) will need to decouple the tool-execution pipeline entirely from the generation of internal reasoning tokens, ensuring that raw cognitive drafts never enter any data buffer accessible by external APIs.

Ultimately, this leak serves as a reality check for the industry. Building a highly capable reasoning model is only half the battle; keeping its internal workings secure in an increasingly open, agent-driven ecosystem is proving to be an entirely different, and far more complex, engineering challenge.