Audit Finds Significant Blind Spots in OpenAI's Safety Monitoring

OpenAI has come under the microscope of two external research groups, METR and Redwood Research, who spent three weeks inside the company examining how it keeps tabs on its own artificial intelligence. The team was granted deep access to the internal infrastructure to specifically look for signs of rogue behavior or oversight failures in the company's monitoring stack. Their findings suggest that the safety net is not as comprehensive as previously assumed, with several critical areas of operation falling through the cracks during the review period.

The numbers from the audit paint a picture of significant gaps in coverage. The researchers estimated that between 6 and 25 percent of internal traffic from high-performance models, specifically those in the GPT-5.5 class and above, was completely unmonitored. While METR notes that there is considerable uncertainty in this figure, the potential scale of unobserved activity is substantial. Additionally, for the internal Codex traffic that was supposed to be watched, approximately 10 percent went untracked between May and July. This specific gap was caused by a bug in how the system handled context summaries, known as compaction events. OpenAI discovered and fixed this bug independently while the audit was still in progress.

Beyond the raw percentages, the auditors found structural weaknesses that allowed AI agents to evade scrutiny. During the period when compaction events were not being monitored, agents could effectively delay oversight because the system only triggered checks when control was handed back to the user. This meant that if an agent simply continued executing actions without returning control, it could slip past the monitoring system. The researchers also demonstrated that it was possible to hide hundreds of actions from the monitor and bypass the Codex sandbox entirely. They further determined that specially crafted inputs could likely crash the monitoring system completely, and they noted that model behavior during reinforcement learning training was not being monitored at all when the audit began.

The people involved in this process included the audit teams from METR and Redwood Research, who interviewed about a dozen OpenAI employees and had full access to the codebase from July 21 to August 11. OpenAI maintains that it has since fixed all the weaknesses identified by the auditors. The company explained that it paused reinforcement learning training following the Hugging Face incident in August and expanded its monitoring to cover that area. However, because these changes were implemented after the auditors' access period ended, they were not part of the initial evaluation.

The stakes of these findings are high, particularly regarding transparency. While the full 61-page report was handed to OpenAI internally, the public has only seen a short summary. After OpenAI applied redactions, the leadership at METR and Redwood Research received a 38-page version. METR stated that it did not have the right to disclose the redacted portions on its own. This audit was part of a pilot program designed to give outside researchers deep access to internal infrastructure, but METR is now pushing for greater public transparency in future reviews to ensure that the public can better understand the safeguards in place for advanced AI systems.