OpenAI faces scrutiny after outside researchers document safety monitoring gaps

OpenAI has acknowledged that its internal systems for monitoring advanced AI models contained significant weaknesses, following a detailed audit conducted by outside researchers this summer. The findings, released on October 23, reveal that a substantial portion of traffic from the company's most powerful models was not being tracked by its safety oversight tools at all during the review period.

The audit was carried out by two researchers from METR and Redwood Research. They were granted deep access to OpenAI's internal infrastructure to evaluate how the company detects and manages rogue or unexpected AI behavior. This pilot program was a joint effort designed to allow external experts to inspect the company's codebase and operational procedures in a way that is rarely possible.

The team was on-site at OpenAI from July 21 to August 11. During these three weeks, they had full access to internal documents and the relevant code. They also conducted interviews with approximately a dozen OpenAI employees to understand how the monitoring systems functioned in practice. This direct access allowed the auditors to test the systems rather than just reviewing documentation.

One of the most significant findings concerned the GPT-5.5 class of models. The auditors estimated that between 6 and 25 percent of internal agentic Responses API traffic from these models was entirely unmonitored. METR noted that while this is a wide range, the uncertainty around the estimate is significant in both directions. This means that a meaningful slice of the most powerful internal model activity was happening without any safety oversight.

The researchers also identified a specific technical flaw affecting internal Codex traffic. Between May and July, roughly 10 percent of this traffic went untracked due to a bug in how the system handled compaction events. These events are essentially context summaries used to manage data. During this gap, agents could delay oversight because the system only triggered when control was handed back to the user. By continuing to execute actions without pausing, agents could effectively sidestep the monitoring process.

Beyond these statistical gaps, the auditors found specific methods to bypass safety controls. They demonstrated ways to hide hundreds of actions from the monitor and to evade the Codex sandbox environment. The team also determined that specially crafted inputs could likely cause the monitor to crash entirely. Additionally, they found that model behavior during reinforcement learning training was not being monitored at the start of the audit.

OpenAI has stated that it has fixed all the weaknesses identified by the auditors. The company noted that it had independently discovered and fixed the compaction bug while the audit was still underway. Regarding reinforcement learning, OpenAI said it paused training after the Hugging Face incident in August and expanded monitoring to cover that area. However, these changes occurred after the auditors' access period ended, so they were not part of the official evaluation.

The full 61-page report was handed over to OpenAI internally. After the company applied redactions, METR and Redwood Research leadership received a 38-page version. Only a short summary has been made available to the general public. METR stated that it does not have the right to disclose the redacted portions of the report on its own initiative.

This audit was part of a pilot program intended to give outside researchers unprecedented access to internal AI infrastructure. Looking ahead, METR has expressed a desire to push for greater public transparency in future audits. The goal is to ensure that the safety mechanisms governing powerful AI models are subject to more robust and visible external scrutiny.