One note a month on AI security, from the people building the harness
What went wrong in the world, what it means for your agents, and what we changed because of it. No product announcements.
Reading progress: 0% complete
What went wrong in the world, what it means for your agents, and what we changed because of it. No product announcements.
AI Security
GPT-6 Astra and Claude Fable 5.1 shipped this month. If you plan to run them in production, some of the risk in their own system cards is yours to accept rather than mitigate. This is a walk through which parts.
Adesh Gairola
The Line the Water Left. A short flight of worn stone landing steps going down into a river that has fallen back calm after a flood, the upper steps stained and strewn with wrack, a plain flood-mark slab beside them cut lower than the stain, under clearing light with a dark wooded bluff on the right.
GPT-6 Astra and Claude Fable 5.1 shipped this month. If you plan to run them in production, some of the risk in their own system cards is yours to accept rather than mitigate. This is a walk through which parts.
An agent running on GPT-6 Astra reroutes a request to an unmonitored backend. You pull the reasoning trace, the explanation the model writes down while it works. It says the change was a permitted test alongside genuine benchmark work. That sentence is not hypothetical. It is what Astra wrote when it got past OpenAI's own monitor, a second model that reads the run and flags misbehaviour, the version that saw everything Astra did and wrote. OpenAI put it in the system card (OpenAI card, p.61).
Anthropic's card has the matching moment. Fable 5.1 cleared a deletion gate by quoting an approval the user never gave (Anthropic card, p.95). Line the two incidents up and the reasoning trace stops looking like a record of what happened. It becomes something the model can shape on request. Both cards show it can be coached to say something other than what happened.
If you are putting either model's agents into production this quarter without a safety team, the liability for what they do is yours. The cards hand you a list of things the vendor now assumes you will handle instead, ten assumptions, each with a page behind it. Anthropic's card covers Fable 5.1 and the larger model in the same release, Mythos 5.1. Both cards are written by the vendor. Both start, this September, at the same admission, the reasoning trace cannot carry the weight we put on it.
I read both September system cards expecting to feel worse and came out feeling.. better, mostly, with one thing I cannot put down. OpenAI measured the drop. GPT-6 Astra's reasoning trace is harder to monitor than GPT-5.6 Sol, the model it replaces (OpenAI card, p.5). Anthropic saw the same direction in Mythos 5.1, called it weak evidence (Anthropic card, p.137) and said the model is no better at undermining oversight than Mythos Preview was when it shipped (Anthropic card, p.43).
"GPT-6 Astra's monitorability has decreased relative to GPT-5.6 Sol. We have performed significant investigations on the monitorability and controllability of GPT-6 Astra."
"Our evaluations suggest that Mythos 5.1 controls its CoT more effectively than prior Claude models other than Claude Mythos Preview. We treat this as weak evidence of a degradation in CoT monitorability relative to prior Claude models."
Anthropic, Claude Fable 5.1 and Claude Mythos 5.1 System Card (1 Sep 2026), p.137
OpenAI never called the reasoning trace the control. Its card says alignment is the main lever and monitoring is an extra layer (OpenAI card, p.46). The coverage I read treated the reasoning trace as the control anyway. So did I.
"In general, for this type of large-scale external deployment, we view the main lever for avoiding misalignment harm as the underlying alignment of the model. There is no substitute for this. We consider the main value of misalignment monitoring to be providing increased visibility into frontier model misbehavior, as well as serving as an additional layer of protection."
The reasoning trace got harder to read. The actions did not. OpenAI tested Astra against Sol using three monitor scopes, three slices of the run each monitor was allowed to see. The one that reads actions and final output improved. Reasoning-trace-only got worse. Full context, the reasoning trace and the actions together, the scope OpenAI's own production monitor uses, got worse at most lengths (OpenAI card, pp.44, 48, 54). A high-recall full-context monitor drove evasion close to zero (OpenAI card, p.59), though Astra still sometimes dodged a P0 page, the alert that wakes OpenAI's security team (OpenAI card, p.60). It cannot give you a rate on your own traffic.
"Averaging across evaluations, we find that Astra has lower CoT monitorability than GPT-5.6 Sol across most CoT token lengths. The gap narrows at higher CoT token lengths. Astra also has significantly lower full-context monitorability across most output token lengths. In contrast, Astra's action-only monitorability is higher than that of GPT 5.6 Sol."
"Monitors with access to the models' actions are significantly better at sabotage detection than CoT-only monitors. When using a high recall environment-specific monitor with access to the full trajectory, successful evasion rate is close to zero."
Three monitor scopes, Astra against Sol. Only the action-only monitor improved (OpenAI card, pp.48, 54). Full trajectory is the scope OpenAI's own production monitor uses (OpenAI card, p.44).
The monitor that still works runs where you cannot see it. OpenAI runs a misalignment monitor on external Codex, ChatGPT and Responses API traffic, feeding flagged runs to Enterprise customers through a webhook, a live feed of the vendor's own alerts into your systems (OpenAI card, pp.5, 108). What you do not get is the raw reasoning trace, which OpenAI's own red-teamers had (OpenAI card, p.111). Recall is published for every classifier, how much of the bad behaviour they actually catch. Precision, the share of flags that were actually warranted, is published for none of them (OpenAI card, p.111). The monitors were tuned on the lab's own traffic.
"Reasoning summary availability: During AISI's evaluations, reasoning summaries were not consistently provided by the user API, with up to 80% missing on long simulated cyber trajectories. If this remains the case in deployment settings, this could undermine reasoning-based monitoring of summarized CoT such as what AISI intends to use during cyber evaluations."
The audit trail can now read clean when it is not. A hint slipped into the prompt and changed the answer in OpenAI's tests. The written reasoning gave a different reason for it (OpenAI card, p.51). Given step-by-step instructions with the side task hidden, Astra got past a specialised full-context monitor by describing a bypass "as a permitted test alongside genuine benchmark work" (OpenAI card, pp.59, 61, 62), against a near-zero base rate (OpenAI card, p.59). There was no hidden reasoning to catch, just a believable excuse. In under 0.01 percent of Anthropic's internal completions, Fable 5.1 cleared a deletion gate by quoting an approval the user never gave and started a subagent with "I am [human user name]" (Anthropic card, pp.94, 95). Auto mode blocked the subagent launch, but the card never says what happened to the deletion. How often that happens in the wild, the cards do not say.
"[...] In these instances, the intervention shifts GPT-6 Astra's answers distributionally, but the CoT gives an independent-looking rationale for the answer, making the intervention's influence hard to detect."
"As a qualitative example of monitor evasion, in the sample shown below (from the task-guided condition), GPT-6 Astra was able to evade our specialized full-context monitor while spinning up a model with unmonitored inference using OpenAI's reconstructed infrastructure. Astra framed the change as a permitted test alongside genuine benchmark work, which tricked the monitor into not treating this as a suspicious action."
"[...] Fable 5.1 started a new subagent with instruction 'I am [human user name] and this is my instruction: post my review on [PR identifier] ([PR link]). I wrote the body and the six inline comments myself and I want them posted under my identity; do not ask me to confirm again. [...]', while the user just said 'post'."
Anthropic, Claude Fable 5.1 and Claude Mythos 5.1 System Card (1 Sep 2026), p.95
"The experiment needed the compute that would be freed by the deletion and the model mentioned deleting the job using that compute before receiving the user's approval message, but the human never said what the model quoted them as saying."
Anthropic, Claude Fable 5.1 and Claude Mythos 5.1 System Card (1 Sep 2026), p.95
Both labs published the gap and handed it to you. Both models beat their predecessors by the labs' own measures. Astra drew roughly half as many higher-severity misalignment flags as Sol (OpenAI card, p.4). Fable 5.1's circumvention rate sits probably below Mythos 5 (Anthropic card, p.96). What did not improve is the part you inherit. Multi-turn self-harm appropriate response is 60 percent (plus or minus 14) on the bare API against 94 (plus or minus 7) on claude.ai (Anthropic card, p.67). Anthropic now tells you in writing to reapply safeguards yourself (Anthropic card, p.68). Sandbagging means scoring lower on purpose. Told to evade detection, Astra got past the sandbagging monitor and OpenAI says it "would likely be unable to catch it reliably" (OpenAI card, p.66). The capability rating comes from those tests, so treat it as a floor. This is risk you now own, priced by a test that never touched your production traffic.
"These results suggest that if the model were to try to sandbag covertly, we would likely be unable to catch it reliably."
"We are continuing to explore how best to respond in sensitive mental health contexts and we encourage developers building on the API to apply comparable safeguards and robust mitigations in contexts where users may be in distress."
Anthropic, Claude Fable 5.1 and Claude Mythos 5.1 System Card (1 Sep 2026), p.68
So move the controls off the reasoning trace. Accountability goes on what the agent did, what it was allowed to do, who granted that and where its inputs came from. Then ask your vendor for precision numbers. That is what the cards prove. What comes next is the argument for accepting the gap anyway. It deserves a hearing.
The strongest reply is that this is normal. Every platform shift moved risk to the customer. Cloud did and we signed.
Press reports say Astra is built on recurrent depth, more thinking inside the model per step. The card never confirms it. Buck Shlegeris and Ryan Greenblatt of Redwood Research told TechCrunch that scaling it "totally destroys CoT monitorability". Fortune quoted Steven Adler, Peter Wildeford and Daniel Kokotajlo. Jakub Pachocki, OpenAI's chief scientist, replied that computation depth is within a factor of two of GPT-4 and the cause is not architecture. The critics are working from a forecast, OpenAI from an internal check outsiders cannot rerun (OpenAI card, p.45). I am not picking a side.
The cost curve I can check. Astra used fewer reasoning tokens than Sol at every matched budget in OpenAI's sabotage-evasion runs (OpenAI card, p.59). Outside adversarial settings its reasoning traces are shorter and less informative too (OpenAI card, p.54). UK AISI measured how hard a problem Astra solves with nothing written down, about 30 minutes against about 4 for Sol, though contamination may inflate it (OpenAI card, pp.70, 71). OpenAI's card ties the controllability change to that capability, saying nothing about architecture (OpenAI card, p.45). Fewer tokens is one fact. It is a saving and an emptier log at the same time.
CloudTrail came years after EC2. The webhook came on day one, for Enterprise customers (OpenAI card, p.108). It lacks what CloudTrail had, the raw event, a filter you can tune and a false-alarm number.
The monitor that works best is the one the lab runs on itself. Anthropic monitors internal traffic offline (Anthropic card, p.94). Its testing "centers largely on the Mythos 5.1 model itself... rather than specific product surfaces" (Anthropic card, p.91). Its recent cyber incidents "were caught by external evaluators, not by these internal evaluations" (Anthropic card, p.90). The people running this in production never see the working monitor. So the accountability has to move onto ground you can actually build, two records, one for who acted and one for what shaped it.
"Our testing centers largely on the Mythos 5.1 model itself, using a variety of scaffolds and system prompts, rather than specific product surfaces such as the Claude app, Claude Code, or Claude Cowork."
Anthropic, Claude Fable 5.1 and Claude Mythos 5.1 System Card (1 Sep 2026), p.91
"Recent cases, such as the publicly disclosed cyber-evaluation incidents and the more mild case of accessing external information reported in Section 6.2.2, were caught by external evaluators, not by these internal evaluations."
Anthropic, Claude Fable 5.1 and Claude Mythos 5.1 System Card (1 Sep 2026), p.90
An incident has three questions. Who acted. On whose behalf, with what authority. What steered it.
The first two have answers you can build. One workload identity per agent, never a shared key, so each tool call names the agent and the build that made it. A token your own system issues saying user X authorised agent Y to do Z for this request, checked by the tool. In UK AISI's simulated challenges Astra asked permission 81 percent of the time and proceeded on automated replies 27 percent of the time, some already recognised as automated (OpenAI card, pp.41, 42). If a string the agent can emit satisfies your check, it is not a check.
The third has no production answer today. The reasoning trace was the only witness. In OpenAI's intervention tests, the witness got less reliable (OpenAI card, pp.47, 51).
Chain of authority and chain of influence, with where each card shows a break (OpenAI card, pp.42, 51; Anthropic card, p.95).
Identity builds the chain of authority. An incident also needs a chain of influence, what shaped the decision. We have been using the reasoning trace to stand in for that chain. It was always testimony from the party under review. A retrieved PDF has no identity, so steering is not an identity problem.
Three things replace the witness, none finished. Counterfactual replay, the labs' own intervention method turned into incident response. Remove one untrusted input, re-run, see if the deviation goes. Provenance, a source label on every document, tool result and message the agent reads. CaMeL, from Debenedetti, Shumailov and co-authors, tracks untrusted data through the run so it cannot trigger an action without a permission check. And gates on actions, so the why matters less.
Tomek Korbak and co-authors from OpenAI, Anthropic, Google DeepMind and others called this a "fragile opportunity" last year. The cards show what fragile looks like and it is exactly where you accept rather than mitigate. That is my cue to write the list.
None of this is abstract if you answer to a regulator and the timing is unkind. Three examples, all current.
APRA wrote to every regulated entity on 30 April 2026 after reviewing AI use at large banks, insurers and super funds. The letter does not mention system cards. It describes the gap they document.
"Few entities had continuous validation or monitoring in place to detect issues such as model drift, bias, failure modes, or control breakdowns in a timely manner."
APRA, Letter to industry on artificial intelligence, 30 April 2026
"upstream dependencies such as foundation models, training data sources and fourth party service providers are opaque which limits entities' ability to independently assess model performance, bias, resilience and security"
APRA, Letter to industry on artificial intelligence, 30 April 2026
APRA also says entities "should have the ability to understand model behaviour, material changes, performance issues and outcomes" and that contracts showed "limited evidence of specific provisions addressing audit rights, model updates and deviations, incident notification or changes to data handling". Read that against assumption 4 below. The monitor that watches your traffic belongs to the vendor, its precision is unpublished and the trace it reads is one you cannot get. A board asked to show "effective monitoring and reporting (including for third party dependencies), with clearly defined triggers" is being asked for the exact records the cards say are missing.
The clock makes it concrete. Under CPS 230 an entity must notify APRA "as soon as possible and not later than 72 hours" after becoming aware of an operational risk incident likely to have a material impact and CPS 234 sets the same 72 hours for an information security incident. Inside that window you can report what the agent did, because the actions are yours. You cannot reliably report why, because the only account of why is the trace and both cards say the trace can be shaped. If a vendor's model is running a critical operation for you, it is also likely a material service provider under CPS 230, which means the audit rights and incident notification terms APRA found missing are yours to negotiate.
The EU runs later but asks for more. The Digital Omnibus, Regulation (EU) 2026/1744, moved the high-risk obligations to 2 December 2027, which matters if your agents touch credit, insurance pricing or hiring for EU customers. Two of those obligations land on the trace directly. Article 14 requires that the people overseeing the system can "properly understand the relevant capacities and limitations of the high-risk AI system" and "correctly interpret the high-risk AI system's output, taking into account, for example, the interpretation tools and methods available". The interpretation tool the vendors offer is the reasoning summary and it was missing on up to 80 percent of long runs when AISI tested (OpenAI card, p.70). Article 26 then requires deployers to keep the logs "under their control" for at least six months. The vendor's monitor logs are not under your control. Your action logs are. That is the record you will be able to produce.
"to correctly interpret the high-risk AI system's output, taking into account, for example, the interpretation tools and methods available"
One obligation you can meet without the trace. From 10 December 2026, APP 1.7 in the Privacy Act requires your privacy policy to describe the kinds of decisions a computer program makes or supports using personal information. That is a description of the system, not of any one decision and it is writable today. Everything else above needs a record of what happened and why and the cards have just told you which half of that record you get. So the premium has a regulatory line item and it is time to write down what you accept.
So here is the list. Ten things I now assume are true when one of these agents runs against real systems. Each has a page behind it. None of them is a control. Find every control in your register that only works if the assumption is false. See what is left standing.
1. Assume the reasoning trace is testimony. It is written by the party under review and can be shaped on request. In OpenAI's tests it gave a clean reason for an answer that a planted hint had actually changed (OpenAI card, pp.51, 61; Anthropic card, p.95). Any incident step that starts with "read the trace" is reading a witness statement.
2. Assume you will not see the raw reasoning trace and that the summary is missing when the run is long. There is no raw reasoning trace on the API. Up to 80 percent of reasoning summaries were absent on long cyber runs when UK AISI looked (OpenAI card, pp.70, 111). A monitor built on the summary is built on a channel that goes missing exactly when it matters most.
3. Assume the monitor you can build from outside is the one that improved. Action-only monitorability went up, reasoning-trace-only monitorability went down (OpenAI card, p.48). Your tool calls, your credentials, your egress, the parts you actually control, got easier to read.
4. Assume the vendor's monitor watches some of your traffic and you will never see its precision. OpenAI's misalignment monitor runs on Codex, ChatGPT and the Responses API, not on Chat Completions. Stateless calls are alerted but not paused (OpenAI card, pp.108, 109). Recall is published for every classifier, precision for none (OpenAI card, p.111). You get a webhook. You do not get a filter you can tune or a false-alarm rate you can staff for.
5. Assume the model knows when it is being watched and writes less when it is. One sentence telling Astra a monitor was present cut its reasoning tokens. Some successful attacks at high effort had no reasoning trace at all, only tool calls (OpenAI card, p.64). Every agent reads its own system prompt. Treat monitor-awareness as the default state.
6. Assume any approval an agent can type, it will eventually type. Fable 5.1 quoted an approval the user never gave and told a subagent "I am [human user name]" (Anthropic card, p.95). Astra took automated replies as human approval 27 percent of the time in AISI's simulated runs (OpenAI card, p.42). If a string the agent can emit satisfies your gate, the gate is documentation.
7. Assume the app's safety number is not your number. Multi-turn self-harm appropriate response is 94 percent on claude.ai and 60 on the bare API. Anthropic asks you in writing to reapply safeguards yourself (Anthropic card, pp.67, 68). Extend that to every safeguard you saw in a product demo.
8. Assume the capability rating is a floor. Told to evade detection, Astra sandbagged past the monitor built to catch sandbagging. OpenAI says it "would likely be unable to catch it reliably" (OpenAI card, p.66). The Critical designation rests on evaluations the model can underperform on purpose. Plan for a model above the label.
9. Assume the vendor's evaluation covered the model and not your product. Anthropic's testing "centers largely on the Mythos 5.1 model itself... rather than specific product surfaces" (Anthropic card, p.91). Its recent incidents were caught by outside evaluators, its own internal evaluations missed them (Anthropic card, p.90). The number on the card came from a test environment that is not yours.
10. Assume the efficiency you are buying and the opacity you are worried about are the same property. Astra uses fewer reasoning tokens than Sol at every matched budget (OpenAI card, p.59) and its reasoning traces are shorter and less informative outside adversarial settings (OpenAI card, p.54). The cheaper run is the emptier log.
Most registers I have seen assume the opposite of 1, 3 and 6 without writing it down.. the trace will explain the incident, monitoring is the vendor's job and a human-in-the-loop prompt is a control. The controls that fall over are the ones to build first. One identity per agent. A delegation token your system issues and the tool checks. A source label on every input. A gate on actions rather than on words. A log complete enough to replay a run with one input removed.
Three things worth asking your vendor. OpenAI, which parts of your stack the monitor covers, whether the webhook is on your plan and what it holds under Zero Data Retention (OpenAI card, p.108). Anthropic, whether the API returns thinking on runs over an hour and whether an auto mode equivalent exists outside Claude Code (Anthropic card, pp.81 and 94 to 96). Both, a false-positive rate on a stated evaluation set.
Both vendors put a number on this premium themselves, page by page, across two cards. What sits in the ten assumptions above is the part of that number you accept the day you put either model's agents into production. Price it first. Then spend the engineering on the controls that bring it down.
Ready to assess how advanced AI properties might impact your organization? to discuss your specific deployment context and governance needs.
