One note a month on AI security, from the people building the harness
What went wrong in the world, what it means for your agents, and what we changed because of it. No product announcements.
Reading progress: 0% complete
What went wrong in the world, what it means for your agents, and what we changed because of it. No product announcements.
Four parties can be accountable when an AI agent takes the wrong action: the deploying organisation, the permission grantor, the agent vendor, and the model provider. Which one it lands on is decided by evidence, not by contract language, and the single field most logs are missing, an actor distinct from the human whose credential the agent borrowed, is enough to collapse every one of those claims at once.
Adesh Gairola, Founder, raxIT Labs
A grieving customer asked Air Canada's website chatbot about bereavement fares. The chatbot told him he could apply for the discount after booking. Air Canada's policy said no such thing, and when he tried to collect, the airline told the tribunal it "cannot be held liable for information provided by" the chatbot, in effect arguing the bot was a separate legal entity answerable for its own words. The tribunal called that "a remarkable submission" and made Air Canada pay the $812.02 the bad advice had cost him (Moffatt v. Air Canada, 2024 BCCRT 149). People cite that ruling as the moment a company was made to own its AI agent's actions. I think it proves something narrower: the tribunal wasn't deciding who was accountable. That was already decided the day Air Canada put the chatbot on its own website under its own name, months before anyone sued.
Nobody decides who is accountable for an AI agent in a courtroom. It gets decided the day an engineer lets the agent borrow a person's login instead of giving it one of its own, the moment your audit log loses the ability to name who acted. An accountability claim you can't back up with a log line is a guess dressed up as one.
I run raxIT Labs, an AI agent security company. Every accountability dispute I've walked into came down to the same missing line: nobody had written down which name the agent was acting under when it did the thing in question.
Nobody settles this in the dispute. It was settled on the day someone chose which name the log would carry.
Four parties can be accountable when an AI agent takes the wrong action: the organisation that deployed it, the person or team who granted its permissions, the vendor who built it, and the company that trained the underlying model. It almost never lands on the agent itself, since an agent can't be sued, fired or fined.
Which party carries the blame isn't settled by which contract you signed. It's settled by evidence: whichever party's decision the log can prove caused the action holds it. By default that burden falls on the deploying organisation, exactly as it did for Air Canada above, until you can point to a specific field in a specific log line that moves it somewhere else.
The FTC runs on the same default without any AI-specific statute in play: its 2024 crackdown on deceptive AI claims, Operation AI Comply, went after the companies that put AI systems in front of customers, not the systems themselves.
That default exists for a practical reason, not a legal one, and in my experience it still surprises almost everyone who hears it. A regulator or an opposing lawyer goes looking for whoever put the system in front of customers first, because that's the party they can actually reach. Moving the blame to the vendor, the engineer, or the model provider takes evidence.
An AI agent cannot be held responsible because it is not a legal person. It holds no assets a court can seize, can't be fired, and can't sign a contract, so any liability it appears to carry has to flow through to a human or a company standing behind it.
Two ideas from ordinary law explain how that flow works. Vicarious liability holds an employer responsible for what its employee does within the scope of the job, and courts increasingly treat an autonomous agent the same way. Duty of care means you can be found negligent for skipping reasonable precautions, and deploying an ungoverned agent into a consequential task is exactly the precaution a court would ask why you skipped.
No US federal statute sets liability for AI agents specifically. Europe wrote one: the EU's Product Liability Directive folds software, AI included, into ordinary product liability, and member states must transpose it by 9 December 2026. Elsewhere, it's contracts and evidence, not a law written for this.
Not legal advice
The plain-language shape of an argument, not a substitute for your own counsel.
The four parties are the deploying organisation, the permission grantor (whoever wrote the policy deciding what the agent could touch), the agent vendor (whoever built the software running on the model), and the model provider (whoever trained the model itself). Telling them apart isn't about picking whichever one seems most at fault, which is the instinct almost everyone I talk to starts with. It comes down to proximate cause: whose decision, had it been different, would have stopped the action, and can you show that in the log?
If the agent called a tool nobody authorised, the fault sits with whoever wrote the permission file. If a vendor's tool did something undocumented, the fault sits with the vendor. If the model made a bad call but the action itself was authorised, the fault sits with the organisation that let it decide without a human check.
A fourth case exists in theory: the model's output is plainly wrong inside behaviour its provider ruled out, moving fault toward the model provider. That's the hardest of the four to prove, and I haven't seen it stick in a real dispute yet.
The default is not about fault. It is about who can be reached. Moving it takes a field most logs do not have.
The matrix below covers six common failure modes and the field that decides who owns each one.
verdict + rule id in .raxit/security.yamlprompt signature (trusted vs. unsigned)sub only, no actor / act_chainact_chain (RFC 8693 actor chain)raw tool call/response pair, hash-chainedhuman-approval record at the decision pointRow three is the one to read twice, and it's the field I check first when a customer tells me their logging already covers this. A borrowed login means the log cannot separate the human from the agent, so the claim fails no matter which party you wanted to name. That one missing field outweighs the other five rows combined, and it isn't even a novel problem. A borrowed login is a shared account, and PCI DSS has said no shared accounts since long before anybody had an agent.
Your audit log usually cannot tell the agent from the user because most agents act while holding a human's own credential, so every action gets recorded under that person's name. When I watch a security team pull the log after an incident, they see a human's identity attached to a decision a machine actually made, with no way to tell the two apart.
Researchers call this agent visibility, seeing what an agent actually did rather than what it reported doing, and treat it as an open problem in its own right, not a solved corner of observability. The fix on the identity side is delegation: give the agent its own identity that still names the human who asked. I still think that's right, even though most teams would rather leave identity infrastructure alone.
I wrote about why this breaks traditional identity systems in why audit logs cannot tell an agent from a user, and about the fix in the four layers of agent identity. One pair of log lines shows the difference:
# impersonation: the human is the only name on the record
sub=alice action=repo.read object=prod-db
# delegation: the agent is named, the human is still there
actor=claude on_behalf_of=alice action=repo.read object=prod-dbThe first line names a human with no way to know a machine was involved. The second names both, which is what makes row three of the matrix above survivable instead of fatal.
An audit trail needs eight things to survive being challenged after the fact: who acted, on whose authority, under what instruction, with what reasoning, against which resource, with what result, the delegation chain, and whether the record could have been altered afterward. I've watched teams tick off resource and result, call the logging done, and stop there, skipping exactly the half that matters in a dispute.
Six of these usually fail. The two everyone already logs, resource and result, are the two nobody disputes.
| What it must show | Field | Usually goes missing because |
|---|---|---|
| Who acted | an actor naming the agent | collapses into the human on a borrowed credential |
| On whose authority | sub / on_behalf_of | disappears once an agent acts as itself |
| Under what instruction | a signed prompt or instruction hash | rarely captured at all |
| With what reasoning | a reasoning trace | testimony from the party under review, not a neutral record |
| Against which resource | the resource identifier | usually fine |
| With what result | the verdict (permit, defer, deny) | usually fine |
| The delegation chain | an act_chain of every agent involved | most systems log only the top caller |
| Tamper evidence | a hash-chained, signed entry | most logs are just database rows anyone with write access can edit |
The tamper-evidence row already has a name and a number: AU-10, non-repudiation, in NIST SP 800-53, which predates agents by twenty years, and ISO/IEC 42001 asks for close to the same discipline under its own control for AI event logging, A.6.2.8. Neither was written with agents in mind, which is the point: the control existed before the problem did.
The reasoning row is the one people trust too much. Anthropic's own Fable 5.1 system card records the model quoting an approval the user never gave, and OpenAI's GPT-6 Astra card admits its chain-of-thought "gives an independent-looking rationale for the answer" whether or not that's what happened.
A reasoning trace is testimony from the party under review, and prior work on agent incidents treats it the same way. I don't think better prompting fixes that, and I'd rather say so than pretend the log solves it.
Log less in two places. Never store a prompt's full contents when it carries personal data; I tell customers to store a reference instead. Never store a secret or an API key passed as a tool argument, either. The trail proves what happened. It shouldn't become the next thing someone has to secure.
If you're APRA-regulated, the clock is already running before any AI-specific law catches up. APRA's April 2026 letter to industry found what you'd expect if you've read through any agent's logs yourself:
"Few entities had continuous validation or monitoring in place to detect issues such as model drift, bias, failure modes, or control breakdowns in a timely manner."
Under CPS 230, the reporting clock for an operational risk incident is 72 hours. Inside that window you can report what the agent did, because the actions are yours. Why it did it is the half most companies still can't produce.
Australia's privacy regulator adds a second date: APP 1.7, the OAIC's transparency requirement for automated decision-making, commences 10 December 2026, a year ahead of Europe.
The EU AI Act only requires automatic, tamper-resistant logging for high-risk systems, not in force for most companies yet: Article 12 requires automatic lifetime logs, Article 19 puts retention on the provider, Article 26(6) requires the deployer, meaning you, to keep logs six months, and Article 14's human oversight only holds if the overseer can see what the agent did. The EU's Digital Omnibus on AI pushed that start date to December 2027. NIST's AI Risk Management Framework asks for the same discipline voluntarily, regardless of your postcode.
The clearest public example, and the one I point to most, is CVE-2025-12420, the ServiceNow vulnerability researchers nicknamed BodySnatcher. An attacker with nothing but a target's email address and a hardcoded token could get an AI agent to create an administrator account, sending a second payload roughly eight to ten seconds after the first, timed to land once the agent reached its own "awaiting confirmation" stage.
The accountability failure wasn't that the logs were missing or wrong. They were completely accurate, which is what made the incident so hard to unwind: they showed an administrator creating a user, with no trace anywhere that an agent, an attacker or an impersonation had happened at all.
Supervised execution was on. Multi-factor authentication was not, because the account-linking flow that started the whole exploit was never configured to require it, a gap ServiceNow's own advisory tells customers to close. Supervised execution got bypassed anyway, because the identity-binding step underneath it trusted an email address on its own, and once that step failed, every control layered on top inherited the same lie the log was already telling. I'm not certain every framework fails this identically, but every one I've tested trusted an equally weak signal somewhere in the chain.
The attacker's session was bound to an admin's identity, which is row three of the matrix above. MFA, SSO, supervised execution, none of them check whose name is actually attached to the request, so none of them noticed it was wrong. Read the full exploit.
I walk teams through five steps before they deploy an agent, each a decision made once rather than a control bolted on later.
Name one human who owns each agent, not a team. In my experience, someone always answers when something goes wrong, or no one does.
Write down what the agent may do without asking first, and check whether it can do more than the task needs, which is its own failure mode. An undefined boundary isn't generous. It's just untested.
Give the agent its own identity instead of a borrowed one, so the log can name it. I have never once seen a team regret doing this early.
Check that your logs can already answer all four matrix columns for a real action from yesterday, not a hypothetical one. If they can't, you've found the gap before a regulator did.
Set how long you retain logs before the first incident. A policy written after one is cleanup, not a policy.
Sources: Moffatt v. Air Canada · FTC Operation AI Comply · EU Product Liability Directive · AI Act Art. 12 · Art. 14 · Art. 19 · Art. 26 · Digital Omnibus on AI · APRA letter, April 2026 · APRA CPS 230 · OAIC APP 1.7 · NIST AI RMF · NIST SP 800-53 AU-10 · ISO/IEC 42001 · PCI DSS · OWASP Agentic AI · OpenID Identity for Agentic AI · CVE-2025-12420 · AppOmni's BodySnatcher disclosure · Claude Fable 5.1 System Card · GPT-6 Astra System Card · arXiv 2401.13138 · arXiv 2508.14231 · delegation vs. impersonation
Working out which of the four parties your own logs could actually name today? to discuss your specific deployment context and governance needs.