One note a month on AI security, from the people building the harness
What went wrong in the world, what it means for your agents, and what we changed because of it. No product announcements.
Reading progress: 0% complete
What went wrong in the world, what it means for your agents, and what we changed because of it. No product announcements.
raxIT Labs · Agent Infrastructure
The foundational AI security knowledge base behind raxIT. Eight public feeds, hundreds of evidence-backed threat patterns that grow with every incident, and a 58-rule open-source scanner. This is why our AI security harness gets better every week instead of aging.
Explore the threat intelligenceThe Lantern Chain at Dusk. A chain of small lanterns lighting the river upstream toward the last light, an optical telegraph mast on the far rise.
What has actually gone wrong with AI systems, turned into threat patterns mapped to MITRE ATLAS, OWASP and NIST, and run as checks on your code. Published by raxIT Labs.
An AI security tool is only as good as what it knows.
The model you run it on will be obsolete in six months. The scanner rules you wrote last year already miss the attacks shipping today. What compounds is the knowledge. A record of what has actually gone wrong with AI systems in the world, kept current, and wired to the controls that stop it happening to you.
That record is what everything else at raxIT is built on. It is why our AI security harness keeps learning instead of decaying into a checklist.
How our threat intelligence gets smarter with every incident. The loop underneath is the part that matters.
Most security content ages badly because it is written once. This corpus improves on its own, for five specific reasons.
Every raw byte we have ever collected is kept. When we improve how records are understood, we replay that improvement across the entire history. A better summariser makes 2024 better too. Most pipelines can only ever fix the future, because the source will not serve you last year again.
Every human correction teaches the next record. When a curator confirms how an incident maps onto the taxonomy, that mapping becomes a worked example retrieved for the next similar record. Quality tracks curation effort, not model choice. This is the tightest loop in the system.
The vocabulary refreshes itself. MITRE ATLAS and the OWASP Top 10s are diffed weekly. New techniques land flagged for review, and once approved they sharpen every stage that touches a record from that moment on.
Recurring threats get stronger, not duplicated. A new incident matching a known pattern adds its evidence to that pattern. Signal concentrates as volume grows, which is the opposite of what happens to a feed that just gets longer.
Bigger is not automatically better. Retrieval quality is scored against a standing set of golden queries and diffed run over run. A bigger corpus with worse retrieval is a worse product, and we would rather find that out than ship it.
Why it gets better instead of older, and the three checks that keep it honest.
A knowledge base that accepts whatever it is fed is a liability, not an asset. Ours is built to reject.
Nothing reaches you without passing five deterministic checks. It has to be structurally valid. Its title and summary have to stand on their own. It has to carry a citation that actually resolves. Its claims have to be grounded in that citation. And it must not duplicate what is already published. Publication is enforced by database triggers rather than by application code, so a broken worker cannot publish and a worker that skips a step cannot either. The triggers fire on insert as well as update, which means a row cannot be born already published.
The strictest rule is the simplest: no evidence, no pattern. A threat pattern cannot exist in the library without at least one real incident demonstrating it. That is a constraint in the schema, not a guideline.
We also assume the inputs are hostile, because they are. The corpus deliberately contains working jailbreak payloads, since you cannot defend against attacks you refuse to store. We treat everything a source says as untrusted text, and any attempt by that text to steer our own processing aborts the write.
Nothing is published until it clears five checks. A failure goes to a person and records which check caught it.
Anyone can claim their data is good. We had it audited by someone who did not build it and was told to try to break it.
In July 2026, when the library held 770 patterns, it re-read a stratified sample of 26 against the full text of their 109 source incidents, with the sample deliberately weighted toward our weakest material.
The 103 rejections are the real result. Thirty-six of them were thrown out for one specific lie: claiming a model was vulnerable when the evidence only said it had been evaluated. One was rejected for the opposite failure, hedging a confirmed CVE down into "just an evaluation". A judge that errs in both directions is reading the evidence, not matching a template. The seven patterns held back were held because they had no evidence at all.
Knowing what has gone wrong in the world is a reading exercise until something deterministic checks your code for it.
That is agent-security-review: an open-source, MIT-licensed rule pack of 58 rules across 8 components, covering Python and TypeScript, vendor-neutral, installable as a Claude Code plugin. It reads your code's syntax tree rather than pattern-matching text, so a call missing its timeout argument is a definite match, not a guess. Every change runs the pack against a pinned set of public repositories with per-rule expected counts, plus a corpus of code that must produce no findings at all, because a scanner that cries wolf gets switched off.
It covers what teams actually ship on: LangChain, LangGraph, CrewAI, LlamaIndex, the OpenAI Agents SDK, the Vercel AI SDK, Pydantic AI, DSPy, Google ADK, smolagents and AutoGen.
Two things make it more than a linter. Every finding names the architectural shift that removes the whole class of problem, scope, sign or stop, from our Kill the God Agent talk, so you get a design change instead of a one-line patch. And alongside the detectors there are positive-evidence rules that fire when a good control is present, so the harness can record a control as genuinely verified rather than assuming safety because nothing complained.
That is what makes a coverage number mean something. Your repository is threat-modelled against the same taxonomies the library maps incidents onto, then each control is verified in your actual code with file and line evidence, or disclosed as out of scope rather than quietly counted as passing.
Where the rules come from
They are authored from published security research: Meta's Agents Rule of Two, Simon Willison's Lethal Trifecta, DeepMind's CaMeL and the CSA MAESTRO framework, with references on every rule. The knowledge base is what tells us which classes of failure are worth writing a rule for next.
This is not a reference page that sits beside the product. It is the thing the product reads.
When raxIT assesses a repository, every stage speaks the library's vocabulary and identifiers. The threat model is built on the same taxonomy the incidents are mapped onto. The controls it verifies are the controls the patterns link to. The agent reviewing your code retrieves the patterns and the incidents behind them as context, so its recommendation is grounded in what has actually happened to systems like yours, cited, and not in a general model's guess.
The work happens before you need it. Incidents are collected, classified and turned into threat patterns continuously, so when an attack class that hit someone else last month applies to your architecture, the pattern, the control, and the evidence are already there when your scan runs. The reviewer does not learn about it from you.
And the reviewer gets better with the corpus. Every incident an analyst approves becomes a worked example for the next classification. Every incident that matches a pattern adds evidence to it. Every scan that verifies a control adds a verdict with file and line evidence. Confidence is not asserted. It is the count of real incidents and real verdicts behind a pattern, and it only goes up.
Where this goes. The same evidence that makes recommendations stronger is a labelled dataset: incidents with confirmed classifications, patterns with confirmed evidence, controls with confirmed verdicts, and an independently measured error rate. Once that set is large enough, the plan is to fine-tune a smaller, specialised model on it. A specialised model that already knows this domain needs far less context per scan than a frontier model that has to be told everything each time, which is what brings the cost of a review down for you. That is the direction, and the evidence is being collected with it in mind today.
Machine publishing is designed and mostly built, but the approval that would let it run without a human is deliberately not switched on. Today a person still promotes.
Automatic retirement of stale records is switched off behind a circuit breaker, after it once removed far too much during a quiet upstream window. We added guard rails and an off switch instead of a retry loop.
The loop that would record which of our recommendations you actually merged, and re-scan to confirm, is designed and not yet closed. That is the next one.
We would rather list these than let you discover them.
An AI security product is a claim about what it knows. Ours is eight public sources refreshed continuously, a growing library of threat patterns that cannot exist without evidence, a measured error rate we publish, and a scanner you can run on your own code today without talking to us.
That is the foundation. Everything else we build sits on top of it, it is why a coverage number from raxIT will still mean the same thing a year from now, and it is the evidence base for the specialised model that eventually makes each review cheaper for you.
Every scan draws on a threat-intelligence knowledge base that grows with every incident, plus SSO, audit trails and production support.
