Anthropic has done something that would have seemed unthinkable just two years ago: it has invited an outside firm to station evaluators permanently inside its own walls, with access to its models, its researchers, and its most sensitive pre-release work. The partner is Accenture — specifically its specialist AI arm, Faculty — and both companies say they each expect to spend at least $1 billion over five years to make the arrangement work.
The deal, announced September 18, is being called an “embedded evaluation” partnership. But in practice, it is something more radical: a bid to make AI safety auditing a continuous, internal function rather than a regulatory box to check after the fact.
Why Embedded?
The conventional model for AI evaluation has always been arm’s-length: a model ships, regulators or third-party testers poke at it, a report gets filed, and development moves on. That approach is built for an era when AI systems were discrete products with clear version numbers. It fits uneasily onto frontier AI labs, where models are continuously retrained, fine-tuned, and deployed in waves — and where the most dangerous capabilities may emerge unpredictably between formal release cycles.
Anthropic’s solution is to stop treating evaluation as a checkpoint and start treating it as infrastructure. Under the new structure, Accenture’s Faculty unit will place teams directly inside Anthropic. These evaluators will have access to models before they are released to the public, participating in red-teaming exercises, alignment assessments, and safeguard testing in real time. The goal is to catch problems — deceptive behavior, hidden capabilities, alignment failures — before they become headlines.
We believe that having evaluators embedded within Anthropic will lead to more rigorous, more honest assessments of what our models can and cannot do, Anthropic wrote in its announcement. External evaluation alone cannot keep pace with the speed of development.
The $1 Billion Question
The financing structure is unusual enough to deserve scrutiny. Anthropic says it will fund Accenture’s work directly — meaning the evaluator is being paid by the company it is evaluating. Both sides acknowledge the obvious tension: an evaluator who depends on the company for payment has a financial incentive to reach comfortable conclusions.
Anthropic’s answer is that the arrangement is explicitly designed to insulate Faculty’s findings from commercial pressure. The partnership agreement reportedly includes provisions requiring Anthropic to publish Faculty’s assessments even when they are unflattering, and to make those assessments available to third parties, including regulators. Details of those provisions have not been made fully public.
The $1 billion each framing, it turns out, requires some unpacking. These are not payments between the two companies — they are separate projections of what each organization expects to invest in building this kind of evaluation capacity over five years. The actual fee structure, headcount commitments, and milestone schedules have not been disclosed.
Industry analysts have noted that $1 billion over five years works out to $200 million per year — substantial, but not staggering for organizations of this size. The significance of the figure may be more about signaling commitment than about the literal dollar amount.
The Pacing Connection
The partnership does not exist in a vacuum. It is a direct response to the AI pacing debate that has consumed the industry since Anthropic CEO Dario Amodei, OpenAI’s Sam Altman, and Elon Musk appeared to converge — however briefly — on the idea that frontier AI development should be voluntarily slowed if safety assessments cannot keep up with capability gains.
Amodei has argued that the bottleneck is not technical capability but evaluation: the industry can build increasingly powerful systems faster than it can assess what those systems are actually doing. Embedded evaluation, in theory, closes that gap by putting safety infrastructure on the same development timeline as the models themselves.
President Trump rejected the pacing proposal in a Truth Social post last week, announcing instead that he would create a dedicated AI Force modeled on the Space Force. The administration’s position — that American AI labs should race ahead of international competitors without self-imposed speed limits — puts Anthropic in a peculiar position: pursuing voluntary constraints that the government has explicitly declined to mandate.
The Accenture deal is, in part, a way for Anthropic to demonstrate that a voluntary, industry-led evaluation regime can work — a proof of concept that might eventually attract regulatory interest, if not formal endorsement.
Faculty’s Track Record
Accenture acquired Faculty, the British AI consultancy, in 2023. Faculty built its reputation on applied AI work for governments and large institutions, including work for the UK Cabinet Office and National Health Service. It has experience running structured AI evaluations, though its embedded-evaluator work at the scale Anthropic envisions would be new territory.
The arrangement is not exclusive. Anthropic said it is in active discussions with METR, the research nonprofit that has conducted external evaluations of frontier models for both Anthropic and Google DeepMind, as well as with other third parties. The intent appears to be to build a network of embedded evaluators over time, with Faculty serving as the first implementation.
What Could Go Wrong
Even enthusiasts for the model acknowledge several points of failure. First, the financial dependency problem: no matter how many contractual firewalls are built, a paying client relationship creates structural pressure. Faculty’s incentives are not perfectly aligned with candor.
Second, there is the access problem running in the other direction: evaluators who are too close to a development team may become normalized to concerning behaviors, or may lack the independent perspective that makes outside evaluation valuable. The ideal evaluator sees the system fresh; embedded evaluators, over time, may stop seeing it at all.
Third, the publication commitment is untested. If Anthropic’s models receive an unflattering assessment, will the company actually publish it? The reputational and competitive costs could be enormous. There is, at present, no enforcement mechanism that compels publication — the arrangement is voluntary, and that makes the commitment meaningful only if it is tested.
A Template for the Industry?
Anthropic is betting that the embedded evaluation model — expensive, complicated, structurally awkward — is still better than the alternative: an industry that develops increasingly capable AI systems with no meaningful evaluation infrastructure at all.
If the arrangement works, it could become a template. Other frontier labs — Google DeepMind, Meta, xAI — face the same evaluation gap. A successful proof of concept at Anthropic, particularly one that produces credible, published findings, might establish that embedded evaluation is both operationally feasible and commercially survivable.
If it fails — if Faculty’s findings are buried, or its access is restricted when conclusions are uncomfortable, or if the structural conflicts prove too deep — it will have provided a cautionary lesson about the limits of voluntary safety commitments in a fiercely competitive industry.
For now, Anthropic has made the bet. The next five years will show whether the inside view was worth the price.
