All walkthroughs · Paper briefs

Evaluating AI Providers' Frontier Safety Frameworks

Lily Stelling, Malcolm Murray, Bruno Galizzi, Max Schaffelder, Siméon Campos, Henry Papadatos (SaferAI)
arXiv 2512.01166, version 5 (30 April 2026); first posted 1 December 2025

Paper · Brief and rating · Dashboard entry

About 22 minutes · Download MP3

AI-generated walkthrough with synthesised voices: a host explains the paper and a discussant questions it. It covers claims, methods, key assumptions, citations and use so far, decision relevance, and what an evaluator should check. Every direct quotation was checked word for word against its source. Public sources only. Not an Unjournal evaluation or decision.

Transcript

Host
This is a walkthrough of Evaluating AI Providers' Frontier Safety Frameworks, by Lily Stelling, Malcolm Murray, Bruno Galizzi, Max Schaffelder, Siméon Campos and Henry Papadatos, all at SaferAI, a French nonprofit that rates AI companies on risk management. It was first posted to arXiv in December twenty twenty-five; we are using version five, dated the thirtieth of April twenty twenty-six. This is an AI-generated walkthrough, not an Unjournal evaluation or decision.
Discussant
Start with the object being studied. What is a frontier safety framework?
Host
After the AI Seoul Summit in May twenty twenty-four, a group of developers of the most capable models, which the paper calls Providers, promised to publish policies saying how they would identify, assess and mitigate catastrophic risks from their systems. The typical structure is an if-then rule: if a model reaches some dangerous capability, for example meaningful help to someone trying to build a biological weapon, then the company applies stronger safeguards, or pauses. Twelve companies have published one. The paper's opening question is direct. The authors write: "But what do the Frameworks actually commit each company to do?"
Discussant
Why should anyone outside the AI safety community care what these documents say?
Host
Because regulation now leans on them. Under the European Union's AI Act, providers of general-purpose models with systemic risk can show compliance by signing a Code of Practice, which requires them to submit a framework meeting stated criteria to the EU AI Office. California's Senate Bill fifty-three requires large frontier developers to publish a framework, and allows fines of up to one million dollars per violation of the company's own framework. So the documents have turned into the thing regulators audit against. If they are vague, there is little to audit.
Discussant
And the headline result?
Host
The twelve frameworks are scored against sixty-five weighted criteria in four dimensions: risk identification, risk analysis and evaluation, risk treatment, and risk governance. Overall scores run from thirty-four percent for Anthropic and thirty-three for OpenAI down to eight percent for Cohere. The median is eighteen percent. Eight of the twelve score twenty percent or below, and no company exceeds fifty percent on any dimension. The second headline is what the authors call the peer ceiling: take the best score any company achieved on each criterion and add them up with the same weights. That gives fifty-four percent, about three times the median. Their reading is that much better commitments already exist somewhere in the industry, and no single company has adopted all of them.
Discussant
Where do the criteria come from?
Host
From an earlier SaferAI paper, Campos and coauthors, twenty twenty-five, which proposes a risk management framework for frontier AI built from practice in aviation, nuclear power and banking. Several authors overlap: Campos, Papadatos and Murray are on both. The criteria translate that framework into checklists. A few terms recur. A Key Risk Indicator, or KRI, is a measurable proxy for risk, usually a model's score on a dangerous-capability evaluation. A Key Control Indicator, or KCI, is a measurable proxy for how well a mitigation works, such as the success rate of jailbreaks, meaning prompts designed to get a model to ignore its safety training. The ideal framework pairs them: if the risk indicator crosses a threshold, the control indicator must meet a matching threshold, and the company should show, through risk modelling, that the pair keeps residual risk under a stated risk tolerance. If the control cannot be met, development or deployment stops.
Discussant
So the benchmark is close to what a regulator in a mature hazardous industry would expect. Do the authors claim companies should meet it today?
Host
No, and they are explicit. The authors write: "These criteria represent an aspirational benchmark rather than a description of current or achievable industry practice." Low scores are partly by construction. The point is to measure distance from what stakeholders might expect a high-risk industry to provide.
Discussant
Walk through the scoring itself.
Host
Each criterion gets one of seven levels: zero, ten, twenty-five, fifty, seventy-five, ninety or one hundred percent. The steps are finer at the ends and coarser in the middle, on the argument that telling absence from a passing mention matters more than distinguishing moderate from good. Firm language scores higher than discretionary language, so we will do something counts for more than we may consider. The scope is the framework document only, not model cards or research papers. Their reasons are that frameworks are the formal ongoing commitment and that larger firms publish more.
Discussant
And how are the sixty-five scores turned into one number?
Host
By nested weights, and this is key equation one in the transcript. The intuition is that every criterion carries a share of the total equal to the product of the weights on the path down to it. Take one criterion, whether the framework defines a risk tolerance at all. It sits in the risk analysis dimension, which carries a quarter of the total score. Within that dimension, setting a risk tolerance carries thirty-five percent. Within that, defining the tolerance carries eighty percent. Multiply a quarter by thirty-five percent, then by eighty percent, and you get seven percent. So that single criterion is worth seven points of the overall score. A company's overall score is the sum, across all criteria, of each criterion's share times the company's score on it.
Discussant
Let me check that. A company scoring fifty percent on defining a risk tolerance gains three and a half points overall from it, and a company scoring zero gains nothing. And the four dimensions each get a quarter regardless of how many criteria sit inside them.
Host
Yes on both. The dimension weights are equal by design. The lower weights, such as eighty percent for defining a tolerance versus twenty percent for the process of setting it, are the authors' judgment of importance. There is no elicitation from outside experts and no sensitivity analysis on the weights in the paper.
Discussant
Then the peer ceiling. How is that built?
Host
That is key equation two. The intuition is a composite company that copies the best existing practice on every item. For each criterion, take the maximum score across the twelve companies. Then aggregate those maxima using the same weights as before. The result is fifty-four percent overall, and by dimension it is seventy-five percent for governance, fifty-four for treatment, forty-seven for identification and thirty-nine for risk analysis.
Discussant
A maximum over twelve noisy measurements is biased upward. If any rater is generous on one company on one item, the ceiling absorbs it. And it assumes the best item from Amazon's security section can be bolted onto the best item from G42's governance section without conflict. Is that discussed?
Host
Partly. They say it is unclear whether practices that work for one provider would transfer to another, and that commitments may not become practice. They do not discuss the statistical point that the maximum inflates with measurement error and with the number of companies. The comparison of a sum of maxima with a median of sums makes the gap look larger than any one company's realistic room to improve.
Discussant
What carries the qualitative argument, beyond the totals?
Host
Five patterns. First, the gap between median and ceiling is largest in governance, fifty-five percentage points, and smallest in risk analysis, twenty-two. The authors read this as two different problems. They write: "If this interpretation is correct, the risk governance gap may reflect an adoption problem (mature practices exist but go unadopted), while the risk analysis gap could reflect an innovation problem (mature practices may not yet exist for anyone to adopt)."
Discussant
Give me the concrete gaps in risk analysis.
Host
No company states a risk tolerance as probability times severity. OpenAI defines severe harm but not what probability of it would be acceptable. Quantitative capability thresholds are rare: eight of twelve score ten percent or below. Most thresholds use phrases like meaningful uplift without saying what counts as meaningful. No company scores above ten percent on quantitative deployment controls; nobody says, for example, that the jailbreak success rate must stay below a stated level. And no company tries to show ex ante that its pairs of thresholds keep residual risk below tolerance.
Discussant
And the other dimensions?
Host
In risk identification, almost every company scores zero on processes for finding risks outside the predefined categories, which are usually chemical, biological, radiological and nuclear weapons, cyberoffense, and sometimes manipulation or autonomous AI research. Only Meta and xAI publish concrete threat scenarios or causal pathways. In risk treatment, the weakest area is what the paper calls assurance processes, meaning affirmative evidence of safety for a model that does have dangerous capabilities, for example interpretability tools that would detect deception. The median there is six percent. Companies describe research directions rather than mitigations. All twelve score zero on committing to let external evaluators publish findings without interference. In governance, eight companies have no executive risk officer, only G42 has a management-level risk committee, and seven score zero on internal audit.
Discussant
You said five patterns. The other two?
Host
Weakening over time and discretion. Two companies revised their frameworks during the assessment. xAI moved from would to may in several places and dropped a commitment to external red-team testing of safeguards. Google DeepMind's version three replaced a specific list of monitoring inputs with generic post-market monitoring, replaced named governance councils with an appropriate governance function, and removed a numerical example from its threshold for accelerating machine learning research. Neither came with a change log or rationale. The discretion point is that pause decisions use phrases like as appropriate, and final authority sits with the chief executive or leadership. Several frameworks also measure risk relative to competitors. Anthropic's policy says that if another developer has the capability without equivalent safeguards, "the incremental increase in risk attributable to us would be small" and it might lower its required safeguards. The authors comment: "These relative standards create inter-dependencies: if one Provider lowers standards, the baseline against which others measure themselves also lowers."
Discussant
To an economist that is a familiar coordination problem. Do they take the other side seriously?
Host
They cite the Frontier Model Forum, an industry body, on the point that static thresholds could disadvantage a developer whose competitors proceed with weaker safeguards, and they say coordination mechanisms may be needed. There is no model of the race or of which commitments are credible.
Discussant
What do they conclude frameworks are for?
Host
They distinguish two functions: internal learning about how to manage a new kind of risk, and external accountability, meaning specific commitments that regulators, investors and the public can check. Discretion, deferred specification and evolving approaches may be fine for the first. For the second they are a problem. The authors write: "Current practice appears more aligned with the former function than the latter." They are careful to say the findings do not determine which function frameworks should serve.
Discussant
Now the obvious objection. They score documents. A firm with excellent internal practice and a terse document scores low.
Host
They raise this themselves in a limitations section. The answer they give is normative: the criteria are things that should be public, because public commitment is what makes accountability possible. In their words: "Hence, a low score means the Framework, taken at face value, likely describes inadequate risk management." They also concede the reverse error, citing xAI releasing Grok four without a model card despite a framework commitment to share evaluation results. They write: "Without independent auditing, external parties cannot distinguish between these cases."
Discussant
And quality of the processes, as opposed to their presence?
Host
Also conceded. The authors write: "Our criteria assess whether Frameworks describe required processes, not whether those processes are well-designed." A firm can define capability thresholds and set them too high to catch anything, or commit to auditors who are not independent, and still score well. They add: "Our methodology cannot detect these quality gaps."
Discussant
How has the paper been used and cited?
Host
Modestly so far, as a paper. OpenAlex, the open bibliographic database, lists three citing works, all preprints on Zenodo, a general research repository, from what looks like a single series of scorecard proposals. The brief on the Unjournal page records an Altmetric attention score of twenty-two, with passing news mentions. The scores themselves have more reach through SaferAI's online tracker of company ratings, which predates this paper. A guide from CASRAI, dated September twenty twenty-six, explains the tracker's rubric for a general audience and reports updated July twenty twenty-six figures, with Anthropic at thirty-five percent and a ceiling near fifty-nine percent. We did not find it cited in a regulator's own document. That is a search miss, not evidence that regulators ignore it; the EU Code of Practice process and the California law were drafted with input from many such groups, and the authors note that their criteria are similar to some of the Code's provisions.
Discussant
The numbers seem to move between versions as well.
Host
They do. An earlier arXiv version put the peer ceiling at fifty-one percent; version five says fifty-four; the tracker's later figure is higher again. Anthropic was scored on its Responsible Scaling Policy version two point two from May twenty twenty-five. Version three, released in February twenty twenty-six, removed unilateral pause commitments, and the paper says its scores do not reflect that. So the top-ranked company's score describes a document it has since replaced, in a way that touches several heavily weighted criteria.
Discussant
Where could this matter for decisions?
Host
Three audiences. Regulators: the EU AI Office and California's enforcement of Senate Bill fifty-three both need some way to judge whether a submitted framework is specific enough to enforce, and a public, criterion-level rubric with quotes is a ready-made checklist. Funders of AI governance work: the gaps it identifies are plausible research or advocacy targets. And the companies: the appendix gives each one specific improvement suggestions, and the authors argue that comparative public ratings create reputational pressure.
Discussant
Why would an Unjournal evaluation add anything?
Host
Because the paper is a measurement instrument whose outputs are being used to compare named firms, and the paper reports almost nothing about the instrument's reliability or validity. That is a good fit for an evaluator with measurement expertise alongside someone who knows risk management in a regulated industry. The data are public: every one of the seven hundred and eighty company-criterion scores comes with quotes and reasoning, so an evaluator can re-score a sample without asking anyone.
Discussant
Then let me list what I would check. First, reliability. How was the scoring done?
Host
One researcher scored the first three dimensions, another scored governance, and two more reviewed everything for consistency. The authors say they iterated until disagreements were minimal. There is no independent double scoring and no agreement statistic, such as Krippendorff's alpha. An evaluator could have two people re-score a random set of criteria blind and report agreement. The seven-level scale has verbal anchors that overlap; the authors themselves say the boundary between partially and moderately addressed is not always sharp.
Discussant
Second, the scale is treated as cardinal. Averaging labels like twenty-five and fifty assumes those gaps are equal in meaning, which the authors deny when they say the end-points need finer steps.
Host
Right, and the weights compound that. Small differences in overall score, such as Meta at twenty-one and Google DeepMind at twenty, are well inside plausible noise, yet the paper contrasts that ordering with AI Lab Watch and the Future of Life Institute index. A sensitivity analysis with alternative weights, and bootstrap intervals over criteria, would show which rankings are robust.
Discussant
Third, independence. The benchmark comes from the same organization, and SaferAI helped write one of the frameworks.
Host
The paper discloses in a footnote that SaferAI contributed to writing G42's framework. G42 ranks third overall and has the top score on several governance items, including the management risk committee and external audit. An evaluator should check those items against the appendix quotes. More generally, scoring against your own organization's ideal rewards frameworks that use your vocabulary, such as explicit KRI and KCI pairs. A defensible alternative approach scores low: Google DeepMind explicitly declines to set risk acceptance criteria for misalignment because its thresholds there are exploratory.
Discussant
Fourth, validity. Does a higher score predict safer behaviour?
Host
The paper does not test this and says so. Its own discussion offers a competing explanation for strong documentation from smaller firms: firms further from the frontier may face less pressure to implement their commitments, so ambitious documents cost them less. Sixteen of the sixty-five criteria have their best score from a company outside what the paper calls the frontier five. If that alternative reading is right, the score partly measures cheap talk. An evaluator could look for outcome correlates, such as missing model cards or third-party evaluation access.
Discussant
Anything at the level of basic care?
Host
A few things worth flagging. Some citations do not support the claims attached to them. The claim that AI capabilities can improve rapidly and unpredictably cites a control-engineering paper on nonlinear robust optimization. The claim that other industries learned from accidents cites a study of drivers' speed in fog. The claim that companies respond to public evaluation cites, among others, a study of contraceptive knowledge among Nigerian adolescents. These look like reference-management errors, but they weaken the background section. There are small internal inconsistencies too: the main text says xAI's bioweapon risk model gives five concrete steps, while the company appendix describes four intervention points. None of this changes the headline numbers, but it is a reason to spot-check the criterion-level scoring.
Discussant
Summing up, then. What is solid?
Host
The descriptive core is solid and useful: a careful, quote-backed reading of twelve documents against an explicit checklist, with transparent rationale for every score. The qualitative findings on vague tolerances, missing searches for unknown risks and unbound decision-makers do not depend on the weights. The weaker parts are the aggregate scores, the rankings between close companies, and the peer-ceiling multiple, which rest on unvalidated weights, an untested scale and a maximum operator that inflates with noise.

Key equations

Sources