Anthropic tests automated researchers to mitigate ten alignment failures
Image generated by the Inferama Editorial Team
01

What Anthropic tested

Anthropic published an experiment in which Claude acted as an automated alignment researcher: it searched for methods and data to train models and reduce specific failures measured by benchmarks. The task was not to prove that a model was safe in every circumstance, but to improve results defined in advance.

The system worked on ten categories of alignment failures, addressing them separately. Anthropic’s examples include privacy violations and sycophancy—the tendency to agree with a user even when doing so does not support a truthful or appropriate answer. Each category was evaluated using specific benchmarks.

Anthropic says Claude’s methods improved the target benchmarks in all ten categories without reducing performance on the capabilities measured in the study. That conclusion depends on this particular set of tests: it does not cover every possible behavior or amount to a general safety certification for models.

02

How the research loop worked

For each failure, Claude searched the literature, proposed a method and training data, trained a model, and assessed the result. Scores informed later attempts, allowing the system to iterate based on previous evaluations rather than submit a single untested proposal.

The study separated benchmarks used to guide the search from others reserved to assess generalization. A monitoring agent also read the methods Claude intended to run. Anthropic additionally prohibited the system from directly copying its own alignment into the target model. These controls constrained the experiment, but did not by themselves guarantee that every intervention was correct or safe.

The main measure was the percentage of the safety gap closed: how far the model moved toward the theoretical perfect score on the selected benchmarks. Interventions that harmed performance on the chosen capability checks were rejected. The metric describes progress on those evaluations, not a universal measure of alignment.

From proposed method to evaluation

The process repeated a sequence of research, training, and evaluation. A favorable score indicated a result on the selected benchmarks, not a guarantee of safety beyond them.

  1. 01Claude reviewed the literature and proposed a method and training data for a specific failure.
  2. 02A monitoring agent examined the method before it ran.
  3. 03The system trained the target model, and an independent evaluator measured the result.
  4. 04Scores informed further attempts; reserved benchmarks were used to test generalization.
03

Results reported by Anthropic

Anthropic says Claude improved results on the target benchmarks in all ten categories studied. The selected methods also worked on alignment evaluations Claude had not seen during its search and on Petri, an open-source tool that simulates adversarial, multi-turn scenarios. Petri is an approximation for evaluating behavior, not a direct test of real-world safety.

Anthropic also reports that the methods remained effective on models up to 4.7 times the size of those optimized during the search. In a deception experiment with Gemma-2-2B, Claude made more than 150 attempts. The best run closed 82% of the safety gap; the reported average across multiple runs was 85%. These figures describe different measurements: one is from the highlighted run, and the other is an average across several runs.

In another experiment, Claude Sonnet 5 worked for 60 hours to mitigate alignment failures in an early version of Claude Opus 4.8. Anthropic says the winning solution achieved alignment scores close to those of its production models and used just over 2,000 training examples. The company’s efficiency comparison concerns its production alignment procedure; it does not show that the method is interchangeable with that procedure in other models or settings.

What the reported results mean

The figures summarize specific tests and configurations. They should not be interpreted as universal measures of safety.

ResultWhat it indicatesWhat it does not show
Improvement across ten categoriesClaude improved the selected benchmarks for each category.That every possible alignment failure has been addressed.
Reserved evaluations and PetriSome methods worked beyond the benchmarks used during the search.That the evaluations fully reproduce risks in deployed systems.
Transfer to larger modelsAnthropic observed effectiveness on models up to 4.7 times the size of those optimized.That the result generalizes to every model or scale.
04

Specific methods and limits of evaluation

For work on sycophancy, Anthropic says 98% of proposed methods used a technique described in published research: training a model on its own non-sycophantic answers. Claude used activation steering to generate less sycophantic examples. The technique adjusts internal representations during generation; its use in this experiment does not establish that it eliminates sycophancy in general.

For privacy violations, Anthropic cites ConfAIde, PrivaCI-Bench, and PrivacyLens as benchmarks. The company says Claude improved measured performance on them, but the published summary does not attribute the result to a single intervention or provide a breakdown here that would allow the benchmarks to be compared individually.

The number and variety of evaluations matter. In an ablation test described in the report, optimizing a single prompt-injection evaluation improved results on that test but did not generalize reliably to other unseen tests. This result shows why a high score on one benchmark is not enough to support the claim that the underlying problem has been mitigated.

05

Comparison with human researchers

Anthropic compared automated methods with proposals from human researchers. The company says Claude outperformed the best human proposal in the seven categories with comparable proposals; in the deception example, its best method performed 20% better than the strongest human proposal.

The comparison was not equivalent. Human researchers could spend up to eight hours developing proposals, but could not test and revise them after seeing their scores. Claude, by contrast, could make successive attempts and use evaluation results to guide later ones. The result does not show that Claude outperforms human researchers under equal conditions.

Anthropic presents the comparison as an indication that automated search can explore promising methods for people to evaluate and refine. That interpretation is narrower than claiming the system can replace alignment researchers.

How to interpret the comparison

The groups had different opportunities to test and improve their methods.

AspectHuman researchersAutomated researchers
Initial proposalUp to eight hours to develop methods.The system proposed methods for evaluation.
Iteration using scoresCould not revise proposals after seeing the results.Could use scores to guide further attempts.
How to read the resultA reference based on proposals without comparable iteration.Evidence of iterative search, not superiority under equal conditions.
06

What the study does not show

The categories examined were narrower than the range of failures possible in production systems. Anthropic acknowledges that it did not measure, for example, political bias, and that some problems may occur so rarely or be so recent that no suitable benchmark exists to measure them.

The capability checks were also limited in scope. The study rejected methods that harmed the measured capabilities, but the selected tests did not cover every important capability. Therefore, the finding that interventions did not degrade those checks does not guarantee that they preserved every other useful ability.

Anthropic also did not test whether improvements persisted after extensive reinforcement learning on other tasks. The company further acknowledges that Petri and other evaluations are approximations of real-world misalignment. The reported effectiveness is an experimental result, not a guarantee about the behavior of deployed systems.

The study offers early indications that automated research could help develop post-training methods for defined failures. Its strongest result is an iterative search that improved selected benchmarks, including some reserved evaluations. General safety, the persistence of improvements, and their usefulness in production conditions remain outside what this experiment establishes.

07

Implications for alignment research

The most concrete contribution is a workflow in which a system searches for, tests, and documents interventions to reduce measurable failures. Monitoring and separating evaluations can help control that process, but they do not replace scrutiny of the tests’ scope or evaluation of the results.

When assessing future applications, it helps to distinguish three questions: whether an intervention improves the tests used to develop it, whether it transfers to evaluations the system did not see, and whether its effects persist outside that configuration. The study reports results on the first two in some cases; it does not resolve the third across broad production conditions.

Anthropic says it has released the automated research harness so others can build on it and use it with their own models. The harness’s availability does not imply that all the data, results, or procedures needed to reproduce every finding are included. The work supports exploring automated tools under constrained controls and evaluations, not a general conclusion that alignment can already be fully automated.

Three questions for interpreting a mitigation

These questions distinguish an improvement on a test from a broader claim about a model’s behavior.

  1. 01Did the model improve on the evaluations used to guide the method?
  2. 02Did the improvement hold on reserved tests the system had not seen?
  3. 03Was the effect tested across other models, tasks, and conditions of use?
08

Keep exploring

Sources consulted

03

Corrections and transparency

If you spot incorrect or outdated information, send us a correction with the page and source we should review.

Submit a correction