Release2026-09-21

Robocurve released RoboHarm, a benchmark of five fixed robot refusal tasks. It tests whether models refuse harmful instructions in set scenes. The scenes include a knife and doll, aerosol can and burner, screwdriver and toaster, power bank and water, and cleaning products. Each task targets 20 rollouts per model. Operators label behavior from three camera views and saved transcripts. The package includes experiment tooling and a labeling dashboard. It builds on Inspect Robots. Tasks use indirect wording to avoid bias. Synthetic data allows hardware-free testing. Physical replication requires inert substitutes for safety.

What it unlocks

Evaluating robot refusal behavior in fixed harmful scenarios using standardized tasks and labeling.

What you need to act on it

  • Python 3.12+
  • uv package manager
  • calibrated robot hardware for physical experiments

Sources

Send this to someone who needs it

Shares the story and its sources. Nothing about you.

What does this mean for your job?

This is the story as everyone gets it. Once a week we send you the version written for your role — what changed, why it matters for the work you actually do, and one thing to try. Free while we tune it.