REBAR: Reference Ethical Benchmark for Autonomy Readiness
Jonathan Diller, David Barnes, Rebekah Bogdanoff, Rhett Collier, Roddy Collins, Keith Fieldhouse, Yonatan Gefen, Cameron Johnson, Anuriha Kodali, Brad Kriel, Varun Murali, James Niehaus, Mish Sukharev, Joseph VanPelt, Anthony Hoogs, Vijay Kumar, Arslan Basharat
THE PROBLEM
This paper focuses on learning. Introduces a quantitative benchmarking framework for evaluating ethical Movement, Mechanics & Robot BodyComplianceThe robot’s ability to yield a little during contact instead of staying rigid. of autonomous systems using neuro-symbolic LLM approaches, LLM-driven test generation, and photorealistic Simulation & Sim-to-RealSimulationA virtual environment where robots can be trained or tested. to produce an Autonomy Readiness Level (ARL) score. Read the paper by tracking the Core ConceptsTaskThe job the robot is supposed to complete, such as pick-and-place, navigation, or drawer opening. definition, the Core ConceptsRobotA physical system with sensors and actuators that can observe the world and take actions. or data assumptions, and the evidence that supports the claimed improvement.
HOW IT WORKS
Task framing
Core method
Data and supervision
Evaluation evidence
KEY RESULTS
REBAR gives you a quantifiable, repeatable way to measure whether your autonomous system behaves ethically and legally—converting vague safety principles into concrete test scores and Autonomy Readiness Levels. This lets developers objectively compare systems, identify failure modes before Simulation & Sim-to-RealDeploymentPutting the trained system on a real robot., and give users interpretable explanations of what their Core ConceptsRobotA physical system with sensors and actuators that can observe the world and take actions. can and cannot safely do.
WHY DEVELOPERS SHOULD CARE
REBAR gives you a quantifiable, repeatable way to measure whether your autonomous system behaves ethically and legally—converting vague safety principles into concrete test scores and Autonomy Readiness Levels. This lets developers objectively compare systems, identify failure modes before Simulation & Sim-to-RealDeploymentPutting the trained system on a real robot., and give users interpretable explanations of what their Core ConceptsRobotA physical system with sensors and actuators that can observe the world and take actions. can and cannot safely do.
LIMITATIONS
The main limitation to check is whether the claimed behavior holds outside the paper's reported setup. That means testing across different Core ConceptsRobotA physical system with sensors and actuators that can observe the world and take actions. embodiments, scenes, objects, and data distributions.
WHAT COMES NEXT
The practical next step is independent reproduction with clear baselines, ablations, and stress tests. For a developer, the useful follow-up is to map the paper's learning assumptions onto a concrete Core ConceptsRobotA physical system with sensors and actuators that can observe the world and take actions. stack, then test the smallest version of the method that could run end to end.