Asynchronous Methods for Deep Reinforcement Learning

The paper frames the work as navigationNavigation & LocomotionNavigationMoving through an environment toward a goal., continuous motor controlControl & PlanningControlThe method used to make the robot move the way you want.. The reported platform or hardware context is simulation-only. The evaluationSimulation & Sim-to-RealEvaluationMeasuring how well a robot system performs. setting is simulationSimulation & Sim-to-RealSimulationA virtual environment where robots can be trained or tested.. Start here because it defines what success means and which assumptions the rest of the method inherits.

The method is organized around asynchronous actor-critic, deep neural network. A3C showed you can train RLImitation & Reinforcement LearningReinforcement Learning (RL)Teaching a robot through trial and error using rewards. agents efficiently on CPUs using parallel workers instead of expensive GPUs—this made deep RLImitation & Reinforcement LearningReinforcement Learning (RL)Teaching a robot through trial and error using rewards. accessible and became the foundation for distributed RLImitation & Reinforcement LearningReinforcement Learning (RL)Teaching a robot through trial and error using rewards. trainingRobot LearningTrainingThe process of fitting a model using data or experience. pipelines that most robotics platforms still use. It's essential background for understanding modern multi-agent and scalable robot learningRobot LearningRobot learningUsing data and algorithms to help robots improve behavior instead of only relying on hand-written rules. systems. When reading the method section, identify the inputs, the learned or engineered representation, and the actionCore ConceptsActionA command the robot sends to its motors, controller, or low-level system. or prediction produced by the system.

For robotics work, the data story is part of the method: check whether the system depends on teleoperationImitation & Reinforcement LearningTeleoperation (teleop)A human remotely controlling the robot, often to collect demonstrations., simulationSimulation & Sim-to-RealSimulationA virtual environment where robots can be trained or tested., internet video, human labels, or robotCore ConceptsRobotA physical system with sensors and actuators that can observe the world and take actions. rollouts.

The key reported result is Asynchronous actor-critic surpasses state-of-the-art on Atari domain while trainingRobot LearningTrainingThe process of fitting a model using data or experience. for half the time on a single multi-core CPU instead of GPU. Look for the gap between the headline result and the deploymentSimulation & Sim-to-RealDeploymentPutting the trained system on a real robot. setting you would actually care about.