TacCoRL: Integrating Tactile Feedback into VLA via Simulation

Figure 1: Left: We collect real and simulated visuo-tactile trajectories from aligned real-world and simulation setups. Center: Sim-real co-training gives the policy an initial tactile-conditioned action prior, and tactile-guided RL in a real-aligned simulator refines closed-loop contact corrections. Right: We deploy the policy directly to the real world, where it achieves high success rates across diverse contact-rich manipulation tasks.

Figure 4: Controller and tactile alignment. (A) Held-out J4 joint-response replay comparing the target, real and simulated responses before and after controller SysID. (B) Normalized tactile-reading histogram from matched contact rollouts after tactile calibration.

Figure 5: Real-world policy rollouts. Representative real-robot executions of our post-trained visuo-tactile VLA policy across four contact-rich bimanual tasks.

Figure 6: Tactile feedback improves simulator RL. Across all tasks, visuo-tactile policies consistently achieve higher success rates than vision-only policies during simulator RL.

Figure 8: Real-world robot setup. (a) Bimanual platform with two AgileX PiPER 6-DoF robotic arms and a fixed front-view RealSense D415 camera. (b) End-effector close-up with wrist-mounted RealSense D405 cameras and two FlexiTac-V2 tactile pads on the gripper contact surfaces.

Figure 12: Representative baseline failure cases. Each failure occurs after the gripper or held object occludes the local contact region. These cases expose missing corrective behavior: angle correction after rack-rim contact, lateral search for the puzzle opening, axis realignment in Assembly #1, and center-of-mass support before release in Assembly #2.

Figure 2: Pipeline. (A) We collect real demonstrations \mathcal{D}_{\mathrm{real}} together with simulated tele-operation data \mathcal{D}^{\mathrm{teleop}}_{\mathrm{sim}} , and further scale up \mathcal{D}^{\mathrm{teleop}}_{\mathrm{sim}} using MimicGen to obtain \mathcal{D}^{\mathrm{Mimic}}_{\mathrm{sim}} . (B) During sim-real co-training, tactile information is encoded and routed through contact-aware gating to modulate both the context of vision-language models (VLM) and the action expert. (C) Interactive simulation rollouts help the tactile-aware policy improve closed-loop task performanc

Figure 7: Ablation of co-training and real-data anchoring. We vary the co-training ratio \alpha and real-anchor weight \beta on the Assembly #2 task and report model performance in terms of simulator success rate (left) , real-data anchor loss (middle) , which measures policy deviation from real demonstrations during simulation, and real-world deployment success rate (right) .