Temporal Difference Calibration in Sequential Tasks: Application to Vision-Language-Action Models

Figure 4 : Averaged success rates over additional simulation steps for action selection configurations . All experiments evaluate 3 unseen tasks from LIBERO-10 taken with OpenVLA, consisting of 50 rollouts each, avereged over 3 seeds. RNN-TDQC achieves highest success rates, while RNN-TDQC with thresholds tradeoff success rate and compute.

Figure 7 : Successful rollout with informative failure scores of TDQC top 10 probabilities on OpenVLA LIBERO-10 benckmark. task: “put both the alphabet soup and the tomato sauce in the basket”. The failure score rises when the policy becomes temporarily stuck while trying to drop the alphabet soup into the basket around step 140. Then, failure score decreases after recovery around step 275, and increases again when the robot is positioned himself and attempts to pick up the tomato soup at step 336.

(a) Example of failure score of a successful episode detected in SAFE-RNN-BCE (features). This is the same rollout as in Figure 7 .

(b) Example of failure score of a successful episode detected in SAFE-RNN-TDQC (features). This is the same rollout as in Figure 7 .

Figure 1: Sequential Brier scores across benchmarks. Sequential Brier score (lower is better) on an unseen validation set averaged over 21 random seeds (train/validation task splits). To compare calibration across rollouts with different lengths, we report Brier score over time quantiles . Each subplot corresponds to a (VLA model, benchmark) pair. Success prediction methods are based on sequences of features or action probabilities. Across all settings, our TD-based methods consistently outperform conventional predictors trained with binary cross entropy (BCE). For \pi_{0} action probabilities

Figure 3 : ROC-AUC vs Brier score over all learned baselines in all benchmarks at the minimum rollout length. Points are grouped by method and split, with a dashed linear fit; the Spearman correlation is \rho=-0.686 which indicates high negative correlation.

Figure 12 : Brier scores val-seen sequential Brier score (lower is better) on the seen validation set averaged over 21 random seeds and all environments. We report Brier score in different time quantiles , where each subplot corresponds to a (model-benchmark) pair. For \pi_{0} , action probabilities are not directly interpretable, hence probability-based TDQC variants are not reported. Across all settings, our TD-based methods consistently outperform conventional predictors trained with binary cross entropy (BCE). For \pi_{0} action probabilities are not directly interpretable, hence probabili

Figure 13 : Brier scores val-unseen sequential Brier score (lower is better) on the unseen validation set averaged over 21 random seeds. We report Brier score in different time quantiles , where each subplot corresponds to a (model-benchmark) pair. For \pi_{0} , action probabilities are not directly interpretable, hence probability-based TDQC variants are not reported. Across all settings, our TD-based methods consistently outperform conventional predictors trained with binary cross entropy (BCE). For \pi_{0} action probabilities are not directly interpretable, hence probability-based TDQC var