RynnValue trains robot reward models on 7,000 hours without preference labels
Open-source value foundation model replaces preference annotations with temporal distance, trains on 3M clips, and lifts real-world policy success from 52.5% to 72.5%.
Reward modeling has become the bottleneck in scaling robot learning. A new preprint argues that temporal distance—the time remaining until a language-specified goal—solves the labeling problem at scale without hand-annotated preferences.
RynnValue, an open-source value foundation model for robotic manipulation, trains on over 7,000 hours of robot data spanning roughly 3 million instruction-conditioned video clips. Rather than relying on hand-labeled preferences or normalized progress scores, the model derives supervision directly from timestamps already embedded in logged trajectories. The directed cost-to-go from an observation to the goal becomes the training signal, sidestepping the annotation bottleneck that has limited prior reward models to smaller, manually curated datasets.
To prevent shortcuts—patterns that ignore task failures or regressions—the training recipe combines random temporal sampling, temporal-order shuffling, and value-isolation attention. On the out-of-distribution split of RBM-EVAL-OOD, RynnValue achieves a Kendall's tau of 0.675, surpassing the previous preference-supervised state of the art (0.655) and more than doubling a progress-only baseline (0.292). The model generalizes zero-shot to unseen tasks, embodiments, and camera viewpoints.
When converted into dense rewards via potential-based shaping, RynnValue raises real-world policy success from 52.5% to 72.5% in online rollouts and from 63.8% to 82.5% in offline evaluation. The preprint, posted August 11, 2026, positions temporal distance as both a scalable supervision target and a practical reward interface for generalist robot policies. Code and weights are available open-source.
