Learning Dynamic Recovery Policies for Dexterous Manipulation

Implicit Contact Representations with Neural Descriptor Fields for Learning Dynamic Recovery Policies

Fan Yang Sergio Francisco Aguilera Marinovic Soshi Iba Rana Soltani Zarrin Dmitry Berenson

RSS 2025 - Workshop on Learned Robot Representations

Real-world dexterous manipulation often encounters unexpected errors and disturbances, which can lead to catastrophic failures, such as dropping the manipulated object. To address this challenge, we investigate the problem of catching a falling object while it remains within grasping range and, more importantly, resetting the system to a configuration favorable for resuming the primary manipulation task. We propose Contact-Aware Dynamic Recover (CADRE), a reinforcement learning framework that incorporates a Neural Descriptor Field (NDF)-inspired module to provide an implicit contact-centric representation. Compared to methods that rely solely on object pose or point cloud input, NDFs can directly reason about finger-object correspondence and naturally adapt to different object geometries. Our experiments show that incorporating contact features improves training efficiency, enhances convergence performance for RL training, and ultimately leads to more successful recoveries. Additionally, CADRE demonstrates zero-shot generalization ability to unseen objects with different geometries.

Downloadable item