Simulation success
52.3%RoboTwin2.0 · 50 tasks
Real-Time (32.9 Hz) Multi-View Visuomotor Policy
for Robotic Manipulation on Edge Devices
1School of Future Science and Engineering, Soochow University
2Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University
3D-Robotics
4State Key Laboratory of Multimodal Artificial
Intelligence Systems,
Institute of Automation, Chinese Academy
of Sciences
* Equal contribution † Project lead ✉ Corresponding authors
Wei Sui: wei.sui@d-robotics.cc Cong Yang: cong.yang@suda.edu.cn
01 / INTRODUCTION
See more. Compute faster.
RTPolicy brings multi-view visual reasoning and fast action generation to robotic manipulation. A frozen DINOv3 ConvNeXt encoder, view-aware feature fusion and spatial compression connect RGB observations to a four-step flow-matching policy. The policy is quantized to INT16 for S100 and S600.
02 / METHOD
RTPolicy architecture
01 / OBSERVATION
Observe the scene
Three camera views and the current robot state provide the observation.
1 / 4
03 / RESULTS
Performance & efficiency
GPU inference
91.6 HzRTX 4090 · TensorRT FP16
Visual tokens
90012640 × 480
Action generation
4 stepsEuler integration per action chunk
RoboTwin2.0 success rate
Clean-to-clean · 50 tasks
50 demonstrations and 100 evaluation episodes per task.
Inference latency
RDK S600 · INT16 · batch 1
INT16 quantization
RTPolicy · compiler estimates
04 / GETTING STARTED
Get started
Setup & commands
AI-assisted setup
Give your coding agent the guide and an artifact directory. It downloads the released Google Drive checkpoint, installs RTPolicy, and validates an ONNX export.
View the handoff prompt
Repository structure
rtpolicy/Policy, training and ONNX export
configs/Model and experiment settings
simulation/robotwin2.0/RoboTwin2.0 interface
scripts/Inference and deployment tools
tools/Dataset preparationdocs/Training and AI setup guides
05 / TRAIN & DEPLOY
Train, export, deploy.
01 / TRAIN
Configure the policy
Set cameras, state and action dimensions, the data backend and training settings in YAML.
02 / EXPORT
Export & verify ONNX
Export using the checkpoint's observation layout and sampling settings, then verify its outputs.
03 / DEPLOY
Run on the target device
Use the ONNX inference tools or build a mixed-FP16 TensorRT engine for compatible NVIDIA hardware.
ACKNOWLEDGEMENTS
Built on shared research.
We thank the authors of DINOv3 and VO-DP for sharing their research and code.