Occupancy prediction at voxel-level granularity is essential for safe robotic navigation and interaction in complex environments. Existing occupancy datasets, however, are predominantly designed for autonomous driving with vehicle-centric biases---forward-facing cameras, far-field geometry, and static road priors---limiting their applicability to embodied humanoid perception.
We present Humanoid-OmniOcc, a large-scale panoramic stereo-based occupancy dataset tailored for humanoid robots. Built on NVIDIA Isaac Sim, the dataset captures full-surround RGB via four synchronized stereo pairs mounted on a head-like rig, and provides high-fidelity voxel ground truth with fine-grained semantic annotations across 15 indoor categories through physically-based rendering and multi-view geometric verification. Humanoid-OmniOcc encompasses 15 diverse simulated indoor scenes and 5 real-world environments, yielding over 155K samples with broad scene and style diversity.
Importantly, the dataset is designed around a Real2Sim2Real closed-loop paradigm: real sensor specifications drive physically accurate simulation, simulation produces large-scale annotated training data, and models trained in simulation are directly evaluated on real-world captures---enabling iterative refinement of the sim-to-real pipeline.
We further propose OmniStereo, a surround stereo-guided occupancy network that exploits robust depth priors for accurate 2D-to-3D lifting. Extensive experiments show that OmniStereo consistently outperforms monocular baselines and generalizes well to both unseen simulated test scenes and real-world environments, validating the effectiveness of the Real2Sim2Real design.
Comparison of different 3D perception paradigms. Top: Monocular-based model. Middle: Multi-sensor fusion model. Bottom: Our proposed stereo-based model.
The pipeline of our proposed OmniStereo framework. An image encoder first extracts features from surround-view stereo images. These features are then processed along two decoupled pathways: the upper pathway lifts 2D features to 3D camera voxel features via a 2D-to-3D transformer for occupancy prediction, while the lower pathway estimates depth. Importantly, the estimated depth serves as an auxiliary input to the 2D-to-3D transformer, enhancing the accuracy of the view transformation. Finally, an occupancy head predicts the 3D occupancy grid from the camera voxel features.
Semantic occupancy prediction results on the test sets and in the real world. We report per-class IoU and mIoU (%).
Ablation Study on Disparity-to-Depth: DDVM vs. SDN.
Ablation Study on Stereo module.
Qualitative comparisons on the test set (first row) and real-world scenes (rows 2–3). The first column shows the left images from the four stereo pairs (front, back, left, and right).
All users can obtain and use this dataset and its subsets only after signing the Agreement and sending it to the official contact email addresses.
For students, research groups, or research institutions who need to obtain this data for academic research, the signatures of both the Responsible Party and the Point of Contact are required. The Responsible Party should be the tutor of the research group or the representative of the research institution, and has the authority to be responsible for the matters related to the Agreement. The Responsible Party should be a fixed personnel of the institution; students and other floating personnel cannot be the Responsible Party. The Point of Contact (POC) is an individual with detailed knowledge of the dataset application. In some cases, the Responsible Party and the POC may be the same person. At the same time, the homepage of the Responsible Party (or the homepage of the research group, laboratory, Department, etc., including the basic information of the Responsible Party) must be provided to prove the role of the Responsible Party.
To request access to the Humanoid-OmniOcc dataset, please send the signed Agreement as a scanned copy to xianda_guo@163.com and CC wei.sui@d-robotics.cc.
Email format for Humanoid-OmniOcc dataset application:
Subject: Humanoid-OmniOcc Dataset Application
To: xianda_guo@163.com
CC: wei.sui@d-robotics.cc
Attachment: License_Agreement_for_Humanoid-OmniOcc_Dataset.pdf (scanned copy)@article{guo2026humanoidomniocc,
author = {Xianda Guo and Bohao Zhang and Chenwei Huang and Shiyuan Chen and Ruilin Wang and Yiqun Duan and Cong Yang and Qin Zou and Wei Sui},
title = {Humanoid-OmniOcc: Stereo-Based Full-View Occupancy Dataset for Embodied AI},
journal = {arXiv preprint arXiv:2606.22971},
year = {2026},
eprint = {2606.22971},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.22971},
}