English | 简体中文

Perspective-Invariant 3D Object Detection

Ao Liang     Lingdong Kong     Dongyue Lu     Youquan Liu     Jian Fang Huaici Zhao     Wei Tsang Ooi

<p align="center">
<a href="https://arxiv.org/abs/2507.17665" target='_blank'>
<img src="https://img.shields.io/badge/Paper-%F0%9F%93%96-darkred">
</a>&nbsp;
<a href="http://pi3det.github.io/" target='_blank'>
<img src="https://img.shields.io/badge/Project-%F0%9F%94%97-blue">
</a>&nbsp;
<a href="https://huggingface.co/datasets/Pi3DET/data" target='_blank'>
<img src="https://img.shields.io/badge/Dataset-%F0%9F%94%97-green">
</a>&nbsp;
<a href="" target='_blank'>
<img src="https://visitor-badge.laobi.icu/badge?page_id=pi3det.Pi3EDT">
</a>
</p>

This work focuses on the practical yet challenging task of 3D object detection from heterogeneous robot platforms: Vehicle, Drone, and Quadruped. To achieve strong generalization ability, we contribute:

• The first dataset for multi-platform 3D object detection, comprising more than 51,000+ LiDAR frames with over 250,000+ meticulously annotated 3D bounding boxes.
• A cross-platform 3D domain adaptation framework, effectively transferring capabilities from vehicles to other platforms by integrating geometric and feature-level representations.
• A comprehensive benchmark study of state-of-the-art 3D object detectors on cross-platform scenarios.

:books: Citation

If you find this work helpful for your research, please kindly consider citing our papers:

@inproceedings{liang2025pi3det,
    title     = {Perspective-Invariant {3D} Object Detection},
    author    = {Ao Liang and Lingdong Kong and Dongyue Lu and Youquan Liu and Jian Fang and Huaici Zhao and Wei Tsang Ooi},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision},
    pages     = {27725-27738},
    year      = {2025},
}

@misc{robosense_challenge_2025,
    title     = {The {RoboSense} Challenge: Sense Anything, Navigate Anywhere, Adapt Across Platforms},
    author    = {Kong, Lingdong and Xie, Shaoyuan and Gong, Zeying and Li, Ye and Chu, Meng and Liang, Ao and Dong, Yuhao and Hu, Tianshuai and Qiu, Ronghe and Li, Rong and Hu, Hanjiang and Lu, Dongyue and Yin, Wei and Ding, Wenhao and Li, Linfeng and Song, Hang and Zhang, Wenwei and Ma, Yuexin and Liang, Junwei and Zheng, Zhedong and Ng, Lai Xing and Cottereau, Benoit R. and Ooi, Wei Tsang and Liu, Ziwei and Zhang, Zhanpeng and Qiu, Weichao and Zhang, Wei and Ao, Ji and Zheng, Jiangpeng and Wang, Siyu and Yang, Guang and Zhang, Zihao and Zhong, Yu and Gao, Enzhu and Zheng, Xinhan and Wang, Xueting and Li, Shouming and Gao, Yunkai and Lan, Siming and Han, Mingfei and Hu, Xing and Malic, Dusan and Fruhwirth-Reisinger, Christian and Prutsch, Alexander and Lin, Wei and Schulter, Samuel and Possegger, Horst and Li, Linfeng and Zhao, Jian and Yang, Zepeng and Song, Yuhang and Lin, Bojun and Zhang, Tianle and Yuan, Yuchen and Zhang, Chi and Li, Xuelong and Kim, Youngseok and Hwang, Sihwan and Jeong, Hyeonjun and Wu, Aodi and Luo, Xubo and Xiao, Erjia and Zhang, Lingfeng and Tang, Yingbo and Cheng, Hao and Xu, Renjing and Ding, Wenbo and Zhou, Lei and Chen, Long and Ye, Hangjun and Hao, Xiaoshuai and Li, Shuangzhi and Shen, Junlong and Li, Xingyu and Ruan, Hao and Lin, Jinliang and Luo, Zhiming and Zang, Yu and Wang, Cheng and Wang, Hanshi and Gong, Xijie and Yang, Yixiang and Ma, Qianli and Zhang, Zhipeng and Shi, Wenxiang and Zhou, Jingmeng and Zeng, Weijun and Xu, Kexin and Zhang, Yuchen and Fu, Haoxiang and Hu, Ruibin and Ma, Yanbiao and Feng, Xiyan and Zhang, Wenbo and Zhang, Lu and Zhuge, Yunzhi and Lu, Huchuan and He, You and Yu, Seungjun and Park, Junsung and Lim, Youngsun and Shim, Hyunjung and Liang, Faduo and Wang, Zihang and Peng, Yiming and Zong, Guanyu and Li, Xu and Wang, Binghao and Wei, Hao and Ma, Yongxin and Shi, Yunke and Liu, Shuaipeng and Kong, Dong and Lin, Yongchun and Yang, Huitong and Lei, Liang and Li, Haoang and Zhang, Xinliang and Wang, Zhiyong and Wang, Xiaofeng and Fu, Yuxia and Luo, Yadan and Etchegaray, Djamahl and Li, Yang and Li, Congfei and Sun, Yuxiang and Zhu, Wenkai and Xu, Wang and Li, Linru and Liao, Longjie and Yan, Jun and Wang, Benwu and Ren, Xueliang and Yue, Xiaoyu and Zheng, Jixian and Wu, Jinfeng and Qin, Shurui and Cong, Wei and He, Yao},
    howpublished = {\url{https://robosense2025.github.io}},
    year      = {2025}
}

Updates

• [12/2025] - We have publised a visiualization toolkit for Pi3DET-Dataset. Have fun at Pi3DET-Visualization.
• [10/2025] - We have published the baseline models and our key methods for data augmentation. See <a href="https://github.com/robosense2025/track5" target="_blank" rel="noopener noreferrer">GitHub repo</a> for more details on data preparation and installation.
• [07/2025] - The Pi3DET dataset has been extended to <strong>Track 5: Cross-Platform 3D Object Detection</strong> of the <a href="https://robosense2025.github.io/" target="_blank">RoboSense Challenge</a> at <a href="https://www.iros25.org/" target="_blank">IROS 2025</a>. See the <a href="https://robosense2025.github.io/track5" target="_blank" rel="noopener noreferrer">track homepage</a> and <a href="https://github.com/robosense2025/track5" target="_blank" rel="noopener noreferrer">GitHub repo</a> for more details.
• [07/2025] - The project page is online. :rocket:
• [07/2025] - This work has been accepted to ICCV 2025. See you in Honolulu! 🌸

Outline

• Updates
• Outline
• :gear: Installation
• :hotsprings: Data Preparation
• :rocket: Getting Started
• Model Zoo
• :triangular_ruler: Pi3DET Benchmark
  • Statistical Analysis
  • Methodology
• Pi3DET Dataset
  • Detailed statistical information
  • Dataset Examples
• :memo: TODO List
• License
• Acknowledgements
• Related Projects

:gear: Installation

For details related to installation and environment setups, kindly refer to INSTALL.md.

:hotsprings: Data Preparation

Kindly refer to our HuggingFace Dataset :hugs: page from here for more details.

:rocket: Getting Started

To learn more usage of this codebase, kindly refer to GET_STARTED.md.

Model Zoo

<details open>
<summary>&nbsp<b>Grid-Based 3D Detector</b></summary>

• [x] [SECOND](), Sensors 2018.
• [x] [PointPillar](), CVPR 2019.
• [x] *[Part A](), TPAMI 2020.**
• [x] [CenterPoint](), CVPR 2021.
• [x] [Transfusion-L](), CVPR 2022.
• [x] [PillarNet](), ECCV 2022.
• [x] [HEDNet](), NeurIPS 2023.
• [x] [SAFNet](), ECCV 2024.

</details>

<details open>
<summary>&nbsp<b>Point-Based 3D Detector</b></summary>

• [x] [PointRCNN](), CVPR 2019.
• [x] [3DSSD](), CVPR 2020.
• [x] [IA-SSD](), CVPR 2022.
• [x] [DBQ-SSD](), arXiv 2022.

</details>

<details open>
<summary>&nbsp<b>Grid-Point 3D Detector</b></summary>

• [x] [PV-RCNN](), CVPR 2020.
• [x] [VoxelRCNN](), AAAI 2021.
• [x] [PV-RCNN++](), IJCV 2023.

</details>

:triangular_ruler: Pi3DET Benchmark

Statistical Analysis

We observe significant cross-platform geometric discrepancies in ego‑motion jitter, point‑cloud elevation distributions, and target pitch‑angle distributions across vehicle, quadruped, and drone platforms, which hinder single‑platform model generalization.

Methodology

Pi3DET‑Net employs a two‑stage adaptation pipeline—Pre‑Adaptation uses random jitter and virtual poses to learn and align global geometric transformations, while Knowledge Adaptation leverages geometry‑aware descriptors and KL‑based probabilistic feature alignment to synchronize feature distributions across platforms.

Pi3DET Dataset

Detailed statistical information

Dataset Examples

:memo: TODO List

• [x] Initial release. 🚀
• [x] Release the dataset for the RoboSense Challenge 2025.
• [x] Release the code for the RoboSense Challenge 2025.
• [x] Release the whole Pi3DET dataset.
• [x] Release the code for the Pi3DET-Net method.

License

This work is under the <a rel="license" href="https://www.apache.org/licenses/LICENSE-2.0">Apache License Version 2.0</a>, while some specific implementations in this codebase might be under other licenses. Kindly refer to LICENSE.md for a more careful check, if you are using our code for commercial matters.

Acknowledgements

This work is developed based on the MMDetection3D codebase.

<img src="https://github.com/open-mmlab/mmdetection3d/blob/main/resources/mmdet3d-logo.png" width="31%"/><br>
MMDetection3D is an open-source toolbox based on PyTorch, towards the next-generation platform for general 3D perception. It is a part of the OpenMMLab project developed by MMLab.

Part of the benchmarked models are from the OpenPCDet and 3DTrans projects.

Related Projects