Runsen Xu 徐润森

Runsen Xu 徐润森

Ph.D. student @ MMLab, CUHK

About Me

Hi! I am a second-year Ph.D. student of MMLab, The Chinese University of Hong Kong, supervised by Prof. Dahua Lin. Currently, I am a visiting student at Shanghai AI Laboratory, working with Jiangmiao Pang. Prior to this, I received my Bachelor’s degree in Computer Science and Technology from Chu Kochen Honors College (Pursuit Science Class) at Zhejiang University in 2022. In ZJU, I was fortunate to be supervsied by Prof. Guofeng Zhang, at the State Key Lab of CAD&CG.

My research interests lie in 3D Computer Vision and Natural Language Processing. At present, my research is focused on developing a language-grounded, holistic 3D visual perception system as capable as humans. If my research sparks your interest or if you have any queries, please feel free to reach out. I am always open for discussions, fresh ideas, and potential collaborations.

News

  • [Sep. 2023] Our HC-Net is accepted by NeurIPS 2023! 🎉
  • [Aug. 2023] We release PointLLM, a multi-modal large language model capable of understanding point clouds! Try our demo here. 🤗
  • [Aug. 2023] We release HC-Net, a SOTA fine-grained cross-view geo-localization model! Demo here. 📊
  • [Mar. 2023] Our paper on MV-JAR, a LiDAR-based self-supervised pre-training method along with a new data-efficient benchmark, has been accepted by CVPR 2023. 🎉
  • [Jun. 2022] Graduated from Zhejiang University. Forever cherishing the memories from ZJU. 🎓
  • [Apr. 2022] Honored to receive the Hong Kong PhD Fellowship (HKPFS) and the CUHK Vice-Chancellor’s PhD Scholarship. Deeply grateful for the recognition! 🏆

Education

cuhk
 The Chinese University of Hong Kong (CUHK)
  August 2022 - June 2026 (Expected)
  Ph.D. in Information Engineering
zju
 Zhejiang University (ZJU)
  September 2018 - June 2022
  B.E. in Computer Science and Technology

Publications

3D Perception and Understanding
PointLLM
PointLLM: Empowering Large Language Models to Understand Point Clouds
Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang#, Dahua Lin
arXiv Preprint, 2023
[Paper] [Code] [Project] [Demo] [Cite]
Self-Supervised 3D Representation Learning
MV-JAR
MV-JAR: Masked Voxel Jigsaw and Reconstruction for LiDAR-Based Self-Supervised Pre-Training
Runsen Xu, Tai Wang, Wenwei Zhang, Runjian Chen, Jinkun Cao, Jiangmiao Pang#, Dahua Lin
Computer Vision and Pattern Recognition, CVPR 2023
[Paper] [Code] [Video] [Slides] [Cite]
CO^3
COˆ3: Cooperative Unsupervised 3D Representation Learning for Autonomous Driving
Runjian Chen, Yao Mu, Runsen Xu, Wenqi Shao, Chenhan Jiang, Hang Xu, Zhenguo Li, Ping Luo
International Conference on Learning Representations, ICLR 2023
[Paper] [Code] [Cite]
Robot Localization and Navigation
HC-Net
Fine-Grained Cross-View Geo-Localization Using a Correlation-Aware Homography Estimator
Xiaolong Wang, Runsen Xu, Zuofan Cui, Zeyu Wan, Yu Zhang
Neural Information Processing Systems, NeurIPS 2023
[Paper] [Code] [Demo] [Cite]
RNIN-VIO
RNIN-VIO: Robust Neural Inertial Navigation Aided Visual-Inertial Odometry in Challenging Scenes
Danpeng Chen, Nan Wang, Runsen Xu, Weijian Xie, Hujun Bao, Guofeng Zhang
International Symposium on Mixed and Augmented Reality, ISMAR 2021, Oral Presentation
[Paper] [Code] [Project] [Cite]

Selected Awards

  • Hong Kong PhD Fellowship (HKPFS), 2022
  • CUHK Vice-Chancellor’s PhD Scholarship, 2022
  • Outstanding Graduates of Zhejiang University, 2022
  • Outstanding Undergraduate Thesis of College of Computer Science and Technology, Zhejiang University, 2022
  • National Scholarship, 2019

Teaching

  • IERG4998: Final Year Project, Spring 2023
  • IERG4998: Final Year Project, Fall 2022