Jinkun Cao

I am a Research Scientist at Meta Superintelligence Lab, MSL (previously at FAIR).

I was a Robotics PhD student at CMU with Kris Kitani. I received bachelor degree from Shanghai Jiao Tong University advised by Cewu Lu. I visited UC Berkeley in 2019, advised by Trevor Darrell. I also visited Yang Gao's group at Tsinghua University in 2020. I have interned in Nvidia Research, Adobe Research, Meta Reality Labs, Tencent and Bytedance AI Lab etc. My research was supported by Meta PhD Fellowship.

Github | Google Scholar | Linkedin | DBLP | Resume (usually out-dated)
Email: jinkuncao [AT] gmail.com / jinkunc [AT] andrew.cmu.edu

Research Interest

I focus on studying the real2sim and sim2real problems to model and imitate human body and hand motions, especially in interaction scenarios.

Estimation and generation of human hand pose and motion from noisy visual input.
Combining physics-based simulations and vision for more plausible human hand interaction imitation.
Applications in AR/VR, animation and robotic tasks to replicate 3D and physical interactions.

News

[2025/10]: GENMO was accepted at ICCV as a Highlight and I would be at ICCV, Honolulu to present.
[2025/7]: I passed my PhD thesis defense on "Estimating and Generating Human Motions from Interactions". Thesis is here.

—— See Older News ——

[2024/9]: Three papers accepted to NeurIPS 2024 and one paper accepted to NeurIPS DB Track 2024. See you in Vancouver! (Hopefully I could get a visa)

[2024/3]: Concluded my tracking research in a thesis. My recent research is more about the generation of human shape, motion and behavior.

[2024/2]: SimXR for pose estimation and simulation from head-mounted cameras is accepted by CVPR 2024. Congrats to Zen!

[2024/2]: CSC-Tracker for multi-object tracking is accepted by ICRA 2024. Stay tuned for more details!

[2024/1]: Two papers are accepted by ICLR as Spotlights. Check UniHSI and PULSE for details!

[2023/7]: One paper about humanoid control is accpeted to ICCV. Congrats, Zen! Looking forward to the trip to Paris.

[2023/4]: Awarded Meta PhD Research Fellowship since 2023. Thank you Meta!

[2023/2]: Deep OC-SORT is available on arxiv and Github, ranking 1st on MOT17, MOT20 and DanceTrack among published papers.

[2023/2]: Two papers are accepted to CVPR 2023 (including OC-SORT). See you at Vancouver.

[2022/9]: A paper is accepted to BMVC'2022 for multi-object tracking. The paper is coming to the public soon.

[2022/9]: MED paper is accepted by NeurIPS'2022. We study the disentanglement property of high-dimensional representation models and introduce contrastive learning methods into disentanglement benchmarks. Stay tuned for a heavily revised veresion of paper.

[2022/8]: OC-SORT is supported by mmtracking now. Try it for more flexible and advanced features!

[2022/3]: The code of OC-SORT is released. It achieves SOTA performance on multiple MOT datasets in a pure motion-based fashion.

[2022/3]: We are organizing "Multiple Object Tracking in Complex Environments Workshop” in ECCV'2022, Tel Aviv, Israel.

[2022/2]: DanceTrack is accepted in CVPR'2022. We propose a challenging multi-object tracking dataset.

Selected Publications

* indicates equal contribution | I am a main contributor of the highlighted projects.

GENMO: A GENarlist Model for Human MOtion

Jiefeng Li, Jinkun Cao, Haotian Zhang, Davis Rampe, Jan Kautz, Umar Iqbal, and Ye Yuan

ICCV 2025 [Highlight][project]

Grasping Diverse Objects with Simulated Humanoids

Zhengyi Luo*, Jinkun Cao*, Sammy Christen, Alexander Winkler, Kris Kitani, Weipeng Xu

NeurIPS 2024 [project]

Mixed Gaussian Flow for Diverse Trajectory Prediction

Jiahe Chen*, Jinkun Cao*, Kris Kitani, Jiangmiao Pang

NeurIPS 2024 [arxiv]

Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions

Rawal Khirodkar*, Jyun-Ting Song*, Jinkun Cao, Zhengyi Luo, Kris Kitani

NeurIPS 2024 (Dataset and Benchmark Track) [project]

Real-Time Simulated Avatar from Head-Mounted Sensors

Zhengyi Luo, Jinkun Cao, Rawal Khirodkar, Alexander Winkler, Jing Huang, Kris Kitani, Weipeng Xu

CVPR 2024 [arxiv] [project]

Multi-Object Tracking by Hierarchical Visual Representations

Jinkun Cao, Jiangmiao Pang, Kris Kitani

ICRA 2024 [arxiv]

Universal Humanoid Motion Representations for Physics-Based Control

Zhengyi Luo, Jinkun Cao, Josh Merel, Alexander Winkler, Jing Huang, Kris Kitani, Weipeng Xu

ICLR 2024 (Spotlight) [arxiv] [project]

Perpetual Humanoid Control for Real-time Simulated Avatars

Zhengyi Luo, Jinkun Cao, Alexander Winkler, Kris Kitani, Weipeng Xu

ICCV 2023 [project page] [arxiv] [code]

Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking

Jinkun Cao, Jiangmiao Pang, Xinshuo Weng, Rawal Khirodkar, Kris Kitani

CVPR 2023 [arxiv] [code] [mmtracking]

Track Targets by Dense Spatio-Temporal Position Encoding

Jinkun Cao, Hao Wu, Kris Kitani

BMVC 2022 [Oral] [arxiv]

An Empirical Study on Disentanglement of Negative-free Contrastive Learning

Jinkun Cao, Ruiqian Nai, Qing Yang, Jialei Huang, Yang Gao

NeurIPS 2022 [arxiv] [code]

DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion

Peize Sun*, Jinkun Cao*, Yi Jiang, Zehuan Yuan, Song Bai, Kris Kitani, Ping Luo

CVPR 2022 [arxiv] [code] [project page] [codalab]

Instance-aware predictive navigation in multi-agent environments

Jinkun Cao, Xin Wang, Trevor Darrell, Fisher Yu

ICRA 2021 [arxiv] [code]

Cross-Domain Adaptation for Animal Pose Estimation

Jinkun Cao, Hongyang Tang, Hao-Shu Fang, Xiaoyong Shen, Cewu Lu, Yu-Wing Tai

ICCV 2019 [Oral] (4.3% acceptance rate) [arxiv] [dataset]

Manuscripts

* indicates equal contribution | I am a main contributor of the highlighted projects.

Humanoidlympics: Sports Environments for Physically Simulated Humanoids

Zhengyi Luo*, Jiashun Wang*, Kangni Liu*, Haotian Zhang, Chen Tessler, Jingbo Wang, Ye Yuan, Jinkun Cao, Zihui Lin, Fengyi Wang, Jessica Hodgins, Kris Kitani

Technical Report, 2024 [project]

Multi-Modal Hand-Object Interaction Generation

Jinkun Cao, Jingyuan Liu, Kris Kitani, Yi Zhou

Technical Report, 2024 [arxiv]

TransTrack: Multiple Object Tracking with Transformer

Peize Sun, Jinkun Cao, Yi Jiang, Rufeng Zhang, Enze Xie, Zehuan Yuan, Changhu Wang, Ping Luo

Technical Report, 2021 [arxiv] [code]

Services

Conference Reviewer:

Computer Vision: ICCV (21, 23), ECCV (20, 22, 24), CVPR (22, 23, 24), ISMAR (23, 24)
Robotics: ICRA (22, 23, 24), IROS (22)
Machine Learning: NeurIPS (22, 23), AAAI (22, 23, 24), ICML (23, 24), ICLR (24)

Journal Reviewer: RA-L, IEEE Trans. Multimedia, TNNLS, TCSVT, Pattern Recoginition, TMLR, IJCV
Workshop Organizer:
- "Multiple Object Tracking in Complex Environments Workshop” in ECCV'2022