Jinkun Cao

I am a Robotics PhD student at CMU with Kris Kitani since 2020.

I received bachelor degree from Shanghai Jiao Tong University advised by Cewu Lu. I visited UC Berkeley in 2019, advised by Trevor Darrell. I also visited Yang Gao's group at Tsinghua University in 2020. I have interned in Nvidia Research, Adobe Research, Meta Reality Labs, Tencent and Bytedance AI Lab etc. My research is supported by Meta PhD Fellowship since 2023.

I am on the job market 2024-25!

Github | Google Scholar | Linkedin | DBLP | Resume (usually out-dated)
Email: jinkuncao [AT] gmail.com / jinkunc [AT] andrew.cmu.edu

Research Interest

I am interested in modeling the human motion from interacting with humans, objects and scenes.

Motion and interaction cause visual occlusion and body deformation/articulation. I study how to estimate human motion and the body pose and shape from visual input with noise and ambiguity. I am also interested in building inclusive temporal prior for human motion.
Visual observations are projected from our physical world. I study combining physics-based tools and visual models to enhance the plausibility and fidelity of human motion modeling. I am also interested in the resulting animation and robotics tasks.

News

[2024/9]: Three papers accepted to NeurIPS 2024 and one paper accepted to NeurIPS DB Track 2024. See you in Vancouver! (Hopefully I could get a visa)
[2024/3]: Concluded my tracking research in a thesis. My recent research is more about the generation of human shape, motion and behavior.
[2024/2]: SimXR for pose estimation and simulation from head-mounted cameras is accepted by CVPR 2024. Congrats to Zen!
[2024/2]: CSC-Tracker for multi-object tracking is accepted by ICRA 2024. Stay tuned for more details!
[2024/1]: Two papers are accepted by ICLR as Spotlights. Check UniHSI and PULSE for details!
[2023/7]: One paper about humanoid control is accpeted to ICCV. Congrats, Zen! Looking forward to the trip to Paris.
[2023/4]: Awarded Meta PhD Research Fellowship since 2023. Thank you Meta!
[2023/2]: Deep OC-SORT is available on arxiv and Github, ranking 1st on MOT17, MOT20 and DanceTrack among published papers.
[2023/2]: Two papers are accepted to CVPR 2023 (including OC-SORT). See you at Vancouver.
[2022/9]: A paper is accepted to BMVC'2022 for multi-object tracking. The paper is coming to the public soon.
[2022/9]: MED paper is accepted by NeurIPS'2022. We study the disentanglement property of high-dimensional representation models and introduce contrastive learning methods into disentanglement benchmarks. Stay tuned for a heavily revised veresion of paper.
[2022/8]: OC-SORT is supported by mmtracking now. Try it for more flexible and advanced features!
[2022/3]: The code of OC-SORT is released. It achieves SOTA performance on multiple MOT datasets in a pure motion-based fashion.
[2022/3]: We are organizing "Multiple Object Tracking in Complex Environments Workshop” in ECCV'2022, Tel Aviv, Israel.
[2022/2]: DanceTrack is accepted in CVPR'2022. We propose a challenging multi-object tracking dataset.

Selected Publications

* indicates equal contribution | I am a main contributor of the highlighted projects.

Grasping Diverse Objects with Simulated Humanoids

Zhengyi Luo*, Jinkun Cao*, Sammy Christen, Alexander Winkler, Kris Kitani, Weipeng Xu

NeurIPS 2024 [project]

Mixed Gaussian Flow for Diverse Trajectory Prediction

Jiahe Chen*, Jinkun Cao*, Kris Kitani, Jiangmiao Pang

NeurIPS 2024 [arxiv]

CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics

Jiawei Gao, Ziqin Wang, Zeqi Xiao, Jingbo Wang, Tai Wang, Jinkun Cao, Xiaolin Hu, Si Liu, Jifeng Dai, Jiangmiao Pang

NeurIPS 2024 (spotlight) [project]

Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions

Rawal Khirodkar*, Jyun-Ting Song*, Jinkun Cao, Zhengyi Luo, Kris Kitani

NeurIPS 2024 (Dataset and Benchmark Track) [project]

Real-Time Simulated Avatar from Head-Mounted Sensors

Zhengyi Luo, Jinkun Cao, Rawal Khirodkar, Alexander Winkler, Jing Huang, Kris Kitani, Weipeng Xu

CVPR 2024 [arxiv] [project]

Multi-Object Tracking by Hierarchical Visual Representations

Jinkun Cao, Jiangmiao Pang, Kris Kitani

ICRA 2024 [arxiv]

Universal Humanoid Motion Representations for Physics-Based Control

Zhengyi Luo, Jinkun Cao, Josh Merel, Alexander Winkler, Jing Huang, Kris Kitani, Weipeng Xu

ICLR 2024 (Spotlight) [arxiv] [project]

Unified Human-Scene Interaction via Prompted Chain-of-Contacts

Zeqi Xiao, Tai Wang, Jingbo Wang, Jinkun Cao, Wenwei Zhang, Bo Dai, Dahua Lin, Jiangmiao Pang

ICLR 2024 (Spotlight) [arxiv] [project]

Perpetual Humanoid Control for Real-time Simulated Avatars

Zhengyi Luo, Jinkun Cao, Alexander Winkler, Kris Kitani, Weipeng Xu

ICCV 2023 [project page] [arxiv] [code]

Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking

Jinkun Cao, Jiangmiao Pang, Xinshuo Weng, Rawal Khirodkar, Kris Kitani

CVPR 2023 [arxiv] [code] [mmtracking]

Track Targets by Dense Spatio-Temporal Position Encoding

Jinkun Cao, Hao Wu, Kris Kitani

BMVC 2022 [Oral] [arxiv]

An Empirical Study on Disentanglement of Negative-free Contrastive Learning

Jinkun Cao, Ruiqian Nai, Qing Yang, Jialei Huang, Yang Gao

NeurIPS 2022 [arxiv] [code]

DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion

Peize Sun*, Jinkun Cao*, Yi Jiang, Zehuan Yuan, Song Bai, Kris Kitani, Ping Luo

CVPR 2022 [arxiv] [code] [project page] [codalab]

Instance-aware predictive navigation in multi-agent environments

Jinkun Cao, Xin Wang, Trevor Darrell, Fisher Yu

ICRA 2021 [arxiv] [code]

Cross-Domain Adaptation for Animal Pose Estimation

Jinkun Cao, Hongyang Tang, Hao-Shu Fang, Xiaoyong Shen, Cewu Lu, Yu-Wing Tai

ICCV 2019 [Oral] (4.3% acceptance rate) [arxiv] [dataset]

Manuscripts

* indicates equal contribution | I am a main contributor of the highlighted projects.

Humanoidlympics: Sports Environments for Physically Simulated Humanoids

Zhengyi Luo*, Jiashun Wang*, Kangni Liu*, Haotian Zhang, Chen Tessler, Jingbo Wang, Ye Yuan, Jinkun Cao, Zihui Lin, Fengyi Wang, Jessica Hodgins, Kris Kitani

Technical Report, 2024 [project]

Multi-Modal Hand-Object Interaction Generation

Jinkun Cao, Jingyuan Liu, Kris Kitani, Yi Zhou

Technical Report, 2024 [arxiv]

TransTrack: Multiple Object Tracking with Transformer

Peize Sun, Jinkun Cao, Yi Jiang, Rufeng Zhang, Enze Xie, Zehuan Yuan, Changhu Wang, Ping Luo

Technical Report, 2021 [arxiv] [code]

Services

Conference Reviewer:

Computer Vision: ICCV (21, 23), ECCV (20, 22, 24), CVPR (22, 23, 24), ISMAR (23, 24)
Robotics: ICRA (22, 23, 24), IROS (22)
Machine Learning: NeurIPS (22, 23), AAAI (22, 23, 24), ICML (23, 24), ICLR (24)

Journal Reviewer: RA-L, IEEE Trans. Multimedia, TNNLS, TCSVT, Pattern Recoginition, TMLR, IJCV
Workshop Organizer:
- "Multiple Object Tracking in Complex Environments Workshop” in ECCV'2022