Kwan-Yee Lin

kwanyee_lin.jpg

I am a Research Fellow in the EECS department at the University of Michigan, Ann Arbor, working with Stella X. Yu. Previously, I was a Postdoctoral Researcher in MMLab at The Chinese University of Hong Kong, working with Xiaogang Wang and Hongsheng Li. I was a Director of R&D at Intelligent Automotive Group of SenseTime Group Inc., working with Chen Qian. I received my PhD in the Department of Information Science, School of Mathematical Sciences at Peking University.

Research

My research interests lie at the intersection of Computer Vision, Computer Graphics, Robotics, and Machine Learning. My long-term goal is to build Realistic and Intelligent Virtual Humans -- embodied human-like systems that not only resemble humans, but also emulate humans' remarkable cognitive and motor capabilities, enabling them to interact with the world with self-autonomy. I view and approach "Virtual Humans" from two complementary perspectives: (1) Virtual Avatars, which focus on digital representations of humans. (2) Humanoid Robots, which serve as the embodied carriers to connect perception to action in the physical world. My research has thus three themes:

  • Realistic Virtual Humans: Constructing high-fidelity 4D volumetric capture systems and datasets, as in DNA-Rendering and RenderMe-360; modeling realistic human appearances and motions, as in MonoHuman, GAR, and TimeWalker; learning context-aware behaviour generation, as in EmbodiedHuman; and developing human foundation models to obtain generic representations, as in CosmicMan.
  • All-round Humanoid Robots: Building the synergistic relations of visual perception, proprioception, and action in embodied agents to develop integrative skills autonomously, as in LEGO-H.
  • Steerable Scene Constrution: Developing steerable scene modeling and simulation frameworks, as a playground, to potentially enhance perception, navigation, and interaction in complex environments, as in UrbanRF (DNMP), UrbanArchitect, and SLR-SFS.

News

Mar, 2025 We are now organizing the 2nd POETS Workshop on Embodied Humans – Symbiotic Intelligence Between Virtual Humans and Humanoid Robots @ CVPR 2025!
Dec, 2024 2 papers with 1 highlight presentation got accepted by CVPR/ECCV 2024.
Mar, 2024 We are now organizing the first POETS workshop on Virtual Humans for Robotics and Autonomous Driving @ CVPR 2024!
Sep, 2023 The part-2 of DNA-Rendering dataset is released.
Sep, 2023 RenderMe-360, a large-scale dataset for high-fidelity head avatar rendering, is accepted by NeuIPS 2023! 🔥
Jul, 2023 6 papers with 1 oral presentations got accepted by CVPR/ICCV 2023.
Sep, 2022 We are starting OpenXDLab, a new large-scale open-source project for datasets of XR research! 🔥
Jul, 2022 3 papers with 1 oral presentations got accepted by CVPR/ECCV 2022.
Mar, 2021 2 papers with 1 oral presentations got accepted by CVPR 2021.
Jul, 2020 2 papers got accepted by CVPR/ECCV in 2020.

Selected Publications

  1. Let Humanoid Robots Go Hiking! Integrative Skill Development over Complex Trails
    Kwan-Yee Lin, and Stella X. Yu
    Computer Vision and Pattern Recognition (CVPR), 2025
    Project Page
  2. CosmicMan: A Text-to-Image Foundation Model for Humans
    Shikai Li, Jianglin Fu, Kaiyuan Liu, Wentao Wang,  Kwan-Yee Lin †, and Wayne Wu
    Computer Vision and Pattern Recognition (CVPR), 2024
    YouTube Project Page
  3. RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars
    Dongwei Pan,  etal., and Kwan-Yee Lin
    Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks, 2023
    arXiv YouTube Project Page
  4. DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering
    Wei Cheng,  etal., and Kwan-Yee Lin
    International Conference on Computer Vision (ICCV), 2023
    arXiv YouTube Project Page
  5. Urban Radiance Field Representation with Deformable Neural Mesh Primitives
    Fan Lu, Yan Xu, Guang Chen, Hongsheng Li,  Kwan-Yee Lin †, and Changjun Jiang
    International Conference on Computer Vision (ICCV), 2023
    arXiv YouTube Project Page
  6. Simulating Fluids in Real-World Still Images
    Siming Fan, Jingtan Piao, Chen Qian, Hongsheng Li, and Kwan-Yee Lin
    International Conference on Computer Vision (ICCV), 2023 (Oral)
    arXiv YouTube Project Page
  7. Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis
    Wei Cheng, Su Xu, Jingtan Piao, Chen Qian, Wayne Wu,  Kwan-Yee Lin †, and Hongsheng Li
    Technical report, arXiv:2204.11798, 2022
    arXiv YouTube Project Page
  8. StyleGAN-Human: A Data-Centric Odyssey of Human Generation
    Jianglin Fu, Shikai Li, Yuming Jiang,  Kwan-Yee Lin, Chen Qian, Chen Change Loy, Wayne Wu, and Ziwei Liu
    European Conference on Computer Vision (ECCV), 2022
    arXiv YouTube Project Page
  9. Semantic Scene Completion via Integrating Instances and Scene in-the-Loop
    Yingjie Cai, Xuesong Chen, Chao Zhang,  Kwan-Yee Lin †, Xiaogang Wang, and Hongsheng Li
    Conference on Computer Vision and Pattern Recognition (CVPR), 2021
    arXiv Project Page
  10. 3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior
    Xiaokang Chen,  Kwan-Yee Lin, Chen Qian, Gang Zeng, and Hongsheng Li
    Conference on Computer Vision and Pattern Recognition (CVPR), 2020
    arXiv Project Page
  11. Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimation
    Kwan-Yee Lin, Xipeng Chen, Wentao Liu, Chen Qian, and Liang Lin
    Conference on Computer Vision and Pattern Recognition (CVPR), 2019 (Oral)
    arXiv Project Page