Xudong Xu     徐旭东

I am currently a researcher at Shanghai Artificial Intelligence Laboratory, working on Content Generation and Digitization. I recieved my Ph.D. degree from Multimedia Laboratory in the Chinese University of Hong Kong, advised by Prof. Dahua Lin. I recieved my B.E in Automation from Nanjing University in June 2018.

My current research interests lie in 3D content generation, especially focusing on geometry refinement and PBR material generation, for photorealistic 3D object and scene creation. I am looking for highly motivated interns at Shanghai AI Laboratory. Shoot me an email (xuxudong@pjlab.org.cn) if you are interested.

LinkedIn / Google Scholar / Github

Aug. 2018 - Mar. 2023, Department of Information Engineering, the Chinese University of Hong Kong

Ph.D. in Information Engineering

Sept. 2014 - Jun. 2018 , School of Management and Engineering, Nanjing University

Bachelor in Automation

GPA: 92.4/100, Rank: 1 / 37

June 2022 - Nov. 2022, Meta Reality Labs, Pittsburgh

MATLABER: Material-Aware Text-to-3D via LAtent BRDF auto-EncodeR
Xudong Xu, Yitong Wang, Zhaoyang Lyu, Xingang Pan, Bo Dai
ArXiv, 2023
[paper] [code] [project]

We propose Material-Aware Text-to-3D via LAtent BRDF auto-EncodeR (MATLABER) that leverages a novel latent BRDF auto-encoder for material generation, enabling photorealistic 3D object generation, relighting, and material editing.

Sounding Bodies: Modeling 3D Spatial Sound of Humans Using Body Pose and Audio
Xudong Xu, Dejan Markovic, Jacob Sandakly, Todd Keebler, Steven Krenn, Alexander Richard
Advances in Neural Information Processing Systems (NeurIPS), 2023
[paper] [code] [dataset]

We present a model that can generate accurate 3D spatial audio for full human bodies. To this end, we collect a first-of-its-kind multimodal dataset of human bodies, recorded with multiple cameras and a spherical array of 345 microphones.

Generative Occupancy Fields for 3D Surface-Aware Image Synthesis
Xudong Xu, Xingang Pan, Dahua Lin, Bo Dai
Advances in Neural Information Processing Systems (NeurIPS), 2021
[paper] [code] [project]

We propose Generative Occupancy Fields(GOF), a 3D-aware generative model that could synthesize realistic images with 3D consistency and simultaneously learn compact object surfaces.

Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation
Hang Zhou*, Xudong Xu*, Dahua Lin, Xiaogang Wang, Ziwei Liu
In Proceedings of the European Conference on Computer Vision (ECCV), 2020
[paper] [code] [project]

We propose to integrate the task of stereophonic audio generation and audio source separation into a unified framework namely Sep-Stereo, which leverages vastly available mono audios to facilitate the training of stereophonic audio generation.