Xudong Xu     徐旭东

I am a final-year PhD student in Multimedia Laboratory in the Chinese University of Hong Kong. My supervisor is Prof. Dahua Lin. I work closely with Bo Dai, Xingang Pan, Hang Zhou and Ziwei Liu.

I recieved my B.E in Automation from Nanjing University in June 2018.

My research interests lie in the area of Computer Vision and Deep Learning. Specifically, I'm pariticularly interested in audio-visual joint learning, 3D-aware image synthesis, neural rendering.

Email / LinkedIn / Google Scholar / Github

Aug. 2018 - Mar. 2023 (Expected), Department of Information Engineering, the Chinese University of Hong Kong

PhD Candidate

Sept. 2014 - Jun. 2018 , School of Management and Engineering, Nanjing University

Bachelor of Automation

GPA: 92.4/100, Rank: 1 / 37

June 2022 - Nov. 2022, Meta Reality Labs, Pittsburgh

Generative Occupancy Fields for 3D Surface-Aware Image Synthesis
Xudong Xu, Xingang Pan, Dahua Lin, Bo Dai
Advances in Neural Information Processing Systems (NeurIPS), 2021
[paper] [code] [project]

We propose Generative Occupancy Fields(GOF), a 3D-aware generative model that could synthesize realistic images with 3D consistency and simultaneously learn compact object surfaces.

A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis
Xingang Pan, Xudong Xu, Chen Change Loy, Christian Theobalt, Bo Dai
Advances in Neural Information Processing Systems (NeurIPS), 2021
[paper] [code] [project]

We propose ShadeGAN, a novel shading-guided generative implicit model which could perform shape-accurate 3D-aware image synthesis by modeling shading in generative implicit models.

A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion
Zhaoyang Lyu*, Zhifeng Kong*, Xudong Xu, Liang Pan, Dahua Lin
International Conference on Learning Representations (ICLR), 2022
[paper] [code]

We propose a novel Point Diffusion-Refinement (PDR) paradigm for the point cloud completion task, which can generate both uniform and high-quality point clouds.

Visually Informed Binaural Audio Generation without Binaural Audios
Xudong Xu*, Hang Zhou*, Ziwei Liu, Bo Dai, Xiaogang Wang, Dahua Lin
In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2021
[paper] [code] [project]

We leverage spherical harmonic decomposition and head-related impulse response (HRIR) to create the pseudo pair of visual scenes and binaural audios, which can be used to guide the training of binaural audio generation.

Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation
Hang Zhou*, Xudong Xu*, Dahua Lin, Xiaogang Wang, Ziwei Liu
In Proceedings of the European Conference on Computer Vision (ECCV), 2020
[paper] [code] [project]

We propose to integrate the task of stereophonic audio generation and audio source separation into a unified framework namely Sep-Stereo, which leverages vastly available mono audios to facilitate the training of stereophonic audio generation.

Vision-Infused Deep Audio Inpainting
Hang Zhou, Ziwei Liu, Xudong Xu, Ping Luo, Xiaogang Wang
In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019
[paper] [code] [project]

We proposed a VIAI-AV framework for novel task, vision-infused audio inpainting, and a new instrument-playing dataset called MUSICES.

Recursive Visual Sound Separation Using Minus-Plus Net
Xudong Xu, Bo Dai, Dahua Lin
In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019
[paper] [code] [project]

We proposed a recursive Minus-Plus network for visual sound separation task.

A Novel DDPG Method with Prioritized Experience Replay
Yuenan Hou, Lifeng Liu, Qing Wei, Xudong Xu, Chunlin Chen
IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2017
[paper] [code]

We proposed a prioritized experience replay method for the DDPG algorithm, where prioritized sampling is adopted instead of uniform sampling.