Xudong Xu

I am a third-year PhD student in Multimedia Laboratory in the Chinese University of Hong Kong. My supervisor is Prof. Dahua Lin. I work closely with Bo Dai, Hang Zhou and Ziwei Liu.

I recieved my B.E in Automation from Nanjing University in June 2018.

My research interests lie in the area of Computer Vision and Deep Learning. To be specific, I'm pariticularly interested in video and audio processing.

Email / LinkedIn / Google Scholar / Github


Aug. 2018 - Jul. 2022 (Expected), Department of Information Engineering, the Chinese University of Hong Kong

Ph.D Candidate


Sept. 2014 - Jun. 2018 , School of Management and Engineering, Nanjing University

Bachelor in Automation

Rank: 1 / 37


Visually Informed Binaural Audio Generation without Binaural Audios
Xudong Xu*, Hang Zhou*, Ziwei Liu, Bo Dai, Xiaogang Wang, Dahua Lin
In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2021
[paper] [code] [project]

We leverage spherical harmonic decomposition and head-related impulse response (HRIR) to create the pseudo pair of visual scenes and binaural audios, which can be used to guide the training of binaural audio generation.


Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation
Hang Zhou*, Xudong Xu*, Dahua Lin, Xiaogang Wang, Ziwei Liu
In Proceedings of the European Conference on Computer Vision (ECCV), 2020
[paper] [code] [project]

We propose to integrate the task of stereophonic audio generation and audio source separation into a unified framework namely Sep-Stereo, which leverages vastly available mono audios to facilitate the training of stereophonic audio generation.


Vision-Infused Deep Audio Inpainting
Hang Zhou, Ziwei Liu, Xudong Xu, Ping Luo, Xiaogang Wang
In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019
[paper] [code] [project]

We proposed a VIAI-AV framework for novel task, vision-infused audio inpainting, and a new instrument-playing dataset called MUSICES.


Recursive Visual Sound Separation Using Minus-Plus Net
Xudong Xu, Bo Dai, Dahua Lin
In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019
[paper] [code] [project]

We proposed a recursive Minus-Plus network for visual sound separation task.


A Novel DDPG Method with Prioritized Experience Replay
Yuenan Hou, Lifeng Liu, Qing Wei, Xudong Xu, Chunlin Chen
IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2017
[paper] [code]

We proposed a prioritized experience replay method for the DDPG algorithm, where prioritized sampling is adopted instead of uniform sampling.

Academic Activities
  • Serve as reviewer for BMVC 2019, CVPR 2020, NeurIPS 2020, CVPR 2021, ICML 2021, ICCV 2021.

Honors and Awards
  • Postgraduate Scholarship, the Chinese University of Hong Kong, 2018 ~ now

  • Outstanding Graduate of Nanjing University, 2018

  • Zhenggang Scholarship, Nanjing University, 2017

  • National Scholarship, Nanjing University, 2016

  • National Outstanding Award, China Education Robot Contest, 2015

  • The First Prize, National High School Mathematics Contest, Jiangxi, China, 2013