High-quality 3D Generation from Single-view Images

PhD Thesis Proposal Defence


Title: "High-quality 3D Generation from Single-view Images"

by

Miss Zifan SHI


Abstract:

Generative models have demonstrated notable advancements recently, particularly 
in the realms of 2D and video synthesis. However, evident inconsistencies, such 
as those related to lighting and geometry, persist in 2D and video generation. 
The inclusion of 3D modeling holds the potential to enhance the coherence and 
realism of 2D and video generation, urging the need for advancements in 3D 
generation. Given the challenges associated with collecting a huge amount of 3D 
data for direct generative modeling, a practical approach to 3D generation 
involves learning 3D distributions from single-view images. This approach is 
viable due to the availability of abundant, unstructured, high-quality, and 
diverse single-view image data. A common strategy for 3D generation from 
single-view images is the adoption of generative adversarial networks (GANs), 
with the generator being replaced by a 3D renderer. This thesis delves into the 
domain of 3D generation from three perspectives. We first look into the 
generated geometry and propose an enhancement of the learned geometry by 
injecting 3D awareness not only to the generator but also to the discriminator. 
Moreover, we analyze the pose requirements for the training of 3D generative 
models and free the generator from the constraints of pose priors, resulting in 
a more flexible 3D generative model. Finally, in the context of complex scene 
synthesis, an analysis of the shortcomings in existing methods is presented, 
along with a proposal to leverage 3D priors to facilitate 3D modeling from 
single-view scene images. In the end, we will discuss the potential future 
directions in 3D generation.


Date:                   Monday, 25 March 2024

Time:                   10:00am - 12:00noon

Venue:                  Room 5501
                        Lifts 25/26

Committee Members:      Prof. Dit-Yan Yeung (Supervisor)
                        Dr. Qifeng Chen (Supervisor)
                        Prof. Chi-Keung Tang (Chairperson)
                        Dr. Long Chen