Search⌘ K
AI Features
Home
Courses
Hands-On Generative Adversarial Networks with PyTorch

Designing GANs for 3D Data Synthesis

Explore the design and training of 3D-GANs to generate 3D point clouds from latent vectors. Understand the architecture of generator and discriminator networks using 3D convolution layers, and learn practical training methods including soft labels and learning rate decay. Gain hands-on experience with 3D data synthesis for various object categories using PyTorch.

We'll cover the following...

3D-GANWu, Jiajun, Chengkai Zhang, Tianfan Xue, Bill Freeman, and Josh Tenenbaum. "Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling." Advances in neural information processing systems 29 (2016). was designed to generate a 3D point cloud of certain types of objects. The design and training process of 3D-GAN is very similar to the vanilla GAN, except that the input and output tensors of the 3D-GAN are five-dimensional rather than four-dimensional.

Generators and discriminators in 3D-GAN

The architecture of the generator network of 3D-GAN is as follows:

Architecture of the generator network in 3D-GAN
Architecture of the generator network in 3D-GAN

The generator network consists of five transposed convolution layers (nn.ConvTranspose3d), in which the first four layers are followed by the batch normalization layer (nn.BatchNorm3d) and ReLU activation function, and the last layer is followed by a sigmoid activation function. The kernel size, stride size, and padding size are set to 4, 2, and 1 in all the transposed convolution layers, respectively. Here, the input latent vector can be gradually expanded to a 1×32×32×321 \times 32 \times 32 \times 32 cube, which can be considered as a 1-channel 3D image. In this 3D image, the pixel value is actually the possibility of whether a point exists at these 32×32×3232 \times 32\times 32 grid locations. Normally, we reserve all the ...