Graph Convolution

Overview

While we have many options for data representations in 3D, it would often be ideal to work with meshes directly. Much of the 3D art world depends on 3D meshes, so AI techniques that can manipulate them directly will be simpler and more efficient. By treating 3D meshes as graph data, we can use some of the concepts from graph neural networks on 3D data.

Graph convolution

Graph convolution is a concept drawn from graph theory. For a graph consisting of nodes $N$ and edges $E$, it defines a convolution operator $f(N, E)$ that combines vertex position and optional features from a vertex $i$ and its set of neighbors $N(i)$. It essentially is a generalization of convolutional layers where connections can be irregular and/or asymmetric. We can think of the convolutional layers seen in computer vision, for example, as a special case of graph convolution, where nodes are densely and bidirectionally connected to adjacent nodes.

Graph convolutions have been used repeatedly for analyzing 3D data in techniques such as Pixel2Mesh, MeshCNN, and Mesh R-CNN. Various techniques utilize graph convolutions to classify 3D meshes, segment 3D meshes into ...