Style Transfer and Image Transformation

Generative models can be used to map artificial images to a space of random numbers; they can also learn the mapping between one kind of image and another. This kind of model can, for example, be used to convert an image of a horse into that of a zebrahttps://www.tensorflow.org/tutorials/generative/images/horse2zebra_2. png , create deepfake videos in which one actor’s face has been replaced with another’s, or transform a photo into a paintingCycleGAN. TensorFlow Core. Retrieved April 26, 2021, from https://www. tensorflow.org/tutorials/generative/cyclegan .

Another fascinating example of applying generative modeling is a study in which lost masterpieces of the artist Pablo Picasso were discovered to have been painted over with another image. After X-ray imaging of The Old Guitarist indicated that earlier images of a woman and a landscape lay underneath, researchers used the other paintings from Picasso’s blue period or other color photographs to train a neural style transfer model that transforms black-and-white images (the X-ray radiographs of the overlying painting) to the coloration of the original artworkBourached, A., Cann, G. (2019). Raiders of the Lost Art. arXiv:1909.05677. https://arxiv.org/pdf/1909.05677.pdf . Then, applying this transfer model to the hidden images allowed them to reconstruct colored-in versions of the lost paintings.

In the figure below, deep learning was used to color in the X-ray images of the painted-over scenes (c), with ...