Artistic Image Colorization with Visual Generative Networks

Visual generative models, such as Generative Adversarial Networks (GANs) [1] and Variational Autoencoders (VAEs) [2], have achieve remarkable results in generating visual images [3, 4, 5, 6]. While most existing work [3, 4] focus on photorealistic images, the problem of generating artistic images is relatively underinvestigated. Different from photorealistic images, artistic images exhibit larger variations in color, visual style and emotion. Therefore, it is challenging for generative models, to capture the richer space of artistic visual domain. In this project, we aim to design visual generative models for the problem of artistic image colorization. We would like to explore multiple settings of colorizing artistic images of different styles. We are interested in the following settings. First, given a gray-scale input image, we expect our system to automatically generate vivid color scheme of the input. Second, given a colorful input image, we would like the generated color scheme to follow user control. To enable this, besides input gray-scale image, our system takes as input one additional k × k color grid, where user can specify color spatially. Moreover, we would like to evaluate our system on various visual styles/media types, for example oil painting and water color, which are both extremely rich in color. The overall design of our systems is illustrated in Figure 1.