Video cloning for paintings via artistic style transfer

Abstract

In the past, visual arts usually represented the static art like paintings, photography and sculptures. In recent years, many museums, artwork galleries, and even art exhibitions demonstrated dynamic artworks for visitors to relish. The most famous dynamic artwork is “The moving painting of Along the River During the Qingming Festival”. Nevertheless, it took 2 years to complete this work. They had to plan each action for every character at first, then drew each video frame by animators. Finally, it could achieve seamless stitching by using lots of projectors to render scene on the screen. In our research, we develop a method for generating animated paintings. It only needs a number of videos on a network of existing databases and requires users to perform some simple auxiliary operations to achieve the effect of animation synthesis. First, our system lets users select an object with the same class from the first video frame. We then employ random forests as learning algorithm to retrieve from a video the object which users want to insert into an artwork. Second, we utilize style transferring, which enables the video frames to be consistent with the style of painting. At last, we use the seamless image cloning algorithm to yield seamless synthesizing result. Our approach allows different users to synthesize animating paintings up to their selected styled video frames. The resulting work not only maintains the original author’s painting style, but also generates a variety of artistic conception for people to enjoy.