Zineng Tang
ABSTRACT
The paper proposes a deep end-to-end model with full differentiability, FilterNet, for image enhancement that could optimize the image filter parameters and recommend the best photo filter for a given image to achieve optimum aesthetic effect. The model learns the aesthetic distribution of images from evaluation network that is pretrained and identifies the binary aesthetic quality of an image, similar to the structure of GAN (generative adversarial network). The proposed model is weakly-supervised and one stage and accompanied by new training methods compared to other state of art models and conditional inputs that help the model to be more content-aware, yielding competitive results compared to professional photo editing. The performance of FilterNet is evaluated on both deep learning and traditional methods including online user studies.
- S. Bhattacharya, R. Sukthankar, and M. Shah. A framework for photo-quality assessment and enhancement based on visual aesthetics. In Proceedings of the 18th ACM international conference on Multimedia, pages 271--280. ACM, 2010. Google Scholar
Digital Library
- J.-Y. Lee, K. Sunkavalli, Z. Lin, X. Shen, and I. So Kweon. Automatic content-aware color and tone stylization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2470--2478, 2016.Google ScholarCross Ref
- W.-T. Sun, T.-H. Chao, Y.-H. Kuo, and W. H. Hsu. Photo filter recommendation by category-aware aesthetic learning. IEEE Transactions on Multimedia, 19(8):1870--1880, 2017.Google Scholar
- H. S. Faridul, T. Pouli, C. Chamaret, J. Stauder, A. Trémeau, E. Reinhard, et al. A survey of color mapping and its applications. Eurographics (State of the Art Reports), 3, 2014.Google Scholar
- A. Gupta, J. Johnson, A. Alahi, and L. Fei-Fei. Characterizing and improving stability in neural style transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4067--4076, 2017.Google ScholarCross Ref
- X. Huang and S. J. Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In ICCV, pages 1510--1519, 2017.Google ScholarCross Ref
- J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision, pages 694--711. Springer, 2016.Google ScholarCross Ref
- Y. Deng, C. C. Loy, and X. Tang. Aesthetic-driven image enhancement by adversarial learning. arXiv preprint arXiv:1707.05251, 2017.Google Scholar
- Y. Ke, X. Tang, and F. Jing. The design of high-level features for photo quality assessment. In Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, volume 1, pages 419--426. IEEE, 2006. Google ScholarDigital Library
- W. Luo, X. Wang, and X. Tang. Content-based photo quality assessment. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 2206--2213. IEEE, 2011. Google ScholarDigital Library
- N. Murray, L. Marchesotti, and F. Perronnin. Ava: A large-scale database for aesthetic visual analysis. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 2408--2415. IEEE, 2012. Google ScholarDigital Library
- X. Lu, Z. Lin, H. Jin, J. Yang, and J. Z. Wang. Rapid: Rating pictorial aesthetics using deep learning. In Proceedings of the 22nd ACM international conference on Multimedia, pages 457--466. ACM, 2014. Google ScholarDigital Library
- X. Jin, L. Wu, Z. He, S. Chen, J. Chi, S. Peng, X. Li, and S. Ge. Efcient deep aesthetic image classication using connected local and global features. arXiv preprint arXiv:1610.02256, 2016.Google Scholar
- N. Murray and A. Gordo. A deep architecture for unied aesthetic prediction. arXiv preprint arXiv:1708.04890, 2017.Google Scholar
- S. Kong, X. Shen, Z. Lin, R. Mech, and C. Fowlkes. Photo aesthetics ranking network with attributes and content adaptation. In European Conference on Computer Vision, pages 662--679. Springer, 2016.Google ScholarCross Ref
- P. S. Chandakkar, V. Gattupalli, and B. Li. A computational approach to relative aesthetics. arXiv preprint arXiv:1704.01248, 2017.Google Scholar
- Y. Kao, R. He, and K. Huang. Deep aesthetic quality assessment with semantic information. IEEE Transactions on Image Processing, 26(3):1482--1495, 2017. Google ScholarDigital Library
- Z. Wang, D. Liu, S. Chang, Q. Ling, Y. Yang, and T. S. Huang. D3: Deep dual-domain based fast restoration of jpeg-compressed images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2764--2772, 2016.Google ScholarCross Ref
- C. Dong, C. C. Loy, K. He, and X. Tang. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2):295--307, 2016. Google ScholarDigital Library
- K. He, J. Sun, and X. Tang. Single image haze removal using dark channel prior. IEEE transactions on pattern analysis and machine intelligence, 33(12):2341--2353, 2011. Google ScholarDigital Library
- Z. Yan, H. Zhang, B. Wang, S. Paris, and Y. Yu. Automatic photo adjustment using deep neural networks. ACM Transactions on Graphics (TOG), 35(2):11, 2016. Google ScholarDigital Library
- J. Yan, S. Lin, S. Bing Kang, and X. Tang. A learning-to-rank approach for image color enhancement. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2987--2994, 2014. Google ScholarDigital Library
- Y. Hwang, J.-Y. Lee, I. So Kweon, and S. Joo Kim. Color transfer using probabilistic moving least squares. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3342--3349, 2014. Google ScholarDigital Library
- A. Ignatov, N. Kobyshev, R. Timofte, K. Vanhoey, and L. Van Gool. Dslr-quality photos on mobile devices with deep convolutional networks. In the IEEE Int. Conf. on Computer Vision (ICCV), 2017.Google ScholarCross Ref
- I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672--2680, 2014. Google ScholarDigital Library
- K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778, 2016.Google ScholarCross Ref
- B. Dai, S. Fidler, R. Urtasun, and D. Lin. Towards diverse and natural image descriptions via a conditional gan. arXiv preprint arXiv:1703.06029, 2017.Google Scholar
Index Terms
- Adaptive Aesthetic Photo Filter by Deep Learning
Recommendations
Photo Filter Recommendation by Category-Aware Aesthetic Learning
Nowadays, social media has become a popular platform for the public to share photos. To make photos more visually appealing, users usually apply filters on their photos without domain knowledge. However, due to the growing number of filter types, it ...
The Aesthetic and the Poietic Elements of Information Design
IV '10: Proceedings of the 2010 14th International Conference Information VisualisationIn this paper I address two types of perspectives on the aesthetic that are of relevance for a discussion of contemporary information design. Firstly, the ’aesthetic’ understood as aesthetic perception of beautiful form. Secondly, the ’aesthetic’ ...
Aesthetic Photo Collage With Deep Reinforcement Learning
Photo collage aims to automatically arrange multiple photos on a given canvas with high aesthetic quality. Existing methods are based mainly on handcrafted feature optimization, which cannot adequately capture high-level human aesthetic senses. Deep ...
Comments