Abstract
As the smartphones are widely used in daily life, people are habituated to take photos in anywhere at anytime. The character photos, which contain both people and landscapes, are the most extensive ones in all kinds of photos. However, it is usually time-consuming and laborious for people to select and manage desired character photos manually. Therefore, automatic photo selection became extremely significant. Most of the existing methods select best character photos by assigning a absolute score or binary label, regardless of the photo’s content. The selected character photos are hence not satisfied. In this paper, we propose an effective automatic framework to select best character photos, which first automatically eliminates photos with unattractive character, then ranks photos and assists users to select the desired ones. Moreover, our framework is especially useful for mobile camera in practical application scenarios. To reduce the burden of photo selection and improve the efficiency of automatic selection from a large number of photos on mobile platform, we divide the photo selection into two stages: eliminating of unattractive photos based on designed efficient and effective features related to faces and human postures, and ranking the remaining photos using a two-stream convolution neural network. To ensure that informative features are selected and less useful ones are suppressed, we design and utilize the attention mechanism module in network. Experiments demonstrate the effectiveness of our method on the automatic selection of the satisfied photos from a large number of photos.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abaza, A., Harrison, M.A., Bourlai, T., Ross, A.: Design and evaluation of photometric image quality measures for effective face recognition. IET Biom. 3(4), 314–324 (2014)
BaiduAI: BaiduAI open platform. https://ai.baidu.com. Accessed 17 Oct 2020
Best-Rowden, L., Jain, A.K.: Learning face image quality from human assessments. IEEE Trans. Inf. Forensics Secur. 13(12), 3064–3077 (2018)
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and regression trees (cart). Biometrics 40(3), 358 (1984)
Bromley, J., et al.: Signature verification using a “Siamese” time delay neural network. Int. J. Pattern Recogn. Artif. Intell. 7(04), 669–688 (1993)
Cavalcanti, C.S.V.C., Gomes, H.M., De Queiroz, J.E.R.: A survey on automatic techniques for enhancement and analysis of digital photography. J. Braz. Comput. Soc. 19(3), 341–359 (2013). https://doi.org/10.1007/s13173-013-0102-1
Chang, H., Yu, F., Wang, J., Ashley, D., Finkelstein, A.: Automatic triage for a photo series. ACM Trans. Graph. (TOG) 35(4), 1–10 (2016)
Choi, J., Lee, B.J., Zhang, B.T.: Human body orientation estimation using convolutional neural network. arXiv preprint arXiv:1609.01984 (2016)
Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006). https://doi.org/10.1007/11744078_23
Deng, Y., Loy, C.C., Tang, X.: Image aesthetic assessment: an experimental survey. IEEE Signal Process. Mag. 34(4), 80–106 (2017)
Dong, Z., Shen, X., Li, H., Tian, X.: Photo quality assessment with DCNN that understands image well. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015. LNCS, vol. 8936, pp. 524–535. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-14442-9_57
Dong, Z., Tian, X.: Multi-level photo quality assessment with multi-view features. Neurocomputing 168, 308–319 (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision & Pattern Recognition (2016)
Hernandez-Ortega, J., Galbally, J., Fierrez, J., Haraksim, R., Beslay, L.: FaceQnet: quality assessment for face recognition based on deep learning. In: 2019 International Conference on Biometrics (ICB), pp. 1–8. IEEE (2019)
Hosu, V., Goldlucke, B., Saupe, D.: Effective aesthetics prediction with multi-level spatially pooled features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9375–9383 (2019)
Kong, S., Shen, X., Lin, Z., Mech, R., Fowlkes, C.: Photo aesthetics ranking network with attributes and content adaptation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 662–679. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_40
Kuzovkin, D., Pouli, T., Cozot, R., Le Meur, O., Kervec, J., Bouatouch, K.: Image selection in photo albums. In: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, pp. 397–404 (2018)
Li, C., Gallagher, A., Loui, A.C., Chen, T.: Aesthetic quality assessment of consumer photos with faces. In: 2010 IEEE International Conference on Image Processing, pp. 3221–3224. IEEE (2010)
Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J.Z.: Rapid: rating pictorial aesthetics using deep learning. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 457–466 (2014)
Mohan, J., Philumon, J.: Automatic selection of better image from a series of same scene. Int. J. Adv. Res. Comput. Sci. 8(5) (2017)
Newbury, R., Cosgun, A., Koseoglu, M., Drummond, T.: Learning to take good pictures of people with a robot photographer. arXiv preprint arXiv:1904.05688 (2019)
Quinlan, J.R.: Induction on decision tree. Mach. Learn. 1 (1986)
Sain, S.R.: The nature of statistical learning theory. Technometrics 38(4), 409 (1997)
Talebi, H., Milanfar, P.: NIMA: neural image assessment. IEEE Trans. Image Process. 27(8), 3998–4011 (2018)
Tong, H., Li, M., Zhang, H.-J., He, J., Zhang, C.: Classification of digital photos taken by photographers or home users. In: Aizawa, K., Nakamura, Y., Satoh, S. (eds.) PCM 2004. LNCS, vol. 3331, pp. 198–205. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30541-5_25
Wang, Y., Ke, Y., Wang, K., Zhang, C., Qin, F.: Aesthetic quality assessment for group photograph. arXiv preprint arXiv:2002.01096 (2020)
Woo, S., Park, J., Lee, J.Y., So Kweon, I.: CBAM: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)
Zhu, J.Y., Agarwala, A., Efros, A.A., Shechtman, E., Wang, J.: Mirror mirror: crowdsourcing better portraits. ACM Trans. Graph. (TOG) 33(6), 1–12 (2014)
Acknowledgement
We thank the anonymous reviewers for helpful suggestions. This work is supported by National Natural Science Foundation of China under Grant 61802142.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, C., Yu, X., Sun, C. (2021). Character Photo Selection for Mobile Platform. In: Nguyen, M., Yan, W.Q., Ho, H. (eds) Geometry and Vision. ISGV 2021. Communications in Computer and Information Science, vol 1386. Springer, Cham. https://doi.org/10.1007/978-3-030-72073-5_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-72073-5_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-72072-8
Online ISBN: 978-3-030-72073-5
eBook Packages: Computer ScienceComputer Science (R0)