Skip to main content
SpringerLink
Log in

Line drawing via saliency map and ETF

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Line drawing is a means of superior visual communication which can effectively convey useful information to viewers. Artists usually draw what they see rather than what exists, which means the most attractive object is emphasized while the rest are inhibited. Moreover, artists draw the whole object with coherent lines instead of fractured lines, which also contribute to the outstanding visual effect. From these perspectives, generating line drawings with saliency and coherence remains a great challenge. Existing line drawing generation methods ignore these important properties and cannot generate coherent lines in some cases since they do not take into account how artists draw a picture. To this end, a novel saliency-aware line drawing method was proposed to better grasp the viewer’s attention on an image. First, a saliency enhanced line extraction method combining saliency map and edge tangent flow was proposed to ensure the saliency and coherence of lines, especially in salient but low contrast areas. Then, the salient information was highlighted while irrelevant details were eliminated by inhibiting lines in less salient areas to enhance the saliency of the line drawing. Finally, the transparency of lines was adjusted to further emphasize important information. Various results showed that our method can generate line drawings with better visual saliency and coherence than the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Son M, Kang H, Lee Y, Lee S. Abstract line drawings from 2D images. In: Proceedings of IEEE Pacific Conference on Computer Graphics and Applications. 2007, 333–342

  2. Zhou J, Li B. Automatic generation of pencil-sketch like drawings from personal photos. In: Proceedings of IEEE International Conference on Multimedia and Expo. 2005, 1026–1029

  3. He J, Wang S, Zhang Y, Zhang J. A computational fresco sketch generation framework. In: Proceedings of IEEE International Conference on Multimedia and Expo. 2013, 1–6

  4. Choi J, Cho H, Song J, Yoon S M. Sketchhelper: real-time stroke guidance for freehand sketch retrieval. IEEE Transactions on Multimedia, 2019, 21(8): 2083–2092

    Article  Google Scholar 

  5. Hata M, Toyoura M, Mao X. Automatic generation of accentuated pencil drawing with saliency map and LIC. Visual Computer, 2012, 28(6–8): 657–668

    Article  Google Scholar 

  6. Doug D, Anthony S. Stylization and abstraction of photographs. ACM Transactions on Graphics, 2002, 21(3): 769–776

    Article  Google Scholar 

  7. Winnemoeller H, Xie J, Hertzmann A, Li W. Portraitsketch: face sketching assistance for novices. In: Proceedings of Annual ACM Symposium on User Interface Software and Technology. 2014, 407–417

  8. Wu Y T, Yeh J S, Wu F C, Chuang Y Y. Tangent-based binary image abstraction. Journal of Imaging, 2017, 3(2): 16–30

    Article  Google Scholar 

  9. Kang H, Lee S, Chui C K. Flow-based image abstraction. IEEE Transactions on Visualization and Computer Graphics, 2009, 15(1): 62–76

    Article  Google Scholar 

  10. Orzan A, Bousseau A, Barla P, Thollot J. Structurepreserving manipulation of photographs. In: Proceeding of International Symposium on Non-Photorealistic Animation and Rendering. 2007, 103–110

  11. Rosin P L, Lai Y K. Artistic minimal rendering with lines and blocks. Graphical Models, 2013, 75(4): 208–229

    Article  Google Scholar 

  12. Kang H, Lee S, Chui C K. Coherent line drawing. In: Proceedings of International Symposium on NonPhotorealistic Animation and Rendering. 2007, 43–50

  13. Wang S, Wu E, Liu Y, Liu X, Chen Y. Abstract line drawings from photographs using flow-based filters. Computers and Graphics, 2012, 36(4): 224–231

    Article  Google Scholar 

  14. Li M, Lin Z, Mech R, Yumer E, Ramanan D. Photosketching: inferring contour drawings from images. In: Proceedings of Winter Conference on Applications of Computer Vision. 2019, 1–10

  15. Liu Y, Cheng M M, Hu X, Wang K, Bai X. Richer convolutional features for edge detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 3000–3009

  16. Xie S, Tu Z. Holistically-nested edge detection. International Journal of Computer Vision, 2017, 125(1): 3–18

    Article  MathSciNet  Google Scholar 

  17. Sheng B, Li P, Gao C, Ma K L. Deep neural representation guided face sketch synthesis. IEEE Transactions on Visualization and Computer Graphics, 2019, 25(12): 3216–3230

    Article  Google Scholar 

  18. Kang H W, Chui C K, Chakraborty U K. A unified scheme for adaptive stroke-based rendering. Visual Computer, 2006, 22: 9–11

    Article  Google Scholar 

  19. Gooch B, Reinhard E, Gooch A. Human facial illustrations: creation and psychophysical evaluation. ACM Transactions on Graphics, 2004, 23(1): 27–44

    Article  Google Scholar 

  20. Nie Y, Cao X, Li P, Zhang Q, Zhang Z, Li G, Sun H. Interactive contour extraction via sketch-alike densevalidation optimization. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(4): 906–916

    Article  Google Scholar 

  21. Wang S, Ao Z, Yao S, Chen Y, Wang G. Adaptive line drawing. Journal of Image and Graphics, 2018, 23(5): 730–739

    Google Scholar 

  22. Zhang J, Wani R Z, Xu D. Automatic generation of sketch-like pencil drawing from image. In: Proceedings of IEEE International Conference on Multimedia and Expo. 2017, 261–266

  23. Li R, Sun S, Yang X, Hu W. Automatic generation of pencil drawings based on multi-level saliency maps. Journal of Computer-Aided Design and Computer Graphics, 2018, 30(1): 108–116

    Article  Google Scholar 

  24. Alexe B, Deselaers T, Ferrari V. Measuring the objectness of image windows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2189–2202

    Article  Google Scholar 

  25. Yan Q, Xu L, Shi J, Jia J. Hierarchical saliency detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2013, 1155–1162

  26. Gu K, Wang S, Yang H, Lin W, Zhai G, Yang X, Member S, Zhang W. Saliency-guided quality assessment of screen content images. IEEE Transactions on Multimedia, 2016, 18(6): 1098–1100

    Article  Google Scholar 

  27. Huang Z, Liu S. Perceptual hashing with visual content understanding for reduced-reference screen content image quality assessment. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(7): 2808–2823

    Article  Google Scholar 

  28. Gao R, Huang Z, Liu S. Multi-task deep learning for no-reference screen content image quality assessment. In: Proceedings of the 27th International Conference on Multimedia Modeling. 2021, 213–226

  29. Jerripothula K R, Cai J, Yuan J. Image co-segmentation via saliency cofusion. IEEE Transactions on Multimedia, 2016, 18(9): 1896–1909

    Article  Google Scholar 

  30. Quan R, Han J, Member S, Zhang D, Nie F, Qian X, Li X. Unsupervised salient object detection via inferring from imperfect saliency models. IEEE Transactions on Multimedia, 2017, 20(5): 1101–1112

    Article  Google Scholar 

  31. Zhao Y, Liu S, Hu Z, Bai Y, Shen C, Shi X. Separate degree based otsu and signed similarity driven level set for segmenting and counting anthrax spores. Computers and Electronics in Agriculture, 2020, 169: 1–15

    Article  Google Scholar 

  32. Qian X, Wang H, Zhao Y, Hou X, Hong R, Wang M, Tang Y Y. Image location inference by multisaliency enhancement. IEEE Transactions on Multimedia, 2017, 19(4): 813–821

    Article  Google Scholar 

  33. Fang Y, Lin W, Member S, Lee B S, Lau C T, Chen Z, Lin C W. Bottom-up saliency detection model based on human visual sensitivity and amplitude spectrum. IEEE Transactions on Multimedia, 2012, 14(1): 187–198

    Article  Google Scholar 

  34. Imamoglu N, Lin W, Fang Y. A saliency detection model using low-level features based on wavelet transform. IEEE Transactions on Multimedia, 2012, 14(1): 187–198

    Google Scholar 

  35. Yang C, Zhang L, Lu H, Ruan X, Yang M H. Saliency detection via graph-based manifold ranking. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2013, 3166–3173

  36. Achanta R, Hemami S, Estrada F, Susstrunk S. Frequency-tuned salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2009, 1597–1604

  37. Cabral B, Leedom L C. Imaging vector fields using line integral convolution. In: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques. 1993, 263–270

  38. Winnemller H, Gooch B, Olsen S C. Real-time video abstraction. ACM Transactions on Graphics, 2006, 25(3): 1221–1226

    Article  Google Scholar 

  39. Jin Y, Li P, Wang W, Zhang S, Lin D, Yin C. Ganbased pencil drawing learning system for art education on large-scale image datasets with learning analytics. Interactive Learning Environments, 2019

  40. Chen Z, Jin Y, Sheng B, Li P, Sun H. Parallel pencil drawing stylization via structure-aware optimization. In: Proceedings of the 31st International Conference on Computer Animation and Social Agents. 2018, 32–37

  41. Jin Y, Li P, Sheng B, Nie Y, Kim J, Wu E. SRNPD: spatial rendering network for pencil drawing stylization. Computer Animation and Virtual Worlds, 2019, 30(3–4): 1–13

    Google Scholar 

  42. Song Z, Liu S. Sufficient image appearance transfer combining color and texture. IEEE Transactions on Multimedia, 2019, 19(4): 702–711

    Article  Google Scholar 

  43. Liu S, Song Z, Zhang X, Zhu T. Progressive complex illumination image appearance transfer based on CNN. Journal of Visual Communication and Image Representation, 2019, 64: 1–11

    Article  Google Scholar 

  44. Zhu T, Liu S. Detail-preserving arbitrary style transfer. In: Proceedings of IEEE International Conference on Multimedia and Expo. 2020, 1–6

  45. Liu S, Zhu T. Structure-guided arbitrary style transfer for artistic image and video. IEEE Transactions on Multimedia, 2021

  46. Li P, Sun H, Sheng B, Shen J. Image stylization with enhanced structure on GPU. Science China Information Sciences, 2012, 55(5): 1093–1105

    Article  MathSciNet  Google Scholar 

  47. Wang R, Li P, Sheng B, Sun H, Wu E. Real-time video stylization using spatial-temporal gabor filtering. In: Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry. 2016, 299–307

Download references

Acknowledgements

This work was partly supported by the National Natural Science Foundation of China (Grant Nos. 62072328 and 61672375).

Author information

Authors and Affiliations

  1. College of Intelligence and Computing, Tianjin University, Tianjin, 300350, China

    Shiguang Liu & Ziqi Liu

Authors
  1. Shiguang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziqi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shiguang Liu.

Additional information

Shiguang Liu received the PhD degree from the State Key Laboratory of CAD CG, Zhejiang University, China. He is currently a Professor with the School of Computer Science and Technology, Tianjin University, China. His research interests include image/video processing, computer graphics, visualization and virtual reality.

Ziqi Liu got his BS degree and ME degree from the School of Computer Science and Technology, College of Intelligence and Computing, Tianjin University, China. His research interests include line drawing synthesis and computer graphics.

Electronic Supplementary Material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, S., Liu, Z. Line drawing via saliency map and ETF. Front. Comput. Sci. 16, 165707 (2022). https://doi.org/10.1007/s11704-021-1027-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-021-1027-z

Keywords

Search

Navigation