Skip to main content
SpringerLink
Log in

Cognitive mechanism related to line drawings and its applications in intelligent process of visual media: a survey

  • Review Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Line drawings, as a concise form, can be recognized by infants and even chimpanzees. Recently, how the visual system processes line-drawings attracts more and more attention from psychology, cognitive science and computer science. The neuroscientific studies revealed that line drawings generate similar neural actions as color photographs, which give insights on how to efficiently process big media data. In this paper, we present a comprehensive survey on line drawing studies, including cognitive mechanism of visual perception, computational models in computer vision and intelligent process in diverse media applications. Major debates, challenges and solutions that have been addressed over the years are discussed. Finally some of the ensuing challenges in line drawing studies are outlined.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Fischetti M. Computers vs. brains. Scientific American, 2011, 305(5): 104–104

    Article  Google Scholar 

  2. Fu X L, Cai L H, Liu Y, Jia J, Chen WF, Zhang Y, Zhao GZ, Liu YJ, Wu C X. A computational cognition model of perception, memory and judgment. Science China Information Sciences, 2014, 57(3): 1–15

    Article  Google Scholar 

  3. Ullman S. High-level vision: Object recognition and visual cognition. Cambridge: MIT press, 2000

    MATH  Google Scholar 

  4. Liu Y J, Fu Q F, Liu Y, Fu X L. A distributed computational cognitive model for object recognition. Science China Information Sciences, 2013, 56(9): 1–13

    Article  Google Scholar 

  5. Riddoch M, Humphreys G. Object Recognition. In Rapp B, ed. Handbook of Cognitive Neuropsychology. Hove: Psychology Press

  6. Canny J. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986 (6): 679–698

    Google Scholar 

  7. Cole F, Golovinskiy A, Limpaecher A, Barros H S, Finkelstein A, Funkhouser T, Rusinkiewicz S. Where do people draw lines? ACM Transactions on Graphics, 2008, 27(3): 88

    Article  Google Scholar 

  8. Sayim B, Cavanagh P. What Line Drawings Reveal About the Visual Brain. Frontiers in Human Neuroscience, 2011, 5

    Google Scholar 

  9. Yonas A, Arterberry M E. Infants perceive spatial structure specified by line junctions. Perception. 1994, 23(12): 1427–1435

    Article  Google Scholar 

  10. Itakura S. Recognition of line-drawing representations by a chimpanzee (Pan troglodytes). The Journal of general psychology, 1994, 121(3): 189–197

    Article  Google Scholar 

  11. Cole F, Sanik K, DeCarlo D, Finkelstein A, Funkhouser T, Rusinkiewicz S, Singh M. How well do line drawings depict shape? ACM Transactions on Graphics, 2009, 28(3): 28

    Article  Google Scholar 

  12. Koenderink J J, Van Doorn A J, Kappers A M L. Surface perception in pictures. Perception & Psychophysics, 1992, 52(5): 487–496

    Article  Google Scholar 

  13. Koenderink J J, van Doorn A J, Kappers A M L, Todd J T. Ambiguity and the “mental eye” in pictorial relief. Perception, 2001, 30(4): 431–448

    Article  Google Scholar 

  14. Marr D, Vision A. A Computational Investigation into the Human Representation and Processing of Visual Information. Cambridge: MIT Press, 2010

    Book  Google Scholar 

  15. Watt R J, Rogers B J. Human vision and cognitive science. Research Directions in Cognitive Science: A European Perspective, 1989, 1: 9–22

    Google Scholar 

  16. Sonka M, Hlavac V, Boyle R. Image processing, analysis, and machine vision. Cengage Learning, 2014

    Google Scholar 

  17. Biederman I. Recognition-by-components: a theory of human image understanding. Psychological review, 1987, 94(2): 115

    Article  Google Scholar 

  18. Biederman I, Ju G. Surface versus edge-based determinants of visual recognition. Cognitive Psychology, 1988, 20(1): 38–64

    Article  Google Scholar 

  19. Fu Q, Liu Y J, Chen W, Fu X. The time course of natural scene categorization in human brain: simple line-drawings vs. color photographs. Journal of Vision, 2013, 13(9): 1060

    Article  Google Scholar 

  20. Del Viva M M, Punzi G, Benedetti D. Information and Perception of Meaningful Patterns. PLoS One, 2013, 8(7): e69154

    Article  Google Scholar 

  21. Morgan M J. Features and the ‘primal sketch’. Vision Research, 2011, 51(7): 738–753

    Article  Google Scholar 

  22. Delorme A, Richard G, Fabre-Thorpe M. Key visual features for rapid categorization of animals in natural scenes. Frontiers in Psychology, 2010, 1: 21

    Google Scholar 

  23. Naber M, Hilger M, Einhäuser W. Animal detection and identification in natural scenes: image statistics and emotional valence. Journal of Vision, 2012, 12(1): 25

    Article  Google Scholar 

  24. Derrington A M, Krauskopf J, Lennie P. Chromatic mechanisms in lateral geniculate nucleus of macaque. The Journal of Physiology, 1984, 357(1): 241–265

    Article  Google Scholar 

  25. Campbell F W, Robson J G. Application of Fourier analysis to the visibility of gratings. The Journal of physiology, 1968, 197(3): 551

    Article  Google Scholar 

  26. Joubert O R, Rousselet G A, Fabre-Thorpe M, Fize D. Rapid visual categorization of natural scene contexts with equalized amplitude spectrum and increasing phase noise. Journal of Vision, 2009, 9(1): 2

    Article  Google Scholar 

  27. Jolicoeur P. The time to name disoriented natural objects. Memory & Cognition, 1985, 13(4): 289–303

    Article  Google Scholar 

  28. Farah M J, Hammond K M. Mental rotation and orientation-invariant object recognition: dissociable processes. Cognition, 1988, 29(1): 29–46

    Article  Google Scholar 

  29. Westheimer G. The Fourier theory of vision. Perception, 2001, 30(5): 531–542

    Article  Google Scholar 

  30. Oppenheim A V, Lim J S. The importance of phase in signals. In: Proceedings of the IEEE. 1981, 69(5): 529–541

    Article  Google Scholar 

  31. Piotrowski L N, Campbell F W. A demonstration of the visual importance and flexibility of spatial-frequency amplitude and phase. Perception, 1982, 11(3): 337–346

    Article  Google Scholar 

  32. Guyader N, Chauvin A, Peyrin C, Hérault J, Marendaz C. Image phase or amplitude? Rapid scene categorization is an amplitude-based process. Comptes Rendus Biologies, 2004, 327(4): 313–318

    Article  Google Scholar 

  33. Chen W F, Liang J, Liu Y J, Fu Q F, Fu X L. Rapid natural scene categorization of line drawings is less influenced by amplitude spectra: evidence from a subliminal perception study. ASSC 8: Poster Session

  34. Morrone M C, Burr D C. Feature detection in human vision: a phasedependent energy model. In: Proceedings of the Royal Society of London. 1988, 235(1280): 221–245

    Article  Google Scholar 

  35. Devlin H, Tracey I, Johansen-Berg H, Clare S. What is Functional Magnetic Resonance Imaging (fMRI)? FMRIB Centre, Department of Clinical Neurology, University of Oxford

  36. Walther D B, Caddigan E, L F F, Beck D M. Natural scene categories revealed in distributed patterns of activity in the human brain. The Journal of Neuroscience, 2009, 29(34): 10573–10581

    Article  Google Scholar 

  37. Walther D B, Chai B, Caddigan E, Beck DM, Li F F. Simple line drawings suffice for functional MRI decoding of natural scene categories. In: Proceedings of the National Academy of Sciences, 2011, 108(23): 9661–9666

    Article  Google Scholar 

  38. Kim S G, Richter W, Uǧurbil K. Limitations of temporal resolution in functional MRI. Magnetic Resonance in Medicine, 1997, 37(4): 631–636

    Article  Google Scholar 

  39. Vanrullen R, Thorpe S J. The time course of visual processing: from early perception to decision-making. Journal of Cognitive Neuroscience, 2001, 13(4): 454–461

    Article  Google Scholar 

  40. Johnson J S, Olshausen B A. Timecourse of neural signatures of object recognition. Journal of Vision, 2003, 3(7): 4

    Article  Google Scholar 

  41. Fu Q F, Liu Y J, Dienes Z, Wu J H, Chen W F, Fu X L. Different time courses of natural scene categorization of color photographs and line-drawings: evidence from event-related potentials. Submitted for publication, 2014

    Google Scholar 

  42. Liu Y J, Ma C X, Fu Q, Fu X L, Qin S F, Xie L X. A Sketch-Based Approach for Interactive Organization of Video Clips. ACM Transactions on Multimedia Computing, Communications, and Applications, 2014, 11(1)

    Google Scholar 

  43. Snodgrass J G, Vanderwart M. A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory, 1980, 6(2): 174

    Google Scholar 

  44. Magnié M N, Besson M, Poncet M, Dolisi C. The Snodgrass and Vanderwart set revisited: norms for object manipulability and for pictorial ambiguity of objects, chimeric objects, and nonobjects. Journal of Clinical and Experimental Neuropsychology, 2003, 25(4): 521–560

    Article  Google Scholar 

  45. Rossion B, Pourtois G. Revisiting Snodgrass and Vanderwart’s object pictorial set: the role of surface detail in basic-level object recognition. Perception, 2004, 33(2): 217–236

    Article  Google Scholar 

  46. Viggiano M P, Vannucci M, Righi S. A new standardized set of ecological pictures for experimental and clinical research on visual object processing. Cortex, 2004, 40(3): 491–509

    Article  Google Scholar 

  47. Abel S, Weiller C, Huber W, Willmes K. Neural underpinnings for model-oriented therapy of aphasic word production. Neuropsychologia, 2014, 57: 154–165

    Article  Google Scholar 

  48. Janssen N, Carreiras M, Barber H A. Electrophysiological effects of semantic context in picture and word naming. Neuroimage, 2011, 57(3): 1243–1250

    Article  Google Scholar 

  49. Schnur T T. The persistence of cumulative semantic interference during naming. Journal of Memory and Language, 2014, 75: 27–44

    Article  Google Scholar 

  50. Strijkers K, Holcomb P J, Costa A. Conscious intention to speak proactively facilitates lexical access during overt object naming. Journal of Memory and Language, 2011, 65(4): 345–362

    Article  Google Scholar 

  51. Kang H, Lee S, Chui C. Coherent line drawing. In: Proceedings of 5th International Symposium on Non-photorealistic Animation and Rendering. 2007, 43–50

    Chapter  Google Scholar 

  52. Arbel T, Ferrie F P. Viewpoint selection by navigation through entropy maps. In: Proceedings of the 7th IEEE International Conference on Computer Vision. 1999, 1: 248–254

    Article  Google Scholar 

  53. Laporte C, Arbel T. Efficient discriminant viewpoint selection for active bayesian recognition. International Journal of Computer Vision, 2006, 68(3): 267–287

    Article  Google Scholar 

  54. Secord A, Lu J, Finkelstein A, Sing M, Nealen A. Perceptual models of viewpoint preference. ACM Transactions on Graphics, 2011, 30(5): 109

    Article  Google Scholar 

  55. Chen D Y, Tian X P, Shen Y T, Ouhyoung M. On visual similarity based 3D model retrieval. Computer graphics forum. Publishing, 2003, 22(3): 223–232

    Google Scholar 

  56. Cyr C M, Kimia B B. 3D object recognition using shape similiarity based aspect graph. In: Proceedings the 8th IEEE International Conference on Computer Vision. 2001, 1: 254–261

    Google Scholar 

  57. Liu Y J, Luo X, Joneja A, Ma C X, Fu X L, Song D W. User-adaptive sketch-based 3-D CAD model retrieval. IEEE Transactions on Automation Science and Engineering, 2013, 10(3): 783–795

    Article  Google Scholar 

  58. Duda R O, Hart P E, Stork D G. Pattern Classification. John Wiley & Sons, 1999.

    MATH  Google Scholar 

  59. Chalechale A, Naghdy G, Mertins A. Sketch-based image matching using angular partitioning. IEEE Transactions on Systems, Man and Cybernetics, 2005, 35(1): 28–41

    Article  Google Scholar 

  60. Eitz M, Richter R, Boubekeur T, Hildebrand K, Alexa M. Sketch-based shape retrieval. ACM Transactions on Graphics, 2012, 31(4): 31

    Google Scholar 

  61. Hung C C, Carlson E T, Connor C E. Medial axis shape coding in macaque inferotemporal cortex. Neuron, 2012, 74(6): 1099–1113

    Article  Google Scholar 

  62. Lescroart M D, Biederman I. Cortical representation of medial axis structure. Cerebral Cortex, 2013, 23(3): 629–637

    Article  Google Scholar 

  63. Li Z, Qin S, Jin X. Skeleton-enhanced line drawings for 3D models. Graphical Models, 2014, 76(6): 620–632

    Article  Google Scholar 

  64. Lake B M, Salakhutdinov R, Tenenbaum J. One-shot learning by inverting a compositional causal process. Advances in neural information processing systems. 2013: 2526–2534

    Google Scholar 

  65. Ma C X, Liu Y J, Yang H Y, Teng D X, Wang H A, Dai G Z. KnitSketch: a sketch pad for conceptual design of 2D garment patterns. IEEE Transactions on Automation Science and Engineering, 2011, 8(2): 431–437

    Article  Google Scholar 

  66. Sivic J, Zisserman A. Video Google: A text retrieval approach to object matching in videos

  67. Baeza-Yates R, Ribeiro-Neto B. Modern Information Retrieval. New York: ACM Press, 1999

    Google Scholar 

  68. Sugihara K. Machine Interpretation of Line Drawings. Cambridge: MIT Press, 1986

    Google Scholar 

  69. Hoffman D D. Visual Intelligence: How We Create What We See. New York: W.W. Norton & Company, 2000

    Google Scholar 

  70. Chen T, Cheng M M, Tan P, Ariel S, Hu S M. Sketch2Photo: internet image montage.ACM Transactions on Graphics, 2009, 28(5): 124

    Google Scholar 

  71. Cao Y, Wang C H, Zhang L Q, Zhang L. Edgel index for large-scale sketch-based image search. IEEE Conference on Computer Vision and Pattern Recognition, 2011: 761–768

    Google Scholar 

  72. Truong B T, Venkatesh S. Video abstraction: a systematic review and classification. ACM Transactions on Multimedia Computing, Communications, and Applications, 2007, 3(1): 3

    Article  Google Scholar 

  73. Ma C X, Liu Y J, Wang H A, Teng D X, Dai G Z. Sketch-based annotation and visualization in video authoring. IEEE Transactions on Multimedia, 2012, 14(4): 1153–1165

    Article  Google Scholar 

  74. Collomosse J P, McNeill G, Qian Y. Storyboard sketches for content based video retrieval. In: Proceedings of the 12th IEEE International Conference on Computer Vision, 2009, 245–252

    Google Scholar 

  75. Igarashi T, Matsuoka S, Tanaka H. Teddy: a sketching interface for 3D freeform design. In: Proceedings of ACM SIGGRAPH Courses, 2007: 21

    Google Scholar 

  76. Nealen A, Igarashi T, Sorkine O, Alexa M. FiberMesh: designing freeform surfaces with 3D curves. ACM Transactions on Graphics, 2007, 26(3): 41

    Article  Google Scholar 

  77. Liu Y J, Ma C X, Zhang D L. EasyToy: plush toy design using editable sketching curves. IEEE Computer Graphics and Applications, 2011, 31(2): 49–57

    Article  MathSciNet  Google Scholar 

  78. Zhu X, Jin X, Liu S, Zhoo H. Analytical solutions for sketch-based convolution surface modeling on the GPU. The Visual Computer, 2012, 28(11): 1115–1125

    Article  Google Scholar 

  79. Yu C C, Liu Y J, Wu T F, Li K Y, Fu X L. A global energy optimization framework for 2.1D sketch extraction from monocular images. Graphical Models, 2014, 76(5): 507–521

    Article  Google Scholar 

  80. Liu Y J, Zhang J B, Hou J C, Ren J C. Cylinder detection in large-scale point cloud of pipeline plant. IEEE Transactions on Visualization and Computer Graphics, 2013, 19(10): 1700–1707

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China

    Yongjin Liu & Minjing Yu

  2. State Key Laboratory of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, 100101, China

    Qiufang Fu, Wenfeng Chen & Ye Liu

  3. Research School of Computer Science, The Australian National University, Canberra, 0200, Australia

    Lexing Xie

Authors
  1. Yongjin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Minjing Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiufang Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenfeng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ye Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lexing Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yongjin Liu.

Additional information

Yongjin Liu received his BE from Tianjin University, China in 1998, and his PhD from Hong Kong University of Science and Technology, China in 2004. He is now an associate professor with Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, China. His research interests include pattern recognition, computer graphics, computational geometry and computer-aided design. He is a member of IEEE, a member of IEEE Computer Society and IEEE Communications Society.

Minjing Yu received her BE from Wuhan University, China, and she is now a PhD student at Department of Computer Science and Technology, Tsinghua University, China. Her research interests include image processing, computer graphics and cognitive science.

Qiufang Fu received her PhD from Institute of Psychology, Chinese Academy of Sciences, in 2006. She is now an associate professor in Institute of Psychology, Chinese Academy of Sciences, China, with interests in implicit learning and unconscious knowledge. She intends to explore the neural and cognitive mechanisms responsible for the dissociation of implicit and explicit processes.

Wenfeng Chen received his BS from Peking University, China in 1999, and his PhD from Institute of Psychology, Chinese Academy of Sciences, China in 2005. He is now an associate professor of psychology in the Institute of Psychology, Chinese Academy of Sciences, and a guest Associate Editor of Frontiers in Psychology. He is interested in face learning and recognition, visual attention, emotion and cognitive modelling.

Ye Liu received her PhD from Institute of Psychology, Chinese Academy of Sciences, China in 2005. She is now an associate professor of psychology in the Institute of Psychology, Chinese Academy of Sciences. She is interested in semantic knowledge representation and semantic processing, metaphor comprehension, and affective computing.

Lexing Xie is Senior Lecturer in the Research School of Computer Science at the Australian National University, Australia. She was research staff member at IBM T.J. Watson Research Center in New York, USA from 2005 to 2010, and adjunct assistant professor at Columbia University, USA from 2007 to 2009. She received BS from Tsinghua University, China, and MS and PhD degrees from Columbia University, all in Electrical Engineering. Her research interests include applied machine learning, multimedia and social media. Her research has received five best student paper and best paper awards between 2002 and 2011. She currently serves as an associate editor of IEEE Transactions on Multimedia, and ACM Transactions on Multimedia Computing, Communications and Applications.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y., Yu, M., Fu, Q. et al. Cognitive mechanism related to line drawings and its applications in intelligent process of visual media: a survey. Front. Comput. Sci. 10, 216–232 (2016). https://doi.org/10.1007/s11704-015-4450-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-015-4450-1

Keywords

Search

Navigation