Skip to main content
SpringerLink
Log in

An algorithm selection based platform for image understanding using high-level symbolic feedback and machine learning

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Natural image processing and understanding encompasses hundreds of different algorithms. Each algorithm generates best results for a particular set of input features and configurations of the objects/regions in the input image (environment). To obtain the best possible result of processing in a reliable manner, we propose an algorithm selection approach that selects the best algorithm for a each input image. The proposed algorithm selection starts by first selecting an algorithm using low level features such as color intensity, histograms, spectral coefficients or so and a user given context if available. The resulting high-level image description is analyzed for logical inconsistencies (contradictions) and image regions that must be processed using a different algorithm are selected. The high-level description and the optional user-given context are used by a Bayesian Network to estimate the cause of the error in the processing. The same Bayesian Network also generates new candidate algorithm for each region containing the contradiction in an iterative manner. This iterative selection stops when the high-level inconsistencies are all resolved or no more different algorithms can be selected. We also show that when inconsistencies can be detected, our framework is able to improve high-level description when compared with single algorithms. In order for such complex and iterative processing being computationally tractable we also introduce a hardware platform based on reconfigurable VLSI that is well suited as the platform of the proposed approach. We show that the algorithm selected approach is ideally suited for either a hybrid type VLSI processor or for a Logic-In-Memory processing platform.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Chau C-P, Siu W-C (2004) Adaptive dual-point hough transform for object recognition. J Comput Vision Image Underst 96:1–16

    Article  Google Scholar 

  2. Krivic J, Solina F (2004) Part-level object recognition using superquadrics. Int J Comput Vision Image Underst 95:105–126

    Article  Google Scholar 

  3. Kosecka J, Zhang W (2005) Extraction, matching, and pose recovery based on dominant rectangular structures. Int J Comput Vision Image Underst 100:274–293

    Article  Google Scholar 

  4. Leibe B, Leonardis A, Schiele B (2008) Robust object detection with interleaved categorization and segmentation. Int J Comput Vision 77:259–289

    Article  Google Scholar 

  5. Hoiem D, Efros AA, Hebert M (2008) Closing the loop on scene interpretation. In: Proc. computer vision and pattern recognition (CVPR)

  6. Heitz G, Gould S, Saxena A, Koller D (2008) Cascaded classification models: combining models for holistic scene understanding. In: Neural information processing systems (NIPS)

  7. Li L-J, Socher R, Fei-Fei L (2009) Towards total scene understanding: classification, annotation and segmentation in an automatic framework. In: Computer vision and pattern recognition (CVPR)

  8. Ladicky L, Russell C, Kohli P, Torr P (2010) Graph cut based inference with co-occurrence statistics. In: Proceedings of the 11th European conference on computer vision

  9. Brox T, Bourdev L, Maji S, Malik J (2011) Object segmentation by alignment of poselet activations to image contours”. In: IEEE international conference on computer vision and pattern recognition

  10. Ion A, Carreira J, Sminchisescu C (2011) Probabilistic joint image segmentation and labeling. In: 25th conference on neural information processing systems

  11. Carreira J, Li F, Sminchisescu C (2012) Object recognition by sequential figure-ground ranking. Int J Comput Vision 98(3):243–262

    Article  MathSciNet  Google Scholar 

  12. Matsuyama T (1987) Knowledge-based aerial image understanding systems and expert systems for image processing. IEEE Trans Geosci Remote Sens GE-25(3):305

    Google Scholar 

  13. Hotz L, Neumann B (2005) Scene interpretation as a configuration task. tech. rep

  14. Ferryman J, Borg M, Thirde D, Fusier F, Valentin V, Bremond F, Thonnat M, Aguilera J, Kampel M (2005) Automated scene understanding for airport aprons. In: Proceedings of 18th Australian joint conference on artificial intelligence. Springer, Sidney

  15. Perera AGA, Brooksby G, Hoogs A, Doretto G (2006) Moving object segmentation using scene understanding. In: Conference on computer vision and pattern recognition

  16. Kembhavi A, Yeh T, Davis L (2010) Why did the person cross the road (there)? Scene understanding using probabilistic logic models and common sense reasoning. In: Proceedings of the ECCV, part II, pp 693–706

  17. Bao SY, Sun M, Savarese S (2011) Toward coherent object detection and scene layout understanding. Image Vis Comput 29:569–579

    Article  Google Scholar 

  18. T.P.V.O.C. Homepage, The pascal visual object classes

  19. Arbelaez P, Hariharan B, Gu C, Gupta S, Bourdev L, Malik J (2012) Finding animals: semantic segmentation using regions and parts. In: International conference on computer vision and pattern recognition

  20. Martin M, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proc. 8th int’l conf. comput vis image underst, vol 2, pp 416–423

  21. Lukac M, Kameyama M, Perkowski M, Kerntopf P (2011) Decomposition of reversible logic function based on cube-reordering. Facta Universitatis 24(3):403–422

    Article  Google Scholar 

  22. Lukac M, Tanizawa R, Kameyama R (2012) Machine learning based adaptive contour detection using algorithm selection and image splitting. Interdiscipl Inf Sci 18(2):123–134

    Google Scholar 

  23. Lukac M, Kameyama M, Fujioka Y (2013) VLSI platform for real-world intelligent integrated systems based on algorithm selection. In: IADIS TPMC

  24. Rice J (1976) The algorithm selection problem. Adv Comput 15:65118

    Google Scholar 

  25. Yong X, Feng D, Rongchun Z (2003) Optimal selection of image segmentation algorithms based on performance prediction. In: Proceedings of the Pan-Sydney area workshop on visual information processing (VIP2003)

  26. Takemoto S, Yokota H (2009) Algorithm selection for intracellular image segmentation based on region similarity. In: Ninth international conference on intelligent systems design and applications

  27. Kolmogorov V, Boykov Y, Rother C (2007) Applications of parametric maxflow in computer vision. In: Proceedings of the international conference on computer vision (2007)

  28. Peng B, Veksler V (2008) Parameter selection for graph cut based image segmentation. In: Proceedings of the British conference on computer vision (2008)

  29. Price B, Morse B, Cohen S (2010) Geodesic graph cut for interactive image segmentation. In: Proceedings of the international conference on computer vision and pattern recognition

  30. Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vision 42(3):145–175

    Article  MATH  Google Scholar 

  31. Olson D, Delen D (2008) Advanced data mining techniques, 1st edn. Springer, Berlin. ISBN 3540769161

  32. Arbelaez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916

    Article  Google Scholar 

  33. Boykov Y, Veksler O, Zabih R (1998) Markov random fields with efficient approximations. In: International conference on computer vision and pattern recognition

  34. Boykov Y, Kolmogorov V (2004) An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans Pattern Anal Mach Intell 26(9):1124–1137

    Article  Google Scholar 

  35. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905

    Article  Google Scholar 

  36. Donoser M, Bischof H (2007) Roi-seg: unsupervised color segmentation by combining differently focused sub results. In: Proceedings of conference on computer vision and pattern recognition

  37. Donoser M, Urschler M, Bischof H (2009) Saliency driven total variational segmentation. In: Proceedings of international conference on computer vision

  38. Malik J, Belongie S, Shi J, Leung T (1999) Textons, contours and regions: cue combination in image segmentation. In: International conference on computer vision

  39. Maire M, Arbelaez P, Fowlkes C, Malik J (2008) Using contours to detect and localize junctions in natural images. In: Conference on vision and pattern recognition

  40. Lukac M, Kameyama M, Hiura K (2013) Natural image understanding using algorithm selection and high level feedback. In: SPIE intelligent robots and computer vision XXX: algorithms and techniques

  41. Brooks R (1981) Symbolic reasoning among 3-d models and 2-d images. Artif Intell 17:285–348

    Article  Google Scholar 

  42. Lam C, Venkatesh S, West G (1997) Hypothesis verification using parametric models and active vision strategies. J Comput Vision Image Underst 68(2):209–236

    Article  Google Scholar 

  43. Neumann B, Ralf Möller R (2008) On scene interpretation with description logics. Image Vision Comput 26(1):82–101

    Article  Google Scholar 

  44. Datta M, Murthy C (2012) Two dimensional synthetic face generation and verification using set estimation technique. Int J Comput Vision Image Underst 116(9):10221031

    Google Scholar 

  45. Felzenszwalb P, Girshick R, McAllester D, Ramanan D (2010) Object detection with discriminatively trained part based models. IEEE Trans Pattern Anal Mach Intell 32(9)

Download references

Author information

Authors and Affiliations

  1. Graduate School of Information Sciences, Tohoku University, Sendai, Japan

    Martin Lukac & Michitaka Kameyama

Authors
  1. Martin Lukac
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michitaka Kameyama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin Lukac.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lukac, M., Kameyama, M. An algorithm selection based platform for image understanding using high-level symbolic feedback and machine learning. Int. J. Mach. Learn. & Cyber. 6, 417–434 (2015). https://doi.org/10.1007/s13042-013-0197-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-013-0197-x

Keywords

Search

Navigation