Skip to main content
SpringerLink
Log in

Automated insect identification through concatenated histograms of local appearance features: feature vector generation and region detection for deformable objects

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

This paper describes a computer vision approach to automated rapid-throughput taxonomic identification of stonefly larvae. The long-term objective of this research is to develop a cost-effective method for environmental monitoring based on automated identification of indicator species. Recognition of stonefly larvae is challenging because they are highly articulated, they exhibit a high degree of intraspecies variation in size and color, and some species are difficult to distinguish visually, despite prominent dorsal patterning. The stoneflies are imaged via an apparatus that manipulates the specimens into the field of view of a microscope so that images are obtained under highly repeatable conditions. The images are then classified through a process that involves (a) identification of regions of interest, (b) representation of those regions as SIFT vectors (Lowe, in Int J Comput Vis 60(2):91–110, 2004) (c) classification of the SIFT vectors into learned “features” to form a histogram of detected features, and (d) classification of the feature histogram via state-of-the-art ensemble classification algorithms. The steps (a) to (c) compose the concatenated feature histogram (CFH) method. We apply three region detectors for part (a) above, including a newly developed principal curvature-based region (PCBR) detector. This detector finds stable regions of high curvature via a watershed segmentation algorithm. We compute a separate dictionary of learned features for each region detector, and then concatenate the histograms prior to the final classification step. We evaluate this classification methodology on a task of discriminating among four stonefly taxa, two of which, Calineuria and Doroneuria, are difficult even for experts to discriminate. The results show that the combination of all three detectors gives four-class accuracy of 82% and three-class accuracy (pooling Calineuria and Doro-neuria) of 95%. Each region detector makes a valuable contribution. In particular, our new PCBR detector is able to discriminate Calineuria and Doroneuria much better than the other detectors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Arbuckle, T., Schroder, S., Steinhage, V., Wittmann, D.: Biodiversity informatics in action: identification and monitoring of bee species using ABIS. In: Proceedings of the 15th International Symposium Informatics for Environmental Protection, vol. 1, pp. 425–430. Zurich (2001)

  2. Bouchard, G., Triggs, B.: Hierarchical part-based visual object categorization. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. I 710–715 (2005). http://lear.inrialpes.fr/pubs/2005/BT05

  3. Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996). http://citeseer.ist.psu.edu/breiman96bagging.html

  4. Breiman L., Friedman J., Olshen R. and Stone C. (1984). Classification and Regression Trees. Chapman and Hall, New York

    MATH  Google Scholar 

  5. Burl, M., Weber, M., Perona, P.: A probabilistic approach to object recognition using local photometry and global geometry. In: Proceedings of the ECCV, pp. 628–641 (1998)

  6. Carter J., Resh V., Hannaford M. and Myers M. (2006). Macroinvertebrates as biotic indicators of env. qual. In: Hauer, F. and Lamberti, G. (eds) Methods in Stream Ecology, pp 1–2. Academic, San Diego

    Google Scholar 

  7. Csurka, G., Dance, C., Fan, L., Williamowski, J., Bray, C.: Visual categorization with bags of keypoints. ECCV’04 workshop on Statistical Learning in Computer Vision, pp. 59–74 (2004)

  8. Csurka, G., Bray, C., Fan, C.L.: Visual categorization with bags of keypoints. ECCV workshop (2004)

  9. Do M., Harp J. and Norris K. (1999). A test of a pattern recognition system for identification of spiders. Bull. Entomol. Res. 89(3): 217–224

    Article  Google Scholar 

  10. Dorko, G., Schmid, C.: Object class recognition using discriminative local features. INRIA—Rhone-Alpes, RR-5497, February, 2005, Rapport de recherche. http://lear.inrialpes.fr/pubs/ 2005/DS05a

  11. Dorkó, G., Schmid, C.: Object class recognition using discriminative local features (2005). http://lear.inrialpes.fr/pubs/2005/DS05. Accepted under major revisions to IEEE Trans. Pattern Anal. Mach. Intell. (updated 13 September)

  12. Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol.~2, pp. 264–271. Madison, Wisconsin (2003)

  13. Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: International Conference on Machine Learning, pp. 148–156 (1996). http://citeseer.ist.psu.edu/freund96experiments.html

  14. Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting (1998). http://citeseer.ist.psu. edu/friedman98additive.html

  15. Gaston K.J. and O’Neill M.A. (2004). Automated species identification: why not?. Philosophical Trans. R. Soc. B: Biol. Sci. 359(1444): 655–667

    Article  Google Scholar 

  16. Harris, C., Stephens, M.: A combined corner and edge detector. Alvey Vision Conference, pp. 147–151 (1988)

  17. Hilsenhoff W.L. (1988). Rapid field assessment of organic pollution with a family level biotic index. J. North Am. Benthol. Soc. 7: 65–68

    Article  Google Scholar 

  18. Hopkins G.W. and Freckleton R.P. (2002). Declines in the numbers of amateur and professional taxonomists: implications for conservation. Anim. Conserv. 5(3): 245–249

    Article  Google Scholar 

  19. Jurie F. and Schmid C. (2004). Scale-invariant shape features for recognition of object categories. CVPR 2: 90–96

    Google Scholar 

  20. Jurie, F., Triggs, B.: Creating efficient codebooks for visual recognition. In: ICCV ’05: Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV’05), vol. 1, pp. 604–610. IEEE Computer Society, Washington, DC, USA (2005). DOI http://dx.doi.org/10.1109/ICCV.2005.66

  21. Kadir, T., Zisserman, A., Brady, M.: An affine invariant salient region detector. In: European Conference on Computer Vision (ECCV04), pp. 228–241 (2004)

  22. Kumar, S., August, J., Hebert, M.: Exploiting inference for approximate parameter learning in discriminative fields: an empirical study. In: 5th International Workshop, EMMCVPR 2005, pp. 153–168. Springer, St. Augustine (2005)

  23. Landwehr, N., Hall, M., Frank, E.: Logistic model trees. Mach. Learn. 59(1–2), 161–205 (2005). DOI 10.1007/s10994-005- 0466-3

  24. Leibe, B., Seemann, E., Schiele, B.: Pedestrian detection in crowded scenes. In: CVPR ’05: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 878–885. IEEE Computer Society, Washington, DC, USA (2005). DOI http://dx.doi.org/10.1109/CVPR.2005.272

  25. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004). DOI 10.1023/B:VISI.0000029664.99615.94

    Google Scholar 

  26. Lucas, S.: Face recognition with continuous n-tuple classifier. In: Proceedings of the British Machine Vision Conference, pp. 222–231. Essex (1997)

  27. Matas J., Chum O., Urban M. and Pajdla T. (2004). Robust wide- baseline stereo from maximally stable extremal regions. Image Vis. Comput. 22(10): 761–767

    Article  Google Scholar 

  28. Mikolajczyk K. and Schmid C. (2002). An affine invariant interest point detector. ECCV 1(1): 128–142

    Google Scholar 

  29. Mikolajczyk K. and Schmid C. (2004). Scale and affine invariant interest point detectors. IJCV 60(1): 63–86

    Article  Google Scholar 

  30. Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Gool, L.V.: A comparison of affine region detectors. IJCV 65(1/2), 43–72 (2005). http://lear.inrialpes.fr/pubs/2005/MTSZMSKG05

    Google Scholar 

  31. O’Neill, M.A., Gauld, I.D., Gaston, K.J., Weeks, P.: Daisy: an automated invertebrate identification system using holistic vision techniques. In: Proceedings of the Inaugural Meeting BioNET-INTERNATIONAL Group for Computer-Aided Taxonomy (BIGCAT), pp. 13–22. Egham (2000)

  32. Opelt, A., Fussenegger, M., Pinz, A., Auer, P.: Weak hypotheses and boosting for generic object detection and recognition. In: 8th European Conference on Computer Vision, vol. 2, pp. 71–84. Prague, Czech Republic (2004)

  33. Opelt A., Pinz A., Fussenegger M. and Auer P. (2006). Generic object recognition with boosting. IEEE Trans. Pattern Anal. Mach. Intell. 28(3): 416–431

    Article  Google Scholar 

  34. Papageorgiou C. and Poggio T. (2000). A trainable system for object detection. Int. J. Comput. Vis. 38(1): 15–33

    Article  MATH  Google Scholar 

  35. Quattoni, A., Collins, M., Darrell, T.: Conditional random fields for object recognition. In: Proceedings of the NIPS 2004. MIT Press, Cambridge (2005)

  36. Quinlan J.R. (1993). C4.5: programs for machine learning. Morgan Kaufmann, San Francisco

    Google Scholar 

  37. Shotton, J., Blake, A., Cipolla, R.: Contour-based learning for object detection. In: ICCV ’05: Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV’05), vol. 1, pp. 503–510. IEEE Computer Society, Washington, DC, USA (2005). DOI http://dx.doi.org/10.1109/ICCV.2005.63

  38. Sokal, R.R., Rohlf, F.J.: Biometry, 3rd edn. W. H. Freeman, Gordonsville (1995)

  39. Steger C. (1998). An unbiased detector of curvilinear structures. PAMI 20(2): 113–125

    Google Scholar 

  40. Sung K.K. and Poggio T. (1998). Example-based learning for view-based human face detection. IEEE Trans. Pattern Anal. Mach. Intell. 20(1): 39–51

    Article  Google Scholar 

  41. Turk, M.A., Pentland, A.P.: Face recognition using eigenfaces. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–591 (1991)

  42. Tuytelaars, T., Gool, L.V.: Wide baseline stereo matching based on local, affinely invariant regions. BMVC, pp. 412–425 (2000)

  43. Tuytelaars T. and Gool L.V. (2004). Matching widely separated views based on affine invariant regions. IJCV 59(1): 61–85

    Article  Google Scholar 

  44. Vincent L. and Soille P. (1991). Watersheds in digital spaces: an efficient algorithm based on immersion simulations. PAMI 13(6): 583–598

    Google Scholar 

  45. Zhang, W., Deng, H., Dietterich, T.G., Mortensen, E.N.: A hierarchical object recognition system based on multi-scale principal curvature regions. International Conference of Pattern Recognition, pp. 1475–1490 (2006)

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, University of Washington, Seattle, WA, USA

    Natalia Larios

  2. School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR, USA

    Hongli Deng, Wei Zhang, Eric N. Mortensen & Thomas G. Dietterich

  3. Department of Mechanical Engineering, Oregon State University, Corvallis, OR, USA

    Matt Sarpola & Robert Paasch

  4. Department of Botany and Plant Pathology, Oregon State University, Corvallis, OR, USA

    Andrew Moldenke

  5. Department of Zoology, Oregon State University, Corvallis, OR, USA

    David A. Lytle

  6. Computer Science and AI Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA

    Jenny Yuen

  7. Department of Diagnostic Imaging and Radiology, Children’s National Medical Center, Washington, DC, USA

    Salvador Ruiz Correa

  8. Department of Computer Science and Engineering, University of Washington, Seattle, WA, USA

    Linda G. Shapiro

Authors
  1. Natalia Larios
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongli Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matt Sarpola
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jenny Yuen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Robert Paasch
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Andrew Moldenke
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. David A. Lytle
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Salvador Ruiz Correa
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Eric N. Mortensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Linda G. Shapiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Thomas G. Dietterich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Natalia Larios.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Larios, N., Deng, H., Zhang, W. et al. Automated insect identification through concatenated histograms of local appearance features: feature vector generation and region detection for deformable objects. Machine Vision and Applications 19, 105–123 (2008). https://doi.org/10.1007/s00138-007-0086-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-007-0086-y

Keywords

Search

Navigation