Skip to main content

Advertisement

Springer Nature Link
Log in

Descriptor evaluation and feature regression for multimodal image analysis

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In this paper, we present feature descriptor evaluation and feature regression for multimodal image analysis. First, we compare the performances of several popular interest point detectors and feature descriptors from multimodal images with focus on visual and infrared images. The performances of detectors are evaluated mainly by the score of repeatability and accuracy and the descriptors are assessed by using the rate of precision and recall. Secondly, we analyze the relationship between the corresponding descriptors computed from multimodal images. The descriptors are regressed by means of linear regression as well as Gaussian process. Then the features on infrared images are predicted by mapping the descriptors from visual images to the infrared modality through the regression results. Predictions are assessed in two ways: the statistics of absolute error between true values and actual values, and the precision score of matching the predicted descriptors to the original infrared descriptors. We believe that this evaluating information will be useful when selecting an appropriate detector and descriptor for multimodal image analysis. Also the experimental results show that regression methods achieve a well-assessed relationship between corresponding descriptors from multiple modalities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Bansal, M., Daniilidis, K.: Joint spectral correspondence for disparate image matching. In: CVPR, pp. 2802–2809 (2013)

  2. Bay, H., Tuytelaars, T., Van Gool, L.: Surf: Speeded up robust features. In: Computer Vision: ECCV 2006, Springer, Graz, pp 404–417 (2006)

  3. Bingham, N.H., Bingham, N., Fry, J.M.: Regression: Linear models in statistics. Springer, New York (2010)

    Book  Google Scholar 

  4. Chang, K.I., Bowyer, K.W., Flynn, P.J.: An evaluation of multimodal 2d+ 3d face biometrics. PAMI 27(4), 619–624 (2005)

    Article  Google Scholar 

  5. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, pp 886–893 (2005)

  6. Firmenichy, D., Brown, M., Susstrunk, S.: Multispectral interest points for rgb-nir image registration. In: ICIP, pp 181–184 (2011)

  7. Gauglitz, S., Höllerer, T., Turk, M.: Evaluation of interest point detectors and feature descriptors for visual tracking. Int. J. Comput. Vis. 94(3), 335–360 (2011)

    Article  MATH  Google Scholar 

  8. Gil, A., Mozos, Ó.M., Ballesta, M., Reinoso, Ó.: A comparative evaluation of interest point detectors and local descriptors for visual slam. Mach. Vis. Appl. 21(6), 905–920 (2010)

    Article  Google Scholar 

  9. Han, J., Pauwels, E.J., de Zeeuw, P.M.: Visible and infrared image registration in man-made environments employing hybrid visual features. Pattern Recognit. Lett. 34(1), 42–51 (2013)

    Article  Google Scholar 

  10. Harris, C., Stephens, M.: A combined corner and edge detector. In: Alvey Vision Conference, Manchester, UK, vol 15, p. 50 (1988)

  11. Hrkać, T., Kalafatić, Z., Krapac, J.: Infrared-visual image registration based on corners and hausdorff distance. In: Scandinavian Conference on Image Analysis, pp 383–392 (2007)

  12. Kern, J.P., Pattichis, M.S.: Robust multispectral image registration using mutual-information models. IEEE Trans. Geosci. Remote Sens. 45(5), 1494–1505 (2007)

    Article  Google Scholar 

  13. Lindeberg, T.: Feature detection with automatic scale selection. Int. J. Comput. Vis. 30(2), 79–116 (1998)

    Article  Google Scholar 

  14. Lowe, D.: Object recognition from local scale-invariant features. In: ICCV, pp 1150–1157 (1999)

  15. Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  16. Lu, C., Jia, J., Tang, C.K.: Range-sample depth feature for action recognition. In: Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, IEEE, pp 772–779 (2014)

  17. Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imaging 16, 187–198 (1997)

    Article  Google Scholar 

  18. Mikolajczyk, K., Schmid, C.: Scale & affine invariant interest point detectors. Int. J. Comput. Vis. 60(1), 63–86 (2004)

    Article  Google Scholar 

  19. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Transa. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)

    Article  Google Scholar 

  20. Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L.: A comparison of affine region detectors. Int. J. Comput. Vis. 65(1–2), 43–72 (2005)

    Article  Google Scholar 

  21. Moreels, P., Perona, P.: Evaluation of features detectors and descriptors based on 3d objects. Int. J. Comput. Vis. 73(3), 263–284 (2007)

    Article  Google Scholar 

  22. Morris, N.J.W., Avidan, S., Matusik, W., Pfister, H.: Statistics of infrared images. In: CVPR, pp 1–7 (2007)

  23. Ni, B., Wang, G., Moulin, P.: Rgbd-hudaact: A color-depth video database for human daily activity recognition. In: Consumer Depth Cameras for Computer Vision, Springer, New York, pp 193–208 (2013)

  24. Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recognit. 29(1), 51–59 (1996)

    Article  Google Scholar 

  25. Rasmussen, C., Williams, C.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  26. Schmid, C., Mohr, R., Bauckhage, C.: Evaluation of interest point detectors. Int. J. Comput. Vis. 37(2), 151–172 (2000)

    Article  MATH  Google Scholar 

  27. Sedai, S., Bennamoun, M. & Huynh, D.: Evaluating shape and appearance descriptors for 3d human pose estimation. In: Industrial Electronics and Applications (ICIEA), 2011 6th IEEE Conference on, IEEE, pp 293–298 (2011)

  28. Varshney, P.K.: Multisensor data fusion. Electron. Commun. Eng. J. 9(6), 245–253 (1997)

    Article  Google Scholar 

  29. Wang, L., He, D.C.: Texture classification using texture spectrum. Pattern Recognit. 23(8), 905–910 (1990)

    Article  Google Scholar 

  30. Xia, L. & Aggarwal, J.: Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera. In: Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, IEEE, pp 2834–2841 (2013)

  31. Yang, M.Y., Yong, X., Rosenhahn, B.: Feature regression for multimodal image analysis. In: CVPR Workshop on Multi-Sensor Fusion for Outdoor Dynamic Scene Understanding, pp 756–763 (2014)

  32. Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003)

    Article  Google Scholar 

Download references

Acknowledgments

The work is partially funded by DFG (German Research Foundation) YA 351/2-1. The authors gratefully acknowledge the support.

Author information

Authors and Affiliations

  1. Institute for Geodesy, TU Darmstadt, Darmstadt, Germany

    Xuanzi Yong

  2. Computer Vision Lab (CVLD), TU Dresden, Dresden, Germany

    Michael Ying Yang

  3. Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR), Singapore, Singapore

    Yanpeng Cao

  4. Institute for Image Processing (TNT), Leibniz University Hannover, Hanover, Germany

    Bodo Rosenhahn

Authors
  1. Xuanzi Yong
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Michael Ying Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yanpeng Cao
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Bodo Rosenhahn
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Michael Ying Yang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yong, X., Yang, M.Y., Cao, Y. et al. Descriptor evaluation and feature regression for multimodal image analysis. Machine Vision and Applications 26, 975–990 (2015). https://doi.org/10.1007/s00138-015-0714-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-015-0714-x

Keywords