Skip to main content

Advertisement

SpringerLink
Log in

Pattern recognition in multilinear space and its applications: mathematics, computational algorithms and numerical validations

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

We clarify the mathematical equivalence between low-dimensional singular value decomposition and low-order tensor principal component analysis for two- and three-dimensional images. Furthermore, we show that the two- and three-dimensional discrete cosine transforms are, respectively, acceptable approximations to two- and three-dimensional singular value decomposition and classical principal component analysis. Moreover, for the practical computation in two-dimensional singular value decomposition, we introduce the marginal eigenvector method, which was proposed for image compression. For three-dimensional singular value decomposition, we also show an iterative algorithm. To evaluate the performances of the marginal eigenvector method and two-dimensional discrete cosine transform for dimension reduction, we compute recognition rates for six datasets of two-dimensional image patterns. To evaluate the performances of the iterative algorithm and three-dimensional discrete cosine transform for dimension reduction, we compute recognition rates for datasets of gait patterns and human organs. For two- and three-dimensional images, the two- and three-dimensional discrete cosine transforms give almost the same recognition rates as the marginal eigenvector method and iterative algorithm, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. For an iterative method, see Refs. [18, 19]. These iterative algorithms are a special case of the HOSVD [23].

References

  1. Itoh, H., Imiya, A., Sakai, T.: Low-dimensional tensor principle component analysis. Proc. CAIP Part (I) 9256, 715–726 (2015)

    Google Scholar 

  2. Makihara, Y., Mannami, H., Tsuji, A., Hossain, M.A., Sugiura, K., Mori, A., Yagi, Y.: The OU-ISIR gait database comprising the treadmill dataset. IPSJ Trans. Comput. Vis. Appl. 4, 53–62 (2012)

    Article  Google Scholar 

  3. von Siebenthal, M., Cattin, P.H., Gamper, U., Lomax, A., Székely, G.: 4D MR imaging using internal respiratory gating. In: Proceedings of the MICCAI. pp. 336–343 (2005)

  4. Leibe, B., Schiele, B.: Analyzing appearance and contour based methods for object categorization. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 409–415 (2003)

  5. Everingham, M., Eslami, S.M.A., Gool, L.V., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111, 98–136 (2015)

    Article  Google Scholar 

  6. Boyer, K.L., Ünsalan, C.: Multispectral Satellite Image Understanding: From Land Classification to Building and Road Detection. Springer, New York (2012)

    Google Scholar 

  7. Ely, G., Aeron, S., Hao, N., Kilmer, M.E.: 5D seismic data completion and denoising using a novel class of tensor decompositions. Geophysics 80(4), V83–V95 (2015)

    Article  Google Scholar 

  8. Cohen, N., Shashua, A.: Simnets: a generalization of convolutional networks. In: Proceedings of the NIPS Workshop on Deep Learning (2014)

  9. Dean, J., Corrado, G., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P., Yang, K., Le, Q.V., Ng, A.Y.: Large scale distributed deep networks. In: Proceedings of the NIPS, pp 1232–1240 (2012)

  10. Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: MPCA: multilinear principal component analysis of tensor objects. IEEE Trans. Neural Netw. 19(1), 18–39 (2008)

    Article  Google Scholar 

  11. Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: A survey of multilinear subspace learning for tensor data. Pattern Recognit. 44, 1540–1551 (2011)

    Article  MATH  Google Scholar 

  12. Yang, J., Zhang, D., Frangi, A.F., Yang, J.-Y.: Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans. PAMI 26, 131–137 (2004)

    Article  Google Scholar 

  13. Otsu. N.: Mathematical studies on feature extraction in pattern recognition. PhD thesis, Electrotechnical Laboratory (1981)

  14. Aase, S.O., Husoy, J.H., Waldemar, P.: A critique of SVD-based image coding systems. In: Proceedings of the IEEE International Symposium on Circuits and Systems vol. 4, pp. 13–16 (1999)

  15. Ding, C., Ye, J.: Two-dimensional singular value decomposition (2DSVD) for 2D maps and images. In: Proceedings of the SIAM International Conference on Data Mining, pp 32–43 (2005)

  16. Golub, G.H., Van Loan, C.F.: Matrix Computations. The Johns Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  17. Ye, J., Janardan, R., Qi, L.: GPCA: an efficient dimension reduction scheme for image compression and retrieval. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 354–363 (2004)

  18. Helmke, U., Moore, J.B.: Singular-value decomposition via gradient and self-equivalent flows. Linear Algebra Appl. 169, 223–248 (1992)

  19. Moore, J.B., Mahony, R.E., Helmke, U.: Numerical gradient algorithms for eigenvalue and singular value calculations. SIAM J. Matrix Anal. Appl. 15, 881–902 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  20. Tucker, L.: Some mathematical notes on three-mode factor analysis. Psychometrika 31(3), 279–311 (1966)

    Article  MathSciNet  Google Scholar 

  21. Kroonenberg, P.M., Leeuw, J.: Principal component analysis of three-mode data by means of alternating least squares algorithms. Psychometrika 45(1), 69–97 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  22. Cichoki, A., Zdunek, R., Phan, A.H., Amari, S.: Nonnegative Matrix and Tensor Factorizations. Wiley, Hoboken (2009)

    Book  Google Scholar 

  23. Lathauwer, L.D., De Moor, B., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21(4), 1253–1278 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  24. Lathauwer, L.D., De Moor, B., Vandewalle, J.: On the best rank-1 and rank-(\(r_1, r_2, r_n\)) approximation of higher-order tensors. SIAM J. Matrix Anal. Appl. 21(4), 1324–1342 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  25. Inoue, K., Hara, K., Urahama, K.: Robust multilinear principal component analysis. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 591–597 (2009)

  26. Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: Uncorrelated multilinear principal component analysis for unsupervised multilinear subspace learning. IEEE Trans. Neural Netw. 20(11), 1820–1836 (2009)

    Article  Google Scholar 

  27. Allen, G.I.: Sparse higher-order principal components analysis. In: Proceedings of the International Conference on Artificial Intelligence and Statistics, pp. 27–36 (2012)

  28. Itoh, H., Sakai, T., Kawamoto, K., Imiya, A.: Topology-preserving dimension-reduction methods for image pattern recognition. In: Proceedings of the Scandinavian Conference on Image Analysis, pp. 195–204 (2013)

  29. Oja, E.: Subspace Methods of Pattern Recognition. Research Studies Press, Baldock (1983)

    Google Scholar 

  30. Hamidi, M., Pearl, J.: Comparison of the cosine and fourier transforms of Markov-1 signals. IEEE Trans. Acoust. Speech Signal Process. 24(5), 428–429 (1976)

    Article  MathSciNet  Google Scholar 

  31. Wang, Y., Gong, S.: Tensor discriminant analysis for view-based object recognition. Proc. CVPR 3, 33–36 (2006)

    Google Scholar 

  32. Hua, G., Viola, P.A., Drucker, S.M.: Face recognition using discriminatively trained orthogonal rank one tensor projections. Proc. CVPR (2007)

  33. Ye, J.: Generalized low rank approximations of matrices. In: Proceedings of the ICML (2004)

  34. Liang, Z., Shi, P.: An analytical algorithm for generalized low-rank approximations of matrices. Pattern Recognit. 38, 2213–2216 (2005)

    Article  MATH  Google Scholar 

  35. Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J.: From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans. PAMI 23, 643–660 (2001)

    Article  Google Scholar 

  36. Samaria, F., Harter, A.: Parameterisation of a stochastic model for human face identification. In: Proceedings of the IEEE Workshop on Applications of Computer Vision (1994)

  37. Mobahi, H., Collobert, R., Weston, J.: Deep learning from temporal coherence in video. In: Proceedings of the 26th International Conference on Machine Learning, Montreal, Canada (2009)

  38. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)

    Article  Google Scholar 

  39. Saito, T., Yamada, H., Yamada, K.: On the data base ETL9 of handprinted characters in JIS Chinese characters and its analysis. IEEJ J. Ind. Appl. 68(4), 757–764 (1985)

    Google Scholar 

Download references

Acknowledgments

This research was supported by the “Multidisciplinary Computational Anatomy and Its Application to Highly Intelligent Diagnosis and Therapy” project funded by a Grant-in-Aid for Scientific Research on Innovative Areas from MEXT, Japan, and by Grants-in-Aid for Scientific Research funded by the Japan Society for the Promotion of Science.

Author information

Authors and Affiliations

  1. Graduate School of Advanced Integration Science, Chiba University, Yayoi-cho 1-33, Inage-ku, Chiba, 263-8522, Japan

    Hayato Itoh

  2. Institute of Management and Information Technologies, Chiba University, Yayoi-cho 1-33, Inage-ku, Chiba, 263-8522, Japan

    Atsushi Imiya

  3. Graduate School of Engineering, Nagasaki University, Bunkyo-cho 1-14, Nagasaki, 852-8521, Japan

    Tomoya Sakai

Authors
  1. Hayato Itoh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Atsushi Imiya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tomoya Sakai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hayato Itoh.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Itoh, H., Imiya, A. & Sakai, T. Pattern recognition in multilinear space and its applications: mathematics, computational algorithms and numerical validations. Machine Vision and Applications 27, 1259–1273 (2016). https://doi.org/10.1007/s00138-016-0806-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-016-0806-2

Keywords

Search

Navigation