Skip to main content
Log in

Analytical form of globally optimal solution of weighted sum of intraclass separation and interclass separation

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The Fisher linear discriminant analysis (FLDA)-based method is a common method for jointly optimizing the intraclass separation and the interclass separation of the projected feature vectors by defining the objective function as the ratio of the intraclass separation over the interclass separation. To address the eigenproblem of the FLDA, a quadratic equality constraint is imposed on the square of the \(l_{2}\) norm of the decision vector. However, the constrained optimization problem is highly nonconvex. This paper proposes to reformulate the objective function as a weighted sum of the intraclass separation and the interclass separation subject to the same quadratic equality constraint on the square of the \(l_{2}\) norm of the decision vector. Although both the objective function and the feasible set of the optimization problem are still nonconvex, this paper shows that the global minimum of the objective functional value is equal to the minimum singular value of the Hessian matrix of the objective function. Also, the globally optimal solution of the optimization problem is in the null space of the Hessian matrix minus this singular value multiplied by the identity matrix. As it is only required to find the singular value of the Hessian matrix, no numerical optimization-based computer aided design tool is required to find the globally optimal solution. Therefore, the globally optimal solution can be found in real time. Experimental results demonstrate the effectiveness of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Wang, Z., Ruan, Q.Q., An, G.Y.: Facial expression recognition using sparse local Fisher discriminant analysis. Neurocomputing 174, 756–766 (2016)

    Article  Google Scholar 

  2. Wang, G.Q., Shi, N.F., Shu, Y.X., Liu, D.T.: Embedded manifold-based kernel Fisher discriminant analysis for face recognition. Neural Process. Lett. 43(1), 1–16 (2016)

    Article  Google Scholar 

  3. Cho, S.Y.: 3D ear shape reconstruction and recognition for biometric applications. Signal Image Video Process. 7(4), 609–618 (2013)

    Article  Google Scholar 

  4. Ouamane, A., Belahcene, M., Benakcha, A., Bourennane, S., Taleb-Ahmed, A.: Robust multimodal 2D and 3D face authentication using local feature fusion. Signal Image Video Process. 10(7), 129–137 (2016)

    Article  Google Scholar 

  5. Gao, J.Q., Xu, L.Z.: A novel spatial analysis method for remote sensing image classification. Neural Process. Lett. 43(3), 805–821 (2016)

    Article  Google Scholar 

  6. Du, Q.: Modified Fisher’s linear discriminant analysis for hyperspectral imagery. IEEE Geosci. Remote Sens. Lett. 4(4), 503–507 (2007)

    Article  Google Scholar 

  7. Giannakopoulos, T., Petridis, S.: Fisher linear semi-discriminant analysis for speaker diarization. IEEE Trans. Audio Speech Lang. Process. 20(7), 1913–1922 (2012)

    Article  Google Scholar 

  8. Shi, H.T., Liu, J.C., Wu, Y.H., Zhang, K., Zhang, L.X., Xue, P.: Fault diagnosis of nonlinear and large-scale processes using novel modified kernel Fisher discriminant analysis approach. Int. J. Syst. Sci. 47(5), 1095–1109 (2016)

  9. Dragovic, S., Onjia, A.: Classification of soil samples according to their geographic origin using gamma-ray spectrometry and principal component analysis. J. Environ. Radioact. 89(2), 150–158 (2006)

    Article  Google Scholar 

  10. Witten, D.M., Tibshirani, R.: Penalized classification using Fisher’s linear discriminant. J. Roy. Stat. Soc. B. 73, 753–772 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  11. Gao, Q.X., Liu, J.J., Zhang, H.L., Gao, X.B., Li, K.: Joint global and local structure discriminant analysis. IEEE Trans. Inf. Forensics. Secur. 8(4), 626–635 (2013)

    Article  Google Scholar 

  12. Wang, Q.Q., Hu, X.L., Gao, Q.X., Li, B., Wang, Y.: Global-local Fisher discriminant approach for face recognition. Neural Comput. Appl. 25(5), 1137–1144 (2014)

    Article  Google Scholar 

  13. Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. Eugen. 7, 179–188 (1936)

    Article  Google Scholar 

  14. Su, J., Xiao, J., Ling, B.W.K., Liu, Q., Tsang, K.F., Chui, K.T., Chi, H., Hancke, G., Zhou, Z.B.: Classifying tachycardias via high dimensional linear discriminant function and perceptron with multi-piece domain activation function. In: IEEE 13th International Conference on Industrial Informatics, INDIN, pp. 1480–1483 (2015)

  15. Yu, Z.T., Lin, L.S., Ling, B.W.K., Ho, C.Y.F. Qing, C.M., Xu, H.L., Dai,W.H., Liang,Y.M., Chen J.M., Dai, Q.Y. : Camera identification for very low bit rate time varying quantization noise videos, 9th International Symposium on Communication Systems, Networks and Digital Signal Processing, CSNDSP, pp. 208–212 (2014)

  16. Ho, C.Y.F., Ling, B.W.L., Iu, H.H.C.: Invariant set of weight of perceptron trained by perceptron training algorithm. IEEE Trans. Syst. Man Cybern. B. 40(6), 1521–1530 (2010)

    Article  Google Scholar 

Download references

Acknowledgements

This paper was supported partly by the National Nature Science Foundation of China (Nos. 61372173 and 61671163), the Guangdong Higher Education Engineering Technology Research Center for Big Data on Manufacturing Knowledge Patent (No. 501130144), the Natural Science Foundation of Guangdong Province China (No. 2014A030310346) and the Science and Technology Planning Project of Guangdong Province China (No. 2015A030401090).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Bingo Wing-Kuen Ling.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Miao, Q., Ling, B.WK. Analytical form of globally optimal solution of weighted sum of intraclass separation and interclass separation. SIViP 12, 393–401 (2018). https://doi.org/10.1007/s11760-017-1171-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-017-1171-y

Keywords

Navigation