Abstract
The Fisher linear discriminant analysis (FLDA)-based method is a common method for jointly optimizing the intraclass separation and the interclass separation of the projected feature vectors by defining the objective function as the ratio of the intraclass separation over the interclass separation. To address the eigenproblem of the FLDA, a quadratic equality constraint is imposed on the square of the \(l_{2}\) norm of the decision vector. However, the constrained optimization problem is highly nonconvex. This paper proposes to reformulate the objective function as a weighted sum of the intraclass separation and the interclass separation subject to the same quadratic equality constraint on the square of the \(l_{2}\) norm of the decision vector. Although both the objective function and the feasible set of the optimization problem are still nonconvex, this paper shows that the global minimum of the objective functional value is equal to the minimum singular value of the Hessian matrix of the objective function. Also, the globally optimal solution of the optimization problem is in the null space of the Hessian matrix minus this singular value multiplied by the identity matrix. As it is only required to find the singular value of the Hessian matrix, no numerical optimization-based computer aided design tool is required to find the globally optimal solution. Therefore, the globally optimal solution can be found in real time. Experimental results demonstrate the effectiveness of our proposed method.
Similar content being viewed by others
References
Wang, Z., Ruan, Q.Q., An, G.Y.: Facial expression recognition using sparse local Fisher discriminant analysis. Neurocomputing 174, 756–766 (2016)
Wang, G.Q., Shi, N.F., Shu, Y.X., Liu, D.T.: Embedded manifold-based kernel Fisher discriminant analysis for face recognition. Neural Process. Lett. 43(1), 1–16 (2016)
Cho, S.Y.: 3D ear shape reconstruction and recognition for biometric applications. Signal Image Video Process. 7(4), 609–618 (2013)
Ouamane, A., Belahcene, M., Benakcha, A., Bourennane, S., Taleb-Ahmed, A.: Robust multimodal 2D and 3D face authentication using local feature fusion. Signal Image Video Process. 10(7), 129–137 (2016)
Gao, J.Q., Xu, L.Z.: A novel spatial analysis method for remote sensing image classification. Neural Process. Lett. 43(3), 805–821 (2016)
Du, Q.: Modified Fisher’s linear discriminant analysis for hyperspectral imagery. IEEE Geosci. Remote Sens. Lett. 4(4), 503–507 (2007)
Giannakopoulos, T., Petridis, S.: Fisher linear semi-discriminant analysis for speaker diarization. IEEE Trans. Audio Speech Lang. Process. 20(7), 1913–1922 (2012)
Shi, H.T., Liu, J.C., Wu, Y.H., Zhang, K., Zhang, L.X., Xue, P.: Fault diagnosis of nonlinear and large-scale processes using novel modified kernel Fisher discriminant analysis approach. Int. J. Syst. Sci. 47(5), 1095–1109 (2016)
Dragovic, S., Onjia, A.: Classification of soil samples according to their geographic origin using gamma-ray spectrometry and principal component analysis. J. Environ. Radioact. 89(2), 150–158 (2006)
Witten, D.M., Tibshirani, R.: Penalized classification using Fisher’s linear discriminant. J. Roy. Stat. Soc. B. 73, 753–772 (2011)
Gao, Q.X., Liu, J.J., Zhang, H.L., Gao, X.B., Li, K.: Joint global and local structure discriminant analysis. IEEE Trans. Inf. Forensics. Secur. 8(4), 626–635 (2013)
Wang, Q.Q., Hu, X.L., Gao, Q.X., Li, B., Wang, Y.: Global-local Fisher discriminant approach for face recognition. Neural Comput. Appl. 25(5), 1137–1144 (2014)
Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. Eugen. 7, 179–188 (1936)
Su, J., Xiao, J., Ling, B.W.K., Liu, Q., Tsang, K.F., Chui, K.T., Chi, H., Hancke, G., Zhou, Z.B.: Classifying tachycardias via high dimensional linear discriminant function and perceptron with multi-piece domain activation function. In: IEEE 13th International Conference on Industrial Informatics, INDIN, pp. 1480–1483 (2015)
Yu, Z.T., Lin, L.S., Ling, B.W.K., Ho, C.Y.F. Qing, C.M., Xu, H.L., Dai,W.H., Liang,Y.M., Chen J.M., Dai, Q.Y. : Camera identification for very low bit rate time varying quantization noise videos, 9th International Symposium on Communication Systems, Networks and Digital Signal Processing, CSNDSP, pp. 208–212 (2014)
Ho, C.Y.F., Ling, B.W.L., Iu, H.H.C.: Invariant set of weight of perceptron trained by perceptron training algorithm. IEEE Trans. Syst. Man Cybern. B. 40(6), 1521–1530 (2010)
Acknowledgements
This paper was supported partly by the National Nature Science Foundation of China (Nos. 61372173 and 61671163), the Guangdong Higher Education Engineering Technology Research Center for Big Data on Manufacturing Knowledge Patent (No. 501130144), the Natural Science Foundation of Guangdong Province China (No. 2014A030310346) and the Science and Technology Planning Project of Guangdong Province China (No. 2015A030401090).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Miao, Q., Ling, B.WK. Analytical form of globally optimal solution of weighted sum of intraclass separation and interclass separation. SIViP 12, 393–401 (2018). https://doi.org/10.1007/s11760-017-1171-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-017-1171-y