Abstract
Multi-target regression (MTR) has attracted an increasing amount of attention in recent years. The main challenge in multi-target regression is to create predictive models for problems with multiple continuous targets by considering the inter-target correlation which can greatly influence the predictive performance. There is another thing that most of existing methods omit, the impact of inputs in target correlations (conditional target correlation). In this paper, a novel MTR framework, termed as Conditionally Decorrelated Multi-Target Regression (CDMTR) is proposed. CDMTR learns from the MTR data following three elementary steps: clustering analysis, conditional target decorrelation and multi-target regression models induction. Experimental results on various benchmark MTR data sets approved that the proposed method enjoys significant advantages compared to other state-of-the art MTR methods.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bakir, G.H., Weston, J., Schölkopf, B.: Learning to find pre-images. In: Advances in Neural Information Processing Systems (NIPS), pp. 449–456 (2003)
Borchani, H., Varando, G., Bielza, C., Larrañaga, P.: A survey on multi-output regression. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 5(5), 216–233 (2015)
Demsar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)
Devis, T., Jochem, V., Luis, A., Fernando, P.C., Gustau, C.V.: Multioutput support vector regression for remote sensing biophysical parameter estimation. Geosci. Remote Sens. Lett. 8, 804–808 (2011)
Gibaja, E., Ventura, S.: A tutorial on multilabel learning. ACM Comput. Surv. 47(3), 52:1–52:38 (2015)
Haiqing, L., Wei, Z., Ying, C., Yumeng, G., Guo-Zheng, L., Xiaoxin, Z.: A novel multi-target regression framework for time-series prediction of drug efficacy. Sci. Rep. 7, 40652 (2017)
Izenman, A.J.: Reduced-rank regression for the multivariate linear model. J. Multivar. Anal. 5(2), 248–264 (1975)
Kocev, D., Dzeroski, S., White, M.D., Newell, G.R., Griffioen, P.A.: Using single- and multi-target regression trees and ensembles to model a compound index of vegetation condition. Ecol. Model. 220(8), 1159–1168 (2009)
Kocev, D., Vens, C., Struyf, J., Dzeroski, S.: Tree ensembles for predicting structured outputs. Pattern Recogn. 46(3), 817–833 (2013)
Melki, G., Cano, A., Kecman, V., Ventura, S.: Multi-target support vector regression via correlation regressor chains. Inf. Sci. 415, 53–69 (2017)
Moyano, J.M., Galindo, E., Ventura, S.: An evolutionary algorithm for optimizing the target ordering in ensemble of regressor chains. In: IEEE Congress on Evolutionary Computation, CEC 2017, Donostia, San Sebastián, Spain, 5–8 June 2017, pp. 2015–2021 (2017)
Petković, M., Džeroski, S., Kocev, D.: Feature ranking for multi-target regression with tree ensemble methods. In: Yamamoto, A., Kida, T., Uno, T., Kuboyama, T. (eds.) DS 2017. LNCS (LNAI), vol. 10558, pp. 171–185. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67786-6_13
Rousseeuw, P.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Comput. Appl. Math. 20, 53–65 (1987)
Tsoumakas, G., Xioufis, E.S., Vilcek, J., Vlahavas, I.P.: MULAN: a Java library for multi-label learning. J. Mach. Learn. Res. 12, 2411–2414 (2011)
Tsoumakas, G., Spyromitros-Xioufis, E., Vrekou, A., Vlahavas, I.: Multi-target regression via random linear target combinations. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014. LNCS, vol. 8726, pp. 225–240. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44845-8_15
Wei, Z., Xianhui, L., Yi, D., Deming, S.: Multi-output LS-SVR machine in extended feature space. In: IEEE International Conference on Computational Intelligence for Measurement Systems and Applications, pp. 130–134, July 2012
Wold, H.: Partial least squares. In: Kotz, S., Johnson, N.L., (eds.), Encyclopedia of Statistical Sciences, vol. 6, pp. 581–591 (1985)
Zhen, X., Yu, M., He, X., Li, S.: Multi-target regression via robust low-rank learning. PAMI 40, 497–504 (2018)
Xioufis, E.S., Tsoumakas, G., Groves, W., Vlahavas, I.P.: Multi-target regression via input space expansion: treating targets as inputs. Mach. Learn. 104(1), 55–98 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Yazar, O., Elghazel, H., Hacid, MS., Castin, N. (2019). Conditionally Decorrelated Multi-Target Regression. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Lecture Notes in Computer Science(), vol 11954. Springer, Cham. https://doi.org/10.1007/978-3-030-36711-4_37
Download citation
DOI: https://doi.org/10.1007/978-3-030-36711-4_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36710-7
Online ISBN: 978-3-030-36711-4
eBook Packages: Computer ScienceComputer Science (R0)