Abstract
In many machine learning applications there exists prior knowledge that the response variable should be increasing (or decreasing) in one or more of the features. This is the knowledge of ‘monotone’ relationships. This paper presents two new techniques for incorporating monotone knowledge into non-linear kernel support vector machine classifiers. Incorporating monotone knowledge is useful because it can improve predictive performance, and satisfy user requirements. While this is relatively straight forward for linear margin classifiers, for kernel SVM it is more challenging to achieve efficiently. We apply the new techniques to real datasets and investigate the impact of monotonicity and sample size on predictive accuracy. The results show that the proposed techniques can significantly improve accuracy when the unconstrained model is not already fully monotone, which often occurs at smaller sample sizes. In contrast, existing techniques demonstrate a significantly lower capacity to increase monotonicity or achieve the resulting accuracy improvements.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Alcalá, J., Fernández, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F.: Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J. Multiple-Valued Logic Soft Comput. 17, 255–287 (2010)
Chen, C.C., Li, S.T.: Credit rating with a monotonicity-constrained support vector machine model. Expert Syst. Appl. 41(16), 7235–7247 (2014)
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)
Daniels, H., Kamp, B.: Application of MLP networks to bond rating and house pricing. Neural Comput. Appl. 8(3), 226–234 (1999)
Duivesteijn, W., Feelders, A.: Nearest neighbour classification with monotonicity constraints. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008. LNCS (LNAI), vol. 5211, pp. 301–316. Springer, Heidelberg (2008). doi:10.1007/978-3-540-87479-9_38
Feelders, A., Pardoel, M.: Pruning for monotone classification trees. In: R. Berthold, M., Lenz, H.-J., Bradley, E., Kruse, R., Borgelt, C. (eds.) IDA 2003. LNCS, vol. 2810, pp. 1–12. Springer, Heidelberg (2003). doi:10.1007/978-3-540-45231-7_1
Feelders, A.J.: Prior knowledge in economic applications of data mining. In: Zighed, D.A., Komorowski, J., Żytkow, J. (eds.) PKDD 2000. LNCS (LNAI), vol. 1910, pp. 395–400. Springer, Heidelberg (2000). doi:10.1007/3-540-45372-5_42
Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D.: Do we need hundreds of classifiers to solve real world classification problems? J. Mach. Learn. Res. 15(1), 3133–3181 (2014)
Kamp, R., Feelders, A., Barile, N.: Isotonic classification trees. In: Adams, N.M., Robardet, C., Siebes, A., Boulicaut, J.-F. (eds.) IDA 2009. LNCS, vol. 5772, pp. 405–416. Springer, Heidelberg (2009). doi:10.1007/978-3-642-03915-7_35
Kotlowski, W.: Statistical approach to ordinal classification with monotonicity constraints. Ph.D. thesis, Pozna Univ of Techn Inst of Computing Science (2008)
Kuhn, H.W., Tucker, A.W.: Nonlinear programming. In: Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability. pp. 481–492. University of California Press, Berkeley, Calif. (1951)
Lauer, F., Bloch, G.: Incorporating prior knowledge in support vector machines for classification: a review. Neurocomputing 71(7), 1578–1594 (2008)
Li, S.T., Chen, C.C.: A regularized monotonic fuzzy support vector machine for data mining with prior knowledge. IEEE Trans. Fuzzy Syst. PP(99) (2014)
Lichman, M.: UCI machine learning repository (2013). http://archive.ics.uci.edu/ml
Mangasarian, O.L., Wild, E.W.: Nonlinear knowledge-based classification. IEEE Trans. Neural Netw. 19(10), 1826–1832 (2008)
Marsala, C., Petturiti, D.: Rank discrimination measures for enforcing monotonicity in decision tree induction. Inf. Sci. 291, 143–171 (2015)
Milstein, I., David, A.B., Potharst, R.: Generating noisy monotone ordinal datasets. Artif. Intell. Res. 3(1), p30 (2013)
Pelckmans, K., Espinoza, M., De Brabanter, J., Suykens, J.A., De Moor, B.: Primal-dual monotone kernel regression. Neural Proc. Letters 22(2), 171–182 (2005)
Potharst, R., Ben-David, A., van Wezel, M.: Two algorithms for generating structured and unstructured monotone ordinal data sets. Eng. App. Art. Intell. 22(4), 491–496 (2009)
Potharst, R., Bioch, J.C.: Decision trees for ordinal classification. Intell. Data Anal. 4(2), 97–111 (2000)
Potharst, R., Feelders, A.J.: Classification trees for problems with monotonicity constraints. ACM SIGKDD Explor. Newsletter 4(1), 1–10 (2002)
Tikhonov, A., Arsenin, V.: Solutions of ill-posed problems. Scripta series in mathematics, Winston (1977)
Velikova, M., Daniels, H.: Decision trees for monotone price models. CMS 1(3–4), 231–244 (2004)
Wang, Y., Ni, H.: Multivariate convex support vector regression with semidefinite programming. Knowl.-Based Syst. 30, 87–94 (2012)
Wellman, M.P.: Fundamental concepts of qualitative probabilistic networks. Artif. Intell. 44(3), 257–303 (1990)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Bartley, C., Liu, W., Reynolds, M. (2016). Effective Monotone Knowledge Integration in Kernel Support Vector Machines. In: Li, J., Li, X., Wang, S., Li, J., Sheng, Q. (eds) Advanced Data Mining and Applications. ADMA 2016. Lecture Notes in Computer Science(), vol 10086. Springer, Cham. https://doi.org/10.1007/978-3-319-49586-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-49586-6_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-49585-9
Online ISBN: 978-3-319-49586-6
eBook Packages: Computer ScienceComputer Science (R0)