Abstract
Machine learning with a reject option is the empowerment of an algorithm to abstain from prediction when the outcome is likely to be inaccurate. Although the topic has been investigated in the literature already some time ago, it has not lost any of its relevance as machine learning models are increasingly delivered to the market. At present, most work on reject strategies addresses classification tasks. Moreover, the majority of approaches deals with classical batch learning scenarios. In this publication, in contrast, we study the important problem of reject options for incremental online regression tasks. We propose different strategies to model this problem and evaluate different approaches, both in a theoretical and a real world setting from the domain of human motion prediction; from the methods which we evaluate, a clear winner emerges as regards accuracy and efficiency.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Asif, A.: Generalized neural framework for learning with rejection. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2020). https://doi.org/10.1109/IJCNN48605.2020.9206612
Bartlett, P.L., Wegkamp, M.H.: Classification with a reject option using a hinge loss. J. Mach. Learn. Res. 9, 1823–1840 (2008). https://dl.acm.org/citation.cfm?id=1442792
Chow, C.: On optimum error and reject trade-off. IEEE Trans. Inf. Theory 16, 41–46 (1970)
Denis, C., Hebiri, M., Zaoui, A.: Regression with reject option and application to KNN (2021)
Duffing, G., Emde, F.: Erzwungene schwingungen bei veränderlicher eigenfrequenz und ihre technische bedeutung
Fischer, L., Hammer, B., Wersing, H.: Optimal local rejection for classifiers. Neurocomputing 214, 445–457 (2016). https://doi.org/10.1016/j.neucom.2016.06.038
Geifman, Y., El-Yaniv, R.: SelectiveNet: a deep neural network with an integrated reject option (2019)
Hendrickx, K., Perini, L., der Plas, D.V., Meert, W., Davis, J.: Machine learning with a reject option: a survey. CoRR abs/2107.11277 (2021). https://arxiv.org/abs/2107.11277
Herbei, R., Wegkamp, M.H.: Classification with reject option. Can. J. Stat./La Revue Canadienne de Statistique 34(4), 709–721 (2006). https://www.jstor.org/stable/20445230
Hinder, F., Artelt, A., Hammer, B.: Towards non-parametric drift detection via dynamic adapting window independence drift detection (DAWIDD). In: Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13–18 July 2020, Virtual Event. Proceedings of Machine Learning Research, vol. 119, pp. 4249–4259. PMLR (2020). https://proceedings.mlr.press/v119/hinder20a.html
Jakob, J., Hasenjäger, M., Hammer, B.: On the suitability of incremental learning for regression tasks in exoskeleton control. In: IEEE Symposium on Computational Intelligence in Data Mining (CIDM). IEEE, December 2021
Jiang, W., Zhao, Y., Wang, Z.: Risk-controlled selective prediction for regression deep neural network models. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2020). https://doi.org/10.1109/IJCNN48605.2020.9207676
Kriegel, H., Kröger, P., Schubert, E., Zimek, A.: Loop: local outlier probabilities (2009)
Lorenz, E.: Deterministic nonperiodic flow. Journal of the atmospheric sciences 20, 130–41.1. Prog. Phys. Geogr. Earth Environ. 32(4), 475–480 (2008). https://doi.org/10.1177/0309133308091948
Losing, V., Hasenjaeger, M.: NEWBEE: a multi-modal gait database of natural everyday-walk in an urban environment. Sci. Data (2022, submitted)
Losing, V., Hammer, B., Wersing, H.: Incremental on-line learning: a review and comparison of state of the art algorithms. Neurocomputing 275, 1261–1274 (2018). https://doi.org/10.1016/j.neucom.2017.06.084
Losing, V., Yoshikawa, T., Hasenjäger, M., Hammer, B., Wersing, H.: Personalized online learning of whole-body motion classes using multiple inertial measurement units. In: International Conference on Robotics and Automation, ICRA 2019, Montreal, QC, Canada, 20–24 May 2019, pp. 9530–9536. IEEE (2019). https://doi.org/10.1109/ICRA.2019.8794251
Montiel, J., et al.: River: machine learning for streaming data in python. J. Mach. Learn. Res. 22(110), 1–8 (2021). https://jmlr.org/papers/v22/20-1380.html
Nadeem, M.S.A., Zucker, J.D., Hanczar, B.: Accuracy-rejection curves (ARCS) for comparing classification methods with a reject option. In: Machine Learning in Systems Biology, pp. 65–81. PMLR (2009)
Nusse, H., Yorke, J.: Dynamics: Numerical Explorations. Springer, New York (1997)
Page, E.S.: Continuous inspection schemes. Biometrika 41(1–2), 100–115 (1954). https://doi.org/10.1093/biomet/41.1-2.100
Rössler, O.: An equation for continuous chaos. Phys. Lett. A 57(5), 397–398 (1976). https://doi.org/10.1016/0375-9601(76)90101-8. https://www.sciencedirect.com/science/article/pii/0375960176901018
Tijjani, I., Kumar, S., Boukheddimi, M.: A survey on design and control of lower extremity exoskeletons for bipedal walking. Appl. Sci. 12(5) (2022). https://doi.org/10.3390/app12052395. https://www.mdpi.com/2076-3417/12/5/2395
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jakob, J., Hasenjäger, M., Hammer, B. (2022). Reject Options for Incremental Regression Scenarios. In: Pimenidis, E., Angelov, P., Jayne, C., Papaleonidas, A., Aydin, M. (eds) Artificial Neural Networks and Machine Learning – ICANN 2022. ICANN 2022. Lecture Notes in Computer Science, vol 13532. Springer, Cham. https://doi.org/10.1007/978-3-031-15937-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-15937-4_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15936-7
Online ISBN: 978-3-031-15937-4
eBook Packages: Computer ScienceComputer Science (R0)