Jochen Merker
ABSTRACT
In this article, we suggest a way how to quantify the sensitivity of a data model w.r.t. training data at a certain input by using a derived model having a single data point as artificial parameter, and we relate our definition of sensitivity w.r.t. training data to the complexity of a data model. In case of linear regression and ridge regression we give an explicit expression for the so defined sensitivity w.r.t. training data and study the properties. Moreover, we discuss the numerical approximation of sensitivity w.r.t. training data for neural networks and provide an application of this numerical approximation to remote sensing in the environmental sciences.
- Peter L. Bartlett, Philip M. Long, Gábor Lugosi, and Alexander Tsigler. 2020. Benign overfitting in linear regression. Proc. Nat. Acad. Sci. U. S. A. 117, 48 (2020), 30063–30070.Google Scholar
Cross Ref
- Jochen Merker and Gregor Schuldt. 2021. An attempt to explain double descent in modern machine learning. In Tagungsband zur 26. Interdisziplinären Wissenschaftlichen Konferenz Mittweida (IWKM), 14.-15.04.2021. Scientific Reports 2021, 141–144. https://doi.org/10.48446/opus-12293Google ScholarCross Ref
- Jochen Merker and Gregor Schuldt. 2021. Why LASSO seems to simultaneously decrease bias and variance in machine learning. In Proceedings of ICoMS 2021. ACM, 86–89. https://doi.org/10.1145/3475827.3475839Google ScholarDigital Library
- W. Schimmel, H. Kalesse-Los, M. Maahn, T. Vogl, A. Foth, P. S. Garfias, and P. Seifert. 2022. Identifying cloud droplets beyond lidar attenuation from vertically pointing cloud radar observations using artificial neural networks. Atmospheric Measurement Techniques 15, 18 (2022), 5343–5366. https://doi.org/10.5194/amt-15-5343-2022Google ScholarCross Ref
- Alexander Tsigler and Peter L. Bartlett. 2020. Benign Overfitting in Ridge Regression. arxiv:2009.14286Google Scholar
- V. N. Vapnik. 1995. The Nature of Statistical Learning Theory. Springer.Google ScholarDigital Library
Index Terms
- On Sensitivity of Data Models w.r.t. Training Data
Recommendations
Computation of Adalines' sensitivity to weight perturbation
In this paper, the sensitivity of Adalines to weight perturbation is discussed. According to the discrete feature of Adalines' input and output, the sensitivity is defined as the probability of an Adaline's erroneous outputs due to weight perturbation ...
Exploiting unlabeled data to enhance ensemble diversity
Ensemble learning learns from the training data by generating an ensemble of multiple base learners. It is well-known that to construct a good ensemble with strong generalization ability, the base learners are deemed to be accurate as well as diverse. ...
Synthetic data generation method for data-free knowledge distillation in regression neural networks
AbstractKnowledge distillation is the technique of compressing a larger neural network, known as the teacher, into a smaller neural network, known as the student, while still trying to maintain the performance of the larger neural network as ...
Comments