Consistency of multilayer perceptron regression estimators

doi:10.1016/S0893-6080(09)80011-7

Neural Networks

Volume 6, Issue 7, 1993, Pages 1019-1022

https://doi.org/10.1016/S0893-6080(09)80011-7 Get rights and content

Abstract

In the paper three layer perceptron with one hidden layer and the output layer consisting of one neuron is considered. This is commonly used architecture to solve regression problems where such a perceptron minimizing the mean squared error criterion for the data points (x_k, y_k), k = 1,…, N is sought. It is shown that in the model: y_k = g₀(x_k) + εk, k = 1, … N, where x_k is independent from zero mean error term ε_k, this procedure is consistent when N → ∞, provided that g₀ is represented as three layer perceptron with Heaviside transfer fucntion. The same result is true when transfer function is an arbitrary continuous function with bounded limits at ±∞ and the hidden-to-output weights in the considered family of perceptions are bounded.

References (13)

F. Murtagh
Multilayer perceptrons for classification and regression
Neurocomputing
(1991)
H. White
Connectionist nonparametric regression: Multilayer feedforward networks can learn arbitrary mappings
Neural Networks
(1990)
E. Baum et al.
What size net gives valid generalization?
Neural Computation
(1989)
Y. Chauvin
Generalization performance of overtrained back propagation networks
R.M. Dudley
A course on empirical processes
G.J. Gibson et al.
On the decision regions for multilayer perceptrons

There are more references available in the full text version of this article.

Cited by (31)

Artificial Intelligence aided pharmaceutical engineering: Development of hybrid machine learning models for prediction of nanomedicine solubility in supercritical solvent
2024, Journal of Molecular Liquids
This study aims to model the relationship between pressure (P) and temperature (T) inputs and the corresponding outputs of supercritical carbon dioxide (SC-CO₂) density and mole fraction of rivaroxaban as the solute. Three models employed in this work included Support Vector Regression (SVR), Multilayer Perceptron (MLP), and LASSO. The hyper-parameter tuning was performed using Tabu Search approach. The results revealed that MLP is the best model for predicting SC-CO₂ density, achieving an R² score of 0.9994 and demonstrating high accuracy and precision for correlating the density dataset. However, for the mole fraction of rivaroxaban (solubility), SVR outperformed the other models. It revealed an R² value of 0.98037, confirming a strong correlation between the inputs and outputs. SVR also exhibits a lower mean squared error and maximum error compared to MLP and LASSO for the mole fraction prediction. Although MLP excels in predicting SC-CO₂ density, SVR proves to be the superior model for accurately estimating the mole fraction of rivaroxaban based on the given inputs. These findings contribute to the understanding and modeling of SC-CO₂ properties, particularly the mole fraction of rivaroxaban, which can be valuable in various research and practical applications.
Numerical optimization of drug solubility inside the supercritical carbon dioxide system using different machine learning models
2023, Journal of Molecular Liquids
This research comprehensively investigates the solubility characteristics of five distinct drugs including: Nystatin, Niflumic acid, Tolfenamic acid, Glibenclamide, and Rivaroxaban, across a range of pressure (P) and temperature (T) conditions. The solubility is computed in supercritical carbon dioxide as the solvent. It was aimed to build a holistic view of solubility estimation using machine learning technique. To predict drug solubility accurately, three regression models— K-Nearest Neighbors (KNN), Multilayer Perceptron (MLP), and Polynomial Regression (PR)—were employed, with hyperparameter optimization conducted using the Harmony Search (HS) algorithm. Performance evaluation metrics, including R-squared (R²) scores, Root Mean Square Error (RMSE), and Maximum Error, were employed to assess model effectiveness. Notably, HS-PR emerged as the top-performing model, achieving an impressive score of 0.96449 in terms of R² metric, highlighting its proficiency in modeling drug solubility under varying conditions.
Machine learning-based prediction and mathematical optimization of Capecitabine solubility through the supercritical CO<inf>2</inf> system
2023, Journal of Molecular Liquids
This work represents development of computational intelligence models for assessment of estimating solubility/bioavailability of orally-taken drugs in SCF (supercritical fluid). The SCF considered in this work is CO₂. This approach can provide an appropriate opportunity for the development of sustainable and environmentally-friendly technology with valuable advantages in the drug delivery industry. We modeled the solubility of Capecitabine in CO₂SCF using multiple machine learning models. The inputs are Pressure and Temperature in the used dataset and the single output (Y) is the solubility of the drug. Three well-known supervised learning models of KNN (K-Nearest Neighbor), MLP (Multilayer Perceptron), and SVR (Support Vector Regression) were selected for modeling in this research. The hyper-parameters of them were optimized using Cuckoo search algorithm (CS) and then final models assessed using multiple metrics. In terms of coefficient of determination, KNN, MLP, and SVR models have score values of 0.959, 0.996, and 0.952, respectively. Also, they have error rates of 13.78, 1.54, and 19.23 with MSE metric. These facts introduce MLP as the best model obviously and so we used this model for final analysis in this study due to its superior architecture.
Machine learning aided drug development: Assessing improvement of drug efficiency by correlation of solubility in supercritical solvent for nanomedicine preparation
2023, Journal of Molecular Liquids
In this study, the solubility of phenytoin and the supercritical CO₂ (solvent) density were investigated using three different models: Multilayer Perceptron (MLP), NU-SVR, and Poisson Regressor (POR). The models were optimized using the BA algorithm. The inputs for the models were temperature (T) in Kelvin and pressure (P) in bar, while the outputs were the solubility (y) of phenytoin and the CO₂ density. For the solubility prediction, the MLP model achieved an impressive R² score of 0.998 with a MSE of 0.051, a MAE of 0.200, and a MAPE of 0.080. The NU-SVR model also performed well with an R² score of 0.966, MSE of 1.094, MAE of 0.708, and MAPE of 0.194. The POR model yielded an R² score of 0.957, MSE of 0.959, MAE of 0.765, and MAPE of 0.337. For the CO₂ density prediction, the MLP model achieved an excellent R² score of 0.998 with an MSE of 75.232, MAE of 5.866, and MAPE of 0.018. The NU-SVR model performed reasonably well with an R² score of 0.858, MSE of 3853.3, MAE of 49.697, and MAPE of 0.119. The POR model yielded an R² score of 0.839, MSE of 4012.3, MAE of 52.297, and MAPE of 0.140. These results reveal that the MLP model provides highly reliable forecasts for both the solubility of phenytoin and the CO₂ density.
Mathematical optimization and prediction of Febuxostat xanthine oxidase inhibitor solubility through supercritical CO<inf>2</inf> system using machine-learning approach
2023, Journal of Molecular Liquids
Scientific research towards the synthesis and discovery of novel therapeutic agents with optimal safety, great biological efficiency and acceptable toxicity profile is known as the driving force of pharmaceutical industry in the recent decades. This research study intends to mathematically predict the solubility of Febuxostat (FBX) as a prevalent type of xanthine oxidase inhibitor in the supercritical CO₂ (SCCO₂) system. For this purpose, Quantile Regression (QR), Multi Linear Regression (MLR), and Multilayer perceptron (MLP) were used to establish models for the prediction of solubility of FBX drug based on temperature and pressure features. The settings of the models were optimized based on their important hyper-parameters and then evaluated and compared. In the final evaluation, models MLP, MLR, and QR have R² scores of 0.999, 0.941, and 0.854. Also, the RMSE error rate for model MLP is equal to 5.97E-02, for model MLR is equal to 4.77E-01, and for model QR is equal to 6.73E-01. Therefore, MLP can be recognized as the best model among these three methods. The MAPE error metric for this model is 8.26E-02.
Development of green technology based on supercritical solvent for production of nanomedicine: Solubility prediction using computational methods
2023, Journal of Molecular Liquids
Supercritical solvent-based engineering integrated nanomedicine has been developed in this study for development of advanced pharmaceutical manufacturing. Analysis of machine learning was employed to estimate the solubility of drug in supercritical CO₂. Indeed, bioavailability of solid-dosage oral formulations is of great importance in which the majority of newly invented drugs possess poor water solubility which make them inefficient for patients. In order to improve drugs bioavailability, their solubility need to be increased which can be done by nanonization of drugs. The method of computation is machine learning in which different algorithms are selected and tuned to best fit the solubility data. For this purpose, we employed a tiny data set with pressure and temperature as input features and solubility as an output. Three models, the Multilayer Perceptron (MLP), the Kernel Ridge Regression (KRR), and the Gaussian Process Regression (GPR), have been employed to examine and model the data. Finally, the tuned models were created by optimizing the three models' hyper-parameters with the help of the Bat optimization algorithm (BA). In the end, the models were assessed using multiple metrics. On the basis of the R² metric, the GPR model was found to be the most effective. In addition, the MAPE criterion yields a final model error of 2.52 × 10^-2, the RMSE criterion yields an error of 1.96 × 10^-2, and the Max error value yields an error of 2.90 × 10^-2.

View all citing articles on Scopus

^*: The work was partly done while the author was a Visiting Researcher with the SANS group at the Department of Numerical Analysis and Computing Science, Royal Institute of Technology, Stockholm, Sweden.

View full text

Consistency of multilayer perceptron regression estimators

Abstract

Neurocomputing

Neural Networks

What size net gives valid generalization?

Neural Computation

Generalization performance of overtrained back propagation networks

A course on empirical processes

On the decision regions for multilayer perceptrons