Constructive multi-output extreme learning machine with application to large tanker motion dynamics identification
Introduction
On the one hand, the promising extreme learning machine (ELM) method for single-hidden layer feedforward networks (SLFNs) has attracted comprehensive and intensive research since Huang et al. [1] proposed the seminal work. The main idea of ELM strategy is intuitively realized as follows. Hidden nodes are randomly generated and output weights are analytically determined by pseudo-inverse technologies. It is evident that the ELM is an extremely fast batch learning algorithm and can provide good generalization performance [2]. As a consequence, the ELM does not need any iterations to determine the hidden node parameters, and dramatically reduces the computational time for training process. Actually, the randomness and diversity of hidden nodes should be guaranteed for high generalization performance. In this case, the determination for the suitable or optimal number of randomly generated hidden nodes becomes an interesting and critical issue to elaborate the ELM advantages. However, the original ELM [3] does not provide any effective solution for architectural design of the network. In most cases, the suitable number of hidden nodes is pre-defined by a trial and error method, which may be tedious in some applications.
In order to circumvent the above-mentioned problems, some improvements to the ELM for optimal structure have been proposed in two heuristic approaches, i.e., destructive and constructive methods, which have been effectively implemented in fuzzy neural networks [4], [5], [6]. For the former approach, Rong et al. [7] have presented a pruned ELM (P-ELM), for classification problems, which starts with a large network and then eliminates the hidden nodes having low relevance to the output. Miche et al. [8] have proposed an optimally pruned ELM (OP-ELM) by using the multi-response sparse regression (MRSR) algorithm [9] and leave-one-out (LOO) validation for pruning strategy. Evidently, these methods within destructive paradigms would face common difficulties that the algorithm starts with a large scale structure which inevitably increase the computational burden. For the latter approach, the incremental extreme learning machine (I-ELM) [2] and its variants [10], [11] proposed by Huang et al. are proposed to add hidden nodes one-by-one to the hidden layer and incrementally update output weights. However, those algorithms cannot lead to an optimal network structure automatically, and hidden nodes are added to the SLFN merely in one-by-one manner. Feng et al. [12] have proposed the error minimized extreme learning machine (EM-ELM), which can add random hidden nodes one-by-one or group-by-group. Unfortunately, the nodes added into the hidden layer are randomly generated and might deteriorate the performance with increasing hidden nodes since no generalization measure is guaranteed. Nevertheless, the resultant network structure would be much similar to original ELM if high prediction performance is required. A constructive hidden nodes selection of extreme learning machine termed as CS-ELM is proposed by Lan et al. [13], whereby the hidden nodes are selected by the MRSR and unbiased risk estimation based criterion Cp. However, the CS-ELM works for single-output regression and the hidden node selection conducts in one-by-one manner.
On the other hand, as the potential application in this paper, the large tanker maneuvering dynamics play a fundamental role in the whole guidance, navigation and control (GNC) system. Focusing on this essential issue, many researchers have proposed varieties of vessel motion models, mainly including Abkowitz model, MMG model and response model [14]. These three types of models preserve distinct features as follows. Abkowitz model pursues accurate hydrodynamic derivatives at the cost of clear presentation for variables, and therefore results in difficulties for control system design. To the contrary, MMG model and response model focus on analysis and synthesis of model based control systems while the model accuracy would be lower since MMG and response models could be considered as simplifications of the Abkowitz model to some extent.
Within the previous model frameworks of vessel motion, studies on system identification for hydrodynamic derivatives and input–output nonlinearities have been conducted by using various methods, i.e., simplified linearization [15], estimation-before-modeling technique [16], and support vector regression method [17], etc. However, the resultant overall mathematical formulation of vessel maneuvering is usually complicated due to the existence of hydrodynamic nonlinearities associated with the vessel dynamics. In this case, there exists a dilemma between the accuracy and interpretation of vessel motion models using traditional methods.
In addition, the use of large tankers becomes an important issue since the demand of transportation for crude oil has increased. System identification for large tanker motion dynamics becomes an involved task due to the maneuverability difficulties caused by their bulk. Unfortunately, comparing with the previous investigations of general vessels, promising results of large tanker maneuvering models are short of appearance and mainly focus on controller design rather than motion dynamics identification [18]. Typically, van Berlekom made an excellent seminal research on Esso Osaka tanker model, whereby the hydrodynamic derivatives have been proposed in detail [14].
Recently, in order to overcome the above-mentioned problems, researchers appeal to artificial neural networks (ANNs) in the field of artificial intelligence technology which can be used to establish nonlinear input–output models for ship maneuvering motion effectively. Mahfouz and Haddara [19] applied the ANN and spectral analysis methods to identify the hydrodynamic derivatives in the mathematical model of marine vehicle motions. Moreira and Guedes Soares [20] proposed a dynamic recurrent neural network (RNN) based maneuvering simulation model for surface ships. Rajesh et al. [18] identified an interesting nonlinear maneuvering model of large tankers based on back propagation (BP) neural networks. Certainly, the ANN based system identification method could obtain considerable performance for approximation and generalization. However, the nonlinearities underlying between input and output variables would also be folded into a “black box” which is difficult to be interpreted and understood.
Motivated by the previous reviews, we present a novel constructive multi-output extreme learning machine (CM-ELM) for large tanker motion dynamics identification in this paper. The underlying main idea could be implemented as follows. A group of well established nonlinear differential equations for tanker motion dynamics are used as the reference model for training and testing data generation. In this case, the dynamics identification is equal to a multi-output regression problem that the states, i.e., surge, sway, yaw speed and rudder angle (u, v, r, δ), are input variables, and state derivatives (, , ) are taken as multiple outputs. With data samples at hand, it is followed by the promising data-driven learning method term as CM-ELM for multi-output regression. A candidate pool of hidden nodes in the SLFN is randomly generated by the ELM strategy in the initial phase, and then hidden node ranking and model selection from the candidate pool is implemented by a novel improved MRSR (I-MRSR) method and generalization measure. In the last phase, from the elite subset of model selections, the resulting CM-ELM model is randomly selected to update the output weight based on the whole training data. It should be noted that the proposed CM-ELM method ranks and adds candidate hidden nodes chunk-by-chunk, rather than one-by-one, which would reasonably reduce computational burden and accelerate learning speed. Simulation studies on benchmark multi-output regression datasets validate the effectiveness and superiority of the CM-ELM compared with ELM and OP-ELM, etc. In order to evaluate the CM-ELM application performance of large tanker motion dynamics identification, comprehensive simulations and comparisons of typical maneuvering scenarios are conducted on sine rudder angle input and zigzag maneuvers. The results demonstrate that compared with the ELM, the CM-ELM based tanker motion model with parsimonious structure achieves much promising identification and generalization performance in terms of both moderate and extreme maneuvering.
The rest of this paper is organized as follows. Section 2 briefly presents preliminary formulations of related works. The main idea and contributions to the CM-ELM including candidate pool generation, I-MRSR based ranking, constructive model selection and generalization measure are unfolded in Section 3. Section 4 implements simulation studies on the CM-ELM for benchmark dataset experiments and applications to tanker motion dynamics identification in detail. In Section 5, conclusions are drawn.
Section snippets
Preliminary formulation
In this section, the preliminary formulation of related works, i.e., extreme learning machine (ELM) and multi-response sparse regression (MRSR), will be briefly presented to enhance the foundation knowledge of our proposed learning scheme, which is applied to large tanker motion dynamics identification.
Constructive multi-output extreme learning machine (CM-ELM)
In this section, a novel constructive method for multi-output ELM, termed as CM-ELM, will be presented in detail. Generally speaking, hidden node candidates are randomly generated via original ELM methodology in the initialization phase, and then an improved MRSR (I-MRSR) method is proposed to select potential regressor nodes chunk by chunk, rather than one by one, to accelerate the selection procedure. Finally, the validation and retraining process will be conducted on the selected subset of
Simulation studies
In this section, the performance evaluation and promising application of our proposed CM-ELM algorithm will be carried out. On the one hand, the effective performance and superiority will be evaluated by conducting simulation studies on multi-output regression benchmark datasets. On the other hand, the innovative application of the CM-ELM to large tanker motion dynamics identification will be presented by considering Esso Osaka tanker [14] as the reference model. All the simulations are carried
Conclusions
In this paper, we propose a novel constructive multiple-output extreme learning machine (CM-ELM) method for multiple-output regression problems, and apply the CM-ELM algorithm to large tanker motion dynamics identification. To be specific, the CM-ELM method can be divided into four stages as follows. Initially, a set of Lmax hidden nodes is randomly generated according to the ELM strategy as the candidate pool of our proposed algorithm. An improved multi-response sparse regression (I-MRSR)
Acknowledgments
The authors would like to thank anonymous referees for their invaluable comments and suggestions. This work is supported by the National Natural Science Foundation of PR China (under Grants 51009017, 51379002 and 61074096), Applied Basic Research Funds from Ministry of Transport of PR China (under Grant 2012-329-225-060), China Postdoctoral Science Foundation (under Grant 2012M520629), Program for Liaoning Excellent Talents in University (under Grant LJQ2013055), and Fundamental Research Funds
Ning Wang received his B.Eng. degree in Marine Engineering and the Ph.D. degree in Control Theory and Engineering from the Dalian Maritime University (DMU), Dalian, China, in 2004 and 2009, respectively. From September 2008 to September 2009, he was financially supported by China Scholarship Council (CSC) to work as a joint-training Ph.D. student at the Nanyang Technological University (NTU), Singapore. He is currently an Associate Professor with the Marine Engineering College, Dalian Maritime
References (21)
- et al.
Extreme learning machinetheory and applications
Neurocomputing
(2006) - et al.
A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks
Neurocomputing
(2009) - et al.
A fast pruned-extreme learning machine for classification problem
Neurocomputing
(2008) - et al.
Enhanced random search based incremental extreme learning machine
Neurocomputing
(2008) - et al.
Convex incremental extreme learning machine
Neurocomputing
(2007) - et al.
Constructive hidden nodes selection of extreme learning machine for regression
Neurocomputing
(2010) - et al.
Mathematical models of ship path prediction in manoeuvring simulation systems
Ocean Eng.
(2002) - et al.
Identification of Abkowitz model for ship maneuvering motion using ε-support vector regression
J. Hydrodynamics
(2011) - et al.
System identification for nonlinear maneuvering of large tankers using artificial neural network
Appl. Ocean Res.
(2008) - et al.
Effects of the damping and excitation on the identification of the hydrodynamic parameters for an underwater robotic vehicle
Ocean Eng.
(2003)
Cited by (56)
Self-organizing data-driven prediction model of ship maneuvering fast-dynamics
2023, Ocean EngineeringMath-data integrated prediction model for ship maneuvering motion
2023, Ocean EngineeringReview on deep learning techniques for marine object recognition: Architectures and algorithms
2022, Control Engineering PracticeCitation Excerpt :The selection of classifiers is a vital step for object recognition. The most common classifiers have gained excellent results, for instance, KNN, softmax, SVM and extreme learning machine (ELM) (Huang, Zhou, Ding, & Zhang, 2012; Wang, Han, Dong and Er, 2014; Zhang, He, & Liu, 2017) and variants (e.g. complex-valued ELM (CELM) (Zhang, Wang, Xu, Wang, & Xu, 2018), parsimonious ELM (Wang, Er and Han, 2014)), and logistic regression classifiers (Ma, Khatibisepehr, & Huang, 2015; Zou et al., 2015). In addition, as can be seen from Table 6, the UW-CNN with classifier SVM, in comparison of that with Softmax and the model fine-tuned in last layer, gain the highest recognition accuracy under the same other conditions.
Multi-output parameter-insensitive kernel twin SVR model
2020, Neural Networks
Ning Wang received his B.Eng. degree in Marine Engineering and the Ph.D. degree in Control Theory and Engineering from the Dalian Maritime University (DMU), Dalian, China, in 2004 and 2009, respectively. From September 2008 to September 2009, he was financially supported by China Scholarship Council (CSC) to work as a joint-training Ph.D. student at the Nanyang Technological University (NTU), Singapore. He is currently an Associate Professor with the Marine Engineering College, Dalian Maritime University (DMU), Dalian 116026, China. His research interests include artificial neural networks, fuzzy systems, machine learning, ship intelligent control, and dynamic ship navigational safety assessment. Dr. Wang also received the DMU Excellent Doctoral Dissertation and the DMU Outstanding Ph.D. Student Award in 2010.
Min Han received the B.S. and M.S. degrees from the Department of Electrical Engineering, Dalian University of Technology, Dalian, China, and the M.S. and Ph.D. degrees from Kyushu University, Fukuoka, Japan, in 1982, 1993, 1996, and 1999, respectively. She is a Professor with the Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology. Her current research interests include neural networks and chaos and their applications to control and identification.
Nuo Dong received the B.S. degree from the Department of Electrical Engineering and Automation in Marine Engineering College, Dalian Maritime University, Dalian 116026, China, in July 2012. He is currently pursuing his M.S. degree at the same university. His current research interests include unmanned crafts and their intelligent modeling and control.
Meng Joo Er is currently a Full Professor in Electrical and Electronic Engineering, Nanyang Technological University, Singapore, and a Chair Professor of the Dalian Marinetime University, PRC. He served as the Founding Director of Renaissance Engineering Programme, College of Engineering (CoE) and an elected member of the NTU Advisory Board from 2009 to 2012. He also served as a member of the NTU Senate Steering Committee from 2010 to 2012 and the Institution of Engineers, Singapore (IES) Council from 2008 to 2012.
He has authored five books entitled Dynamic Fuzzy Neural Networks: Architectures, Algorithms and Applications and Engineering Mathematics with Real-World Applications published by McGraw Hill in 2003 and 2005, and Theory and Novel Applications of Machine Learning published by In-Tech in 2009, New Trends in Technology: Control, Management, Computational Intelligence and Network Systems and New Trends in Technology: Devices, Computer, Communication and Industrial Systems, both published by SCIYO, 16 book chapters and more than 400 refereed journal and conference papers in his research areas of interest.
In recognition of the significant and impactful contributions to Singapore's development by the research project entitled Development of Intelligent Techniques for Modeling, Controlling and Optimising Complex Manufacturing Systems, Professor Er won the Institution of Engineers, Singapore (IES) Prestigious Engineering Achievement Award 2011. Under his leadership, the NTU Team emerged first runner-up in the Freescale Technology Forum Design Challenge 2008. The NTU is the only Asian Team among the top three positions at the first Freescales first green engineering design contest, reaffirming NTUs strength in design, creativity and innovation. He is also the only dual winner in Singapore IES Prestigious Publication Award in Application (1996) and IES Prestigious Publication Award in Theory (2001). He received the Teacher of the Year Award for the School of EEE in 1999, School of EEE Year 2 Teaching Excellence Award in 2008 and the Most Zealous Professor of the Year Award 2009. He also received the Best Session Presentation Award at the World Congress on Computational Intelligence in 2006. On top of this, he has more than 40 awards at international and local competitions.
Currently, Professor Er serves as the Editor-in-Chief of the International Journal of Electrical and Electronic Engineering and Telecommunications, an Area Editor of International Journal of Intelligent Systems Science and an Associate Editor of 11 refereed international journals, namely International Journal of Fuzzy Systems, Neurocomputing, International Journal of Humanoid Robots, Journal of Robotics, International Journal of Mathematical Control Science and Applications, International Journal of Applied Computational Intelligence and Soft Computing, International Journal of Fuzzy and Uncertain Systems, International Journal of Automation and Smart Technology, International Journal of Modeling, Simulation and Scientific Computing, International Journal of Intelligent Information Processing and the Open Electrical and Electronic Engineering Journal. Furthermore, he served as an Associate Editor of IEEE Transactions on Fuzzy Systems from 2006 to 2011 and a Guest Editor of International Journal of Neural Systems.
Professor Er has been invited to deliver more than 60 keynote speeches and invited talks overseas. He has also been active in professional bodies. Currently, he is the Vice-Chairman of IEEE Computational Intelligence Society (CIS) Standards Committee, Chairman of IEEE Computational Intelligence Society Singapore Chapter and Chairman of IES Electrical and Electronic Engineering Technical Committee. Under his leadership, the IEEE CIS Singapore Chapter won the CIS Outstanding Chapter Award 2012. The Singapore Chapter is the first chapter in Asia to win the award since it was inaugurated in 2006. In recognition of his outstanding contribution to IES, he was awarded the IES Silver Medal in 2011.
Professor Er Meng Joo's areas of expertise are intelligent control theory and applications, fuzzy logic and neural networks and robotics and automation. His current works focus on control theory and applications, fuzzy logic and neural networks, computational intelligence, cognitive systems, robotics and automation, sensor networks and biomedical engineering.