Xiang Li
ABSTRACT
Despite been extensively explored, current techniques in sequential data modeling and prediction are generally designed for solving regression tasks in a batch learning setting, making them not only computationally inefficient but also poorly scalable in real-world applications, especially for real-time intelligent ocean data quality control (QC), where the data arrives sequentially and the QC should be conducted in real time. This paper investigates the online learning for ocean data streams by resolving two main challenges: (i) how to develop a deep learning model to capture the complex ocean data distribution that could evolve dynamically, namely tackling the 'concept drift' problem for non-stationary time series; (ii) how to develop a deep learning model that can dynamically adapt its structure from shallow to deep with the inflow of the data to overcome under-fitting problem, namely tackling the 'model selection' problem. To tackle these challenges, we propose one Evolutive Convolutional Neural Network (ECNN) that dynamically re-weighting the sub-structure of the model from data streams in a sequential or online learning fashion, by which the capacity scalability and sustainability are introduced into the model. The experiments on real ocean observation data verify the effectiveness of our model. As far as we know, it is the first work that introduce online deep learning techniques into ocean data prediction research.
- Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, and Yoram Singer. 2006. Online passive aggressive algorithms. (2006).Google Scholar
- Mark Dredze, Koby Crammer, and Fernando Pereira. 2008. Confidence-weighted linear classification. In Proceedings of the 25th international conference on Machine learning. 264--271.Google ScholarDigital Library
- Heitor M Gomes, Albert Bifet, Jesse Read, Jean Paul Barddal, Fabrício Enembreck, Bernhard Pfharinger, Geoff Holmes, and Talel Abdessalem. 2017. Adaptive random forests for evolving data stream classification. Machine Learning 106, 9 (2017), 1469--1495.Google ScholarDigital Library
- Sudipto Guha, Nina Mishra, Gourav Roy, and Okke Schrijvers. 2016. Robust random cut forest based anomaly detection on streams. In International conference on machine learning. PMLR, 2712--2721.Google Scholar
- James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114, 13 (2017), 3521--3526.Google ScholarCross Ref
- Boduo Li, Yanlei Diao, and Prashant Shenoy. 2015. Supporting scalable analytics with latency constraints. Proceedings of the VLDB Endowment 8, 11 (2015), 1166--1177.Google ScholarDigital Library
- Shuyu Lin, Ronald Clark, Robert Birke, Sandro Schönborn, Niki Trigoni, and Stephen Roberts. 2020. Anomaly Detection for Time Series Using VAE-LSTM Hybrid Model. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 4322--4326.Google Scholar
- Anh Vu Luong, Tien Thanh Nguyen, and Alan Wee-Chung Liew. 2020. Streaming active deep forest for evolving data stream classification. arXiv preprint arXiv:2002.11816 (2020).Google Scholar
- Sepehr Maleki, Sasan Maleki, and Nicholas R Jennings. 2021. Unsupervised anomaly detection with LSTM autoencoders using statistical data-filtering. Applied Soft Computing 108 (2021), 107443.Google ScholarCross Ref
- H Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, et al. 2013. Ad click prediction: a view from the trenches. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. 1222--1230.Google ScholarDigital Library
- Mehdi Mohammadi, Ala Al-Fuqaha, Sameh Sorour, and Mohsen Guizani. 2018. Deep learning for IoT big data and streaming analytics: A survey. IEEE Communications Surveys & Tutorials 20, 4 (2018), 2923--2960.Google ScholarDigital Library
- Mohsin Munir, Shoaib Ahmed Siddiqui, Andreas Dengel, and Sheraz Ahmed. 2018. DeepAnT: A deep learning approach for unsupervised anomaly detection in time series. Ieee Access 7 (2018), 1991--2005.Google ScholarCross Ref
- German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. 2019. Continual lifelong learning with neural networks: A review. Neural Networks 113 (2019), 54--71.Google ScholarDigital Library
- Doyen Sahoo, Steven Hoi, and Peilin Zhao. 2016. Cost sensitive online multiple kernel classification. In Asian Conference on Machine Learning. PMLR, 65--80.Google Scholar
- Doyen Sahoo, Steven CH Hoi, and Bin Li. 2014. Online multiple kernel regression. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 293--302.Google ScholarDigital Library
- Doyen Sahoo, Steven CH Hoi, and Bin Li. 2019. Large scale online multiple kernel regression with application to time-series prediction. ACM Transactions on Knowledge Discovery from Data (TKDD) 13, 1 (2019), 1--33.Google Scholar
- Sebastian Thrun and Lorien Pratt. 2012. Learning to learn. Springer Science & Business Media.Google Scholar
- Huiju Wang, Mengxuan Li, and Xiao Yue. 2021. IncLSTM: Incremental Ensemble LSTM Model towards Time Series Data. Computers & Electrical Engineering 92 (2021), 107156.Google ScholarCross Ref
- Yue Wu, Steven CH Hoi, Chenghao Liu, Jing Lu, Doyen Sahoo, and Nenghai Yu. 2017. SOL: A library for scalable online learning algorithms. Neurocomputing 260 (2017), 9--12.Google ScholarDigital Library
- Martin Zinkevich. 2003. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (icml-03). 928--936.Google ScholarDigital Library
Index Terms
- ECNN: One Online Deep Learning Model for Streaming Ocean Data Prediction
Recommendations
Soil seismic response modeling of KiK-net downhole array sites with CNN and LSTM networks
AbstractAccurate prediction of soil seismic response is necessary for geotechnical engineering. The conventional physics-based models such as the finite element method (FEM) usually fail to obtain accurate predictions due to the model assumption and ...
Highlights- CNN and LSTM models are proposed to develop data-driven surrogate models for the prediction of soil seismic response.
- Recorded acceleration time histories from downhole array sites are applied to train the deep learning models.
- The ...
Application of machine learning in ocean data
AbstractIn recent years, machine learning has become a hot research method in various fields and has been applied to every aspect of our life, providing an intelligent solution to problems that could not be solved or difficult to be solved before. Machine ...
Deep visual tracking
The first comprehensive survey on deep-learning-based trackers.Review existing deep visual trackers from three different perspectives.Large-scale benchmark evaluations of deep visual trackers.Summarize cutting-edge research works and discuss future ...
Comments