Nikolaos Bakalos
ABSTRACT
Man overboard incidents in a maritime vessel are serious accidents where, the efficient and rapid detection is crucial in the recovery of the victim. The severity of such accidents, urge the use of intelligent systems that are able to automatically detect a fall and provide relevant alerts. To this end the use of novel deep learning and computer vision algorithms have been tested and proved efficient in problems with similar structure. This paper presents the use of a deep learning framework for automatic detection of man overboard incidents. We investigate the use of simple RGB video streams for extracting specific properties of the scene, such as movement and saliency, and use convolutional spatiotemporal autoencoders to model the normal conditions and identify anomalies. Moreover, in this work we present a dataset that was created to train and test the efficacy of our approach.
- E. Örtlund and M. Larsson, “Man Overboard detecting systems based on wireless technology,” 2018.Google Scholar
- Sevin, C. BAYILMIŞ, İ. ERTÜRK, H. EKİZ, and A. Karaca, “Design and implementation of a man-overboard emergency discovery system based on wireless sensor networks,” Turk. J. Electr. Eng. Comput. Sci., vol. 24, no. 3, pp. 762–773, 2016.Google Scholar
- I. Katsamenis , E. Protopapadakis, A. S. Voulodimos, D. Dres, and D. Drakoulis. "Man overboard event detection from rgb and thermal imagery: Possibilities and limitations," In Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive Environments, pp. 1-6. 2020.Google Scholar
- A. S. Voulodimos, D. I. Kosmopoulos, N. D. Doulamis, and T. A. Varvarigou, “A top-down event-driven approach for concurrent activity recognition,” Multimed. Tools Appl., vol. 69, no. 2, pp. 293–311, Mar. 2014, doi: 10.1007/s11042-012-0993-4.Google ScholarDigital Library
- N. D. Doulamis, A. S. Voulodimos, D. I. Kosmopoulos, and T. A. Varvarigou, “Enhanced Human Behavior Recognition Using HMM and Evaluative Rectification,” in Proceedings of the First ACM International Workshop on Analysis and Retrieval of Tracked Events and Motion in Imagery Streams, New York, NY, USA, 2010, pp. 39–44, doi: 10.1145/1877868.1877880.Google ScholarDigital Library
- K. Makantasis, E. Protopapadakis, A. Doulamis, N. Doulamis, and N. Matsatsinis, “3D measures exploitation for a monocular semi-supervised fall detection system,” Multimed. Tools Appl., vol. 75, no. 22, pp. 15017–15049, Nov. 2016, doi: 10.1007/s11042-015-2513-9.Google ScholarDigital Library
- C. Rougier, J. Meunier, A. St-Arnaud, and J. Rousseau, “Robust video surveillance for fall detection based on human shape deformation,” IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 5, pp. 611–622, 2011.Google ScholarDigital Library
- M. Yu, A. Rhuma, S. M. Naqvi, L. Wang, and J. Chambers, “A posture recognition-based fall detection system for monitoring an elderly person in a smart home environment,” IEEE Trans. Inf. Technol. Biomed., vol. 16, no. 6, pp. 1274–1286, 2012.Google ScholarDigital Library
- S. Lee, H. G. Kim and Y. M. Ro, "BMAN: Bidirectional Multi-Scale Aggregation Networks for Abnormal Event Detection," IEEE Trans. on Image Proc., vol. 29, pp. 2395-2408, 2020.Google ScholarDigital Library
- A. S. Voulodimos, N.D. Doulamis, D.I. Kosmopoulos, and T.A. Varvarigou, “Improving multi-camera activity recognition by employing neural network based readjustment,” Applied Artificial Intelligence, 26(1-2), 97-118, 2012.Google ScholarCross Ref
- N. Bakalos, "Protecting water infrastructure from cyber and physical threats: Using multimodal data fusion and adaptive deep learning to monitor critical systems." IEEE Signal Processing Magazine, 36.2, pp. 36-48, 2019.Google ScholarCross Ref
- S. Wan, X. Xu, T. Wang and Z. Gu, "An Intelligent Video Analysis Method for Abnormal Event Detection in Intelligent Transportation Systems," IEEE Trans. on Intell. Transportation Systems, (to be published)Google Scholar
- R. Leyva, V. Sanchez and C. Li, "Fast Detection of Abnormal Events in Videos with Binary Features," IEEE ICASSP, Calgary, AB, pp. 1318-1322, 2018.Google Scholar
- K.-W. Cheng, Y.-T. Chen, and W.-H. Fang, “Video anomaly detection and localization using hierarchical feature repre- sentation and Gaussian process regression,” IEEE CVPR, pp. 2909–2917, 2015.Google Scholar
- C. Lu, J. Shi, and J. Jia, “Abnormal Event Detection at 150 FPS in MATLAB,” IEEE ICCV, pages 2720– 2727, 2013.Google ScholarDigital Library
- H. Ren, W. Liu, S. I. Olsen, S. Escalera, and T. B. Moes- lund, “Unsupervised Behavior-Specific Dictionary Learning for Abnormal Event Detection,” Proc. of BMVC, pp. 28.1–28.13, 2015.Google Scholar
- M. Hasan, J. Choi, J. Neumann, A. K. Roy-Chowdhury, and L. S. Davis, “Learning temporal regularity in video sequences,” IEEE CVPR, pages 733–742, 2016.Google ScholarCross Ref
- Xu Dan, Ricci Elisa, Yan Yan, Song Jingkuan and Sebe Nicu, "Learning Deep Representations of Appearance and Motion for Anomalous Event Detection", BMVC, 2015.Google Scholar
- L. Wang, F. Zhou, Z. Li, W. Zuo and H. Tan, "Abnormal Event Detection in Videos Using Hybrid Spatio-Temporal Autoencoder," 25th IEEE International Conference on Image Processing (ICIP), Athens, 2018, pp. 2276-2280, 2018.Google Scholar
- A. Del Giorno, J. Bagnell, and M. Hebert, “A Discrimina- tive Framework for Anomaly Detection in Large Videos,” Proc. of ECCV, pp. 334–349, 2016.Google Scholar
- J. K. Dutta and B. Banerjee, “Online Detection of Abnormal Events Using Incremental Coding Length,” In Proceedings of AAAI, pages 3755–3761, 2015.Google ScholarCross Ref
- R.T. Ionescu, S. Smeureanu, B. Alexe, and M. Popescu, “Un-masking the abnormal events in video,” IEEE ICCV, pp. 2895–2903, 2017.Google Scholar
- S. Smeureanu, R. T. Ionescu, M. Popescu, and B. Alexe, “Deep Appearance Features for Abnormal Behavior Detection in Video,” In Proceedings of ICIAP, Volume 10485, pages 779–789, 2017.Google ScholarCross Ref
- Y. Liu, C.-L. Li, and B. Poczos, “Classifier Two-Sample Test for Video Anomaly Detections,” In Proceedings of BMVC, 2018.Google Scholar
- X. Mo, V. Monga, R. Bala, and Z. Fan, “Adaptive sparse representations for video anomaly detection,” IEEE Trans. Circuits Syst. Video Technol., vol. 24, no. 4, pp. 631–645, Apr. 2014.Google ScholarCross Ref
- F. Jiang, Y. Wu, and A. K. Katsaggelos, “A dynamic hierarchical clustering method for trajectory-based unusual video event detection,” IEEE Trans. Image Process., vol. 18, no. 4, pp. 907–913, Apr. 2009.Google ScholarDigital Library
- L. Li, K. Jamieson, G. DeSalvo, A. Rostamizadeh, and A. Talwalkar, “Hyperband: A novel bandit-based approach to hyperparameter optimization,” The Journal of Machine Learning Research, 18(1), pp.6765-6816, 2016.Google Scholar
- Kaselimi, Maria, "Bayesian-optimized bidirectional LSTM regression model for non-intrusive load monitoring." ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019Google Scholar
Recommendations
Man overboard event detection from RGB and thermal imagery: possibilities and limitations
PETRA '20: Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive EnvironmentsA man overboard is an emergency incident, in which fast detection is the most crucial factor, for the quickest and most efficient recovery of the victim. As such, efficient monitoring methodologies should be employed. A variety of sensors is available ...
Using simulation to measure the performance of a search operation for a man overboard
ANSS '18: Proceedings of the Annual Simulation SymposiumA man overboard is defined as a situation in which a person falls out at sea from a ship and is in need of rescue. In such cases, a quick and effective search and rescue (SAR) operation are crucial to increase the survival probability of the victim. If ...
MOBDrone: A Drone Video Dataset for Man OverBoard Rescue
Image Analysis and Processing – ICIAP 2022AbstractModern Unmanned Aerial Vehicles (UAV) equipped with cameras can play an essential role in speeding up the identification and rescue of people who have fallen overboard, i.e., man overboard (MOB). To this end, Artificial Intelligence techniques can ...
Comments