SALINA: Towards Sustainable Live Sonar Analytics in Wild Ecosystems
Authors: 
Chi Xu, Rongsheng Qian, Hao Fang, Xiaoqiang Ma, William I. Atlas, Jiangchuan Liu, Mark A. SpoljaricAuthors Info & Claims
Chi Xu, Rongsheng Qian, Hao Fang, Xiaoqiang Ma, William I. Atlas, Jiangchuan Liu, Mark A. SpoljaricAuthors Info & ClaimsPages 68 - 81
Abstract
Sonar radar captures visual representations of underwater objects and structures using sound wave reflections, making it essential for exploration, mapping, and continuous surveillance in wild ecosystems. Real-time analysis of sonar data is crucial for time-sensitive applications, including environmental anomaly detection and in-season fishery management, where rapid decision-making is needed. However, the lack of both relevant datasets andpre-trained DNN models, coupled with resource limitations in wild environments, hinders the effective deployment and continuous operation of live sonar analytics.
We present SALINA, a sustainable live sonar analytics system designed to address these challenges. SALINA enables real-time processing of acoustic sonar data with spatial and temporal adaptations, and features energy-efficient operation through a robust energy management module. Deployed for six months at two inland rivers in British Columbia, Canada, SALINA provided continuous 24/7 underwater monitoring, supporting fishery stewardship and wildlife restoration efforts. Through extensive real-world testing, SALINA demonstrated an up to 9.5% improvement in average precision and a 10.1% increase in tracking metrics. The energy management module successfully handled extreme weather, preventing outages and reducing contingency costs. These results offer valuable insights for long-term deployment of acoustic data systems in the wild.
References
[1]
Avi Abu and Roee Diamant. 2020. Enhanced Fuzzy-Based Local Information Algorithm for Sonar Image Segmentation. IEEE Transactions on Image Processing 29 (2020), 445--460.
[2]
Keni Bernardin and Rainer Stiefelhagen. 2008. Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. EURASIP Journal on Image and Video Processing 2008 (2008), 1--10.
[3]
Cassandra Bongiovanni, Heather A. Stewart, and Alan J. Jamieson. 2022. High-resolution multibeam sonar bathymetry of the deepest place in each ocean., 108--123 pages.
[4]
John Canny. 1986. A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-8, 6 (1986), 679--698.
[5]
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. 2020. End-to-End Object Detection with Transformers. arXiv:2005.12872 [cs.CV] https://arxiv.org/abs/2005.12872
[6]
Hao Fang, Haoyuan Zhao, Feng Wang, Yi Ching Chou, Long Chen, Jianxin Shi, and Jiangchuan Liu. 2024. Streaming Media over LEO Satellite Networking: A Measurement-Based Analysis and Optimization. ACM Trans. Multimedia Comput. Commun. Appl. (Sept. 2024).
[7]
John Folkesson, John Leonard, Jacques Leederkerken, and Rob Williams. 2007. Feature tracking for underwater navigation using sonar. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. 3678--3684.
[8]
David A. Forsyth and Jean Ponce. 2002. Computer Vision: A Modern Approach. Prentice Hall (2002), 144--145.
[9]
David E. Goldberg. 1989. Genetic Algorithms in Search, Optimization and Machine Learning (1st ed.). Addison-Wesley Longman Publishing Co., Inc., USA.
[10]
Jashwant Raj Gunasekaran, Cyan Subhra Mishra, Prashanth Thinakaran, Bikash Sharma, Mahmut Taylan Kandemir, and Chita R. Das. 2021. Cocktail: A Multidimensional Optimization for Model Serving in Cloud. In Proceedings of the 18th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 123--136. https://www.usenix.org/conference/nsdi21/presentation/gunasekaran
[11]
Kaiming He, Jian Sun, and Xiaoou Tang. 2013. Guided Image Filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 6 (2013), 1397--1409.
[12]
Lianchen Jia, Chao Zhou, Tianchi Huang, Chaoyang Li, and Lifeng Sun. 2023. RDladder: Resolution-Duration Ladder for VBR-encoded Videos via Imitation Learning. In Proceedings of the IEEE Conference on Computer Communications (INFOCOM). 1--10.
[13]
Glenn Jocher, Akash Chaurasia, Jirka Qiu, Robby Stoken, Jakub Borovec, NanoCode012, Yash Fang, Ayush Shah, Lou Changyu, V. Abhiram, Alex Laughing, and David Hogan. 2023. YOLOv8: You Only Look Once Version 8. https://github.com/ultralytics/yolov8 Accessed: 2024-07-01.
[14]
Myounghee Kang. 2011. Semiautomated Analysis of Data from an Imaging Sonar for Fish Counting, Sizing, and Tracking in a Post-Processing Application. Fisheries and aquatic sciences 14 (09 2011).
[15]
Justin Kay, Peter Kulits, Suzanne Stathatos, Siqi Deng, Erik Young, Sara Beery, Grant Van Horn, and Pietro Perona. 2022. The Caltech Fish Counting Dataset: A Benchmark for Multiple-Object Tracking and Counting. arXiv:2207.09295 [cs.CV] https://arxiv.org/abs/2207.09295
[16]
S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. 1983. Optimization by Simulated Annealing. Science 220, 4598 (1983), 671--680.
[17]
Peter Kulits, Angelina Pan, Sara M Beery, Erik Young, Pietro Perona, and Grant Van Horn. 2020. Automated Salmonid Counting in Sonar Data. In NeurIPS 2020 Workshop on Tackling Climate Change with Machine Learning. https://www.climatechange.ai/papers/neurips2020/54
[18]
V7 Labs. 2024. V7 Lab: Training AI to Perform Tasks Faster than Ever. https://www.v7labs.com/ Accessed: 2024-07-01.
[19]
Chongyi Li, Chunle Guo, Wenqi Ren, Runmin Cong, Junhui Hou, Sam Kwong, and Dacheng Tao. 2020. An Underwater Image Enhancement Benchmark Dataset and Beyond. IEEE Transactions on Image Processing 29 (2020), 4376--4389.
[20]
Haotian Liu and Chunyuan Shi. 2023. pLLAVA: Prompted Large Language and Vision Assistant. arXiv preprint arXiv:2304.08485 (2023). https://arxiv.org/abs/2304.08485
[21]
Haotian Liu and Chunyuan Shi. 2023. Visual Instruction Tuning. arXiv preprint arXiv:2304.08485 (2023). https://arxiv.org/abs/2304.08485
[22]
Liang Liu, Hao Lu, Zhiguo Cao, and Yang Xiao. 2018. Counting Fish in Sonar Images. In Proceedings of the IEEE International Conference on Image Processing (ICIP). 3189--3193.
[23]
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C. Berg. 2016. SSD: Single Shot MultiBox Detector. In Proceedings of the European Conference on Computer Vision (ECCV). 21--37.
[24]
Jonathon Luiten, Aljosa Osep, Patrick Dendorfer, Philip HS Torr, Laura Leal-Taixé, and Bastian Leibe. 2021. HOTA: A Higher Order Metric for Evaluating Multi-object Tracking. International Journal of Computer Vision 129, 2 (2021), 548--578.
[25]
Sami Ma, Yi Ching Chou, Haoyuan Zhao, Long Chen, Xiaoqiang Ma, and Jiangchuan Liu. 2023. Network Characteristics of LEO Satellite Constellations: A Starlink-Based Measurement from End Users. In Proceedings of the IEEE Conference on Computer Communications (INFOCOM). 1--10.
[26]
Yuhao Nie, Yuchi Sun, Yuanlei Chen, Rachel Orsini, and Adam Brandt. 2020. PV power output prediction from sky images using convolutional neural network: The comparison of sky-condition-specific sub-models and an end-to-end model. Journal of Renewable and Sustainable Energy 12 (08 2020), 046101.
[27]
NVIDIA. 2024. Jetson Orin Nano Developer Kit. https://developer.nvidia.com/embedded/jetson-orin-nano-developer-kit Accessed: 2024-07-01.
[28]
OpenAI. 2024. Hello GPT-4o. https://openai.com/index/hello-gpt-4o/ Accessed: 2024-07-01.
[29]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning (ICML). https://proceedings.mlr.press/v139/radford21a.html
[30]
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2016. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv:1506.01497 [cs.CV] https://arxiv.org/abs/1506.01497
[31]
Ergys Ristani, Francesco Solera, Roger Zou, Rita Cucchiara, and Carlo Tomasi. 2016. Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking. In European Conference on Computer Vision (ECCV). Springer, 17--35.
[32]
Wei Shen, Zhanfei Peng, and Jin Zhang. 2024. Identification and counting of fish targets using adaptive resolution imaging sonar. Journal of Fish Biology 104, 2 (2024), 422--432.
[33]
Chris Stauffer and W. Eric L. Grimson. 1999. Adaptive background mixture models for real-time tracking. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). 246--252.
[34]
Yannik Steiniger, D. Kraus, and Tobias Meisen. 2022. Survey on deep learning based computer vision for sonar imagery. Engineering Applications of Artificial Intelligence 114 (9 2022), 105157.
[35]
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodríguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. LLaMA: Open and Efficient Foundation Language Models. arXiv:2302.13971 [cs.CL] https://arxiv.org/abs/2302.13971
[36]
Can Wang, Sheng Zhang, Yu Chen, Zhuzhong Qian, Jie Wu, and Mingjun Xiao. 2020. Joint Configuration Adaptation and Bandwidth Allocation for Edge-based Real-time Video Analytics. In Proceedings of the IEEE Conference on Computer Communications (INFOCOM). 257--266.
[37]
Xuezhi Wang, Guanyu Gao, Xiaohu Wu, Yan Lyu, and Weiwei Wu. 2022. Dynamic DNN model selection and inference offloading for video analytics with edge-cloud collaboration. In Proceedings of the ACM Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV '22). 64--70.
[38]
David P. Williams. 2015. Fast Unsupervised Seafloor Characterization in Sonar Imagery Using Lacunarity. IEEE Transactions on Geoscience and Remote Sensing 53, 11 (2015), 6022--6034.
[39]
Xuedou Xiao, Juecheng Zhang, Wei Wang, Jianhua He, and Qian Zhang. 2022. DNN-Driven Compressive Offloading for Edge-Assisted Semantic Video Segmentation. In Proceedings of the IEEE Conference on Computer Communications (INFOCOM). 1888--1897.
[40]
Chi Xu, Xiaoqiang Ma, Ryan Shea, Haiyang Wang, and Jiangchuan Liu. 2018. Enhancing performance and energy efficiency for hybrid workloads in virtualized cloud environment. IEEE Transactions on Cloud Computing 9, 1 (2018), 168--181.
[41]
Minchen Yu, Tingjia Cao, Wei Wang, and Ruichuan Chen. 2023. Following the Data, Not the Function: Rethinking Function Orchestration in Serverless Computing. In Proceedings of the 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI). https://www.usenix.org/conference/nsdi23/presentation/yu
[42]
Xiao Zeng, Biyi Fang, Haichen Shen, and Mi Zhang. 2020. Distream: scaling live video analytics with workload-adaptive distributed edge intelligence. In Proceedings of the ACM Conference on Embedded Networked Sensor Systems (SenSys). 409--421.
[43]
Ben Zhang, Xin Jin, Sylvia Ratnasamy, John Wawrzynek, and Edward A. Lee. 2018. AWStream: adaptive wide-area streaming analytics. In Proceedings of the ACM Conference on Special Interest Group on Data Communication (SIGCOMM). 236--252.
[44]
Haoyuan Zhao, Hao Fang, Feng Wang, and Jiangchuan Liu. 2023. Realtime Multimedia Services over Starlink: A Reality Check. In Proceedings of the ACM Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV '23). 43--49.
[45]
Qianyu Zhou, Xiangtai Li, Lu He, Yibo Yang, Guangliang Cheng, Yunhai Tong, Lizhuang Ma, and Dacheng Tao. 2023. TransVOD: End-to-End Video Object Detection With Spatial-Temporal Transformers. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 6 (2023), 7853--7869.
[46]
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. 2021. Deformable DETR: Deformable Transformers for End-to-End Object Detection. arXiv:2010.04159 [cs.CV] https://arxiv.org/abs/2010.04159
[47]
Z. Zivkovic. 2004. Improved adaptive Gaussian mixture model for background subtraction. In Proceedings of the 17th International Conference on Pattern Recognition (ICPR), Vol. 2. 28--31.
Index Terms
- SALINA: Towards Sustainable Live Sonar Analytics in Wild Ecosystems
Recommendations
Toward a Sustainable Fishery Management Policy: An Artificial Neural Network Model for Predicting Bull Shark (Carcharhinus Leucas) Presence
This article proposes an Artificial Neural Network (ANN) model to predict neonatal and juvenile bull shark habitat usage in the Sabine Pass, located within the Gulf of Mexico between Louisiana and Texas. Given continuing discussion regarding overfishing ...
Design and Development of Sustainable Weed Cutter
ICECCS '14: Proceedings of the 2014 3rd International Conference on Eco-friendly Computing and Communication SystemsAquatic Vegetation serve variety of uses in a water eco system, some of which are vital to the overall health of the aquatic environment and also to add to the aesthetics of its surroundings and to provide a suitable habitat for aquaculture. Excess ...
Remote Sensing of Coastal Ecosystems Using Spectral Indices
ICVISP 2019: Proceedings of the 3rd International Conference on Vision, Image and Signal ProcessingSpectral indices are algorithms performed to improve the signal of certain features, such as vegetation, water and soil in satellite images. The objective of this work was to utilize the normalized difference vegetation index (NDVI), the modified ...
Comments
Information & Contributors
Information
Published In
November 2024
950 pages
ISBN:9798400706974
DOI:10.1145/3666025
- Chair:
- Jie Liu,
- Co-chairs:
- Yuanchao Shu,
- Jiming Chen,
- Program Chair:
- Yuan He,
- Program Co-chair:
- Rui Tan
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
- SIGARCH: ACM Special Interest Group on Computer Architecture
- SIGBED: ACM Special Interest Group on Embedded Systems
- SIGMETRICS: ACM Special Interest Group on Measurement and Evaluation
- SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing
- SIGOPS: ACM Special Interest Group on Operating Systems
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 November 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- NSERC Discovery Grant
- British Columbia Salmon Recovery and Innovation Fund
- MITACS Accelerate Cluster Grant
Conference
SenSys '24
SenSys '24: 22nd ACM Conference on Embedded Networked Sensor Systems
November 4 - 7, 2024
Hangzhou, China
Acceptance Rates
Overall Acceptance Rate 198 of 990 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 199Total Downloads
- Downloads (Last 12 months)199
- Downloads (Last 6 weeks)36
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in