Poster Abstract: Tasking Heterogeneous Sensor Systems with LLMs
Authors: 
Kaiwei Liu, Bufang Yang, Lilin Xu, Yunqi Guo, Neiwen Ling, Zhihe Zhao, Guoliang Xing, Xian Shuai, Xiaozhe Ren, Xin Jiang, 
Zhenyu YanAuthors Info & Claims
Kaiwei Liu, Bufang Yang, Lilin Xu, Yunqi Guo, Neiwen Ling, Zhihe Zhao, Guoliang Xing, Xian Shuai, Xiaozhe Ren, Xin Jiang, Zhenyu YanAuthors Info & ClaimsPages 901 - 902
Abstract
Despite the extensive use of sensors enabling intelligent applications, the complementary potential of co-existing sensor systems is often not fully utilized, limiting more advanced applications. This paper introduces a novel solution using Large Language Models (LLMs) to coordinate sensor systems for handling complex user queries. It defines a sensor language for sensor systems, including vocabulary set and grammar rules, analogous to natural language components, enabling LLMs to translate user intentions into sensor coordination plans. Preliminary results show that our approach significantly outperforms the existing solution at plan generation, execution and response generation stages.
References
[1]
Zhaoyang Liu, Zeqiang Lai, Zhangwei Gao, Erfei Cui, Zhiheng Li, Xizhou Zhu, Lewei Lu, Qifeng Chen, Yu Qiao, Jifeng Dai, et al. 2023. Controlllm: Augment language models with tools by searching on graphs. arXiv preprint arXiv:2310.17796 (2023).
[2]
Xiaomin Ouyang, Xian Shuai, Yang Li, Li Pan, Xifan Zhang, Heming Fu, Sitong Cheng, Xinyan Wang, Shihua Cao, Jiang Xin, et al. 2024. ADMarker: A Multi-Modal Federated Learning System for Monitoring Digital Biomarkers of Alzheimer's Disease. In MobiCom.
[3]
Dhruv Shah, Błażej Osiński, Sergey Levine, et al. 2023. Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action. In CoRL.
[4]
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2024. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. NeurIPS 36 (2024).
[5]
Amit Kumar Sikder, Leonardo Babun, Z Berkay Celik, Hidayet Aksu, Patrick McDaniel, Engin Kirda, and A Selcuk Uluagac. 2022. Who's controlling my device? Multi-user multi-device-aware access control system for shared smart home environment. ACM TIOT 3, 4 (2022).
[6]
Huatao Xu, Liying Han, Qirui Yang, Mo Li, and Mani Srivastava. 2024. Penetrative ai: Making llms comprehend the physical world. In HotMobile.
[7]
Bufang Yang, Lixing He, Neiwen Ling, Zhenyu Yan, Guoliang Xing, Xian Shuai, Xiaozhe Ren, and Xin Jiang. 2023. Edgefm: Leveraging foundation model for open-set learning on the edge. In Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems. 111--124.
[8]
Bufang Yang, Siyang Jiang, Lilin Xu, Kaiwei Liu, Hai Li, Guoliang Xing, Hongkai Chen, Xiaofan Jiang, and Zhenyu Yan. 2024. DrHouse: An LLM-empowered Diagnostic Reasoning System through Harnessing Outcomes from Sensor Data and Expert Knowledge. arXiv preprint arXiv:2405.12541 (2024).
Index Terms
- Poster Abstract: Tasking Heterogeneous Sensor Systems with LLMs
Recommendations
Poster abstract: ASWP: a long-term WSN deployment for environmental monitoring
IPSN '13: Proceedings of the 12th international conference on Information processing in sensor networksASWP testbed, a long-term wireless sensor network (WSN) deployment for environmental monitoring, is presented. This testbed integrates 42 MICAz nodes that implement a periodic sampling application for external sensors exposed to a forested nature ...
Poster abstract: connected wireless camera network deployment with visibility coverage
IPSN '13: Proceedings of the 12th international conference on Information processing in sensor networksFirst responder applications often require safety surveillance using wireless camera networks~\cite{breadcrums}. To ensure visual sensing coverage, it is crucial to place optical sensor nodes at proper locations. Under the scenario of energy constrained ...
Area coverage of heterogeneous wireless sensor networks in support of Internet of Things demands
As the Internet of Things (IoT) evolves, more and more Wireless Sensor Networks (WSNs) are being deployed in the real world. Connected vehicles, smart grids, smart cities, smart healthcare, networks of robots, and disaster recovery networks are some ...
Comments
Information & Contributors
Information
Published In
November 2024
950 pages
ISBN:9798400706974
DOI:10.1145/3666025
- Chair:
- Jie Liu,
- Co-chairs:
- Yuanchao Shu,
- Jiming Chen,
- Program Chair:
- Yuan He,
- Program Co-chair:
- Rui Tan
Copyright © 2024 Copyright held by the owner/author(s).
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
Sponsors
- SIGARCH: ACM Special Interest Group on Computer Architecture
- SIGBED: ACM Special Interest Group on Embedded Systems
- SIGMETRICS: ACM Special Interest Group on Measurement and Evaluation
- SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing
- SIGOPS: ACM Special Interest Group on Operating Systems
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 November 2024
Check for updates
Author Tags
Qualifiers
- Poster
Funding Sources
- National Natural Science Foundation of China
- Research Grants Council of Hong Kong
Conference
SenSys '24
SenSys '24: 22nd ACM Conference on Embedded Networked Sensor Systems
November 4 - 7, 2024
Hangzhou, China
Acceptance Rates
Overall Acceptance Rate 198 of 990 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 189Total Downloads
- Downloads (Last 12 months)189
- Downloads (Last 6 weeks)44
Reflects downloads up to 25 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in