Shivenkumar Parmar
ABSTRACT
With the fast development of the Internet of Things (IoT), smart speakers for voice assistance have become increasingly important in smart homes, which offers a new type of human-machine interaction interface. Voice localization with microphone arrays can improve smart speaker's performance and enable many new IoT applications. To address the challenges of complex indoor environments, such as non-line-of-sight (NLOS) and multi-path propagation, we propose voice fingerprinting for indoor localization using a single microphone array. The proposed system consists of a ReSpeaker 6-mic circular array kit connected to a Raspberry Pi and a deep learning model, and operates in offline training and online test stages. In the offline stage, the models are trained with spectrogram images obtained from audio data using short-time Fourier transform (STFT). Transfer learning is used to speed up the training process. In the online stage, a top-K probabilistic method is used for location estimation. Our experimental results demonstrate that the Inception-ResNet-v2 model can achieve a satisfactory localization performance with small location errors in two typical home environments.
- M. Wang, W. Sun, and L. Qiu, "MAVL: Multiresolution analysis of voice localization," in Proc. USENIX NSDI'21, Virtual Conference, Apr. 2021, pp. 845--858.Google Scholar
- W. Wang, J. Li, Y. He, and Y. Liu, "Symphony: Localizing multiple acoustic sources with a single microphone array," in Proc. ACM SenSys'20, Virtual Conference, Nov. 2020, pp. 82--94.Google ScholarDigital Library
- M. E. Epstein and L. Vasserman, "Generating language models," US Patent 9,437,189, Sept. 2016.Google Scholar
- Q. Lin, Z. An, and L. Yang, "Rebooting ultrasonic positioning systems for ultrasound-incapable smart devices," in Proc. ACM MobiCom'19, Los Cabos, Mexico, Oct. 2019, pp. 1--16.Google Scholar
- T. C. Collier, A. N. Kirschel, and C. E. Taylor, "Acoustic localization of antbirds in a Mexican rainforest using a wireless sensor network," J. Acoustical Soc. America, vol. 128, no. 1, pp. 182--189, July 2010.Google ScholarCross Ref
- S. Shen, D. Chen, Y.-L. Wei, Z. Yang, and R. R. Choudhury, "Voice localization using nearby wall reflections," in Proc. ACM MobiCom'20, London, UK, Sept. 2020, pp. 1--14.Google Scholar
- J. Purohit, X. Wang, S. Mao, X. Sun, and C. Yang, "Fingerprinting-based indoor and outdoor localization with LoRa and deep learning," in Proc. IEEE GLOBECOM'20, Taipei, Taiwan, Dec. 2020, pp. 1--6.Google Scholar
- X. Wang, L. Gao, S. Mao, and S. Pandey, "CSI-based fingerprinting for indoor localization: A deep learning approach," IEEE Trans. Veh. Technol., vol. 66, no. 1, pp. 763--776, Jan. 2017.Google Scholar
- X. Wang, L. Gao, and S. Mao, "BiLoc: Bi-modality deep learning for indoor localization with 5GHz commodity Wi-Fi," IEEE Access J., vol. 5, no. 1, pp. 4209--4220, Mar. 2017.Google ScholarCross Ref
- X. Wang, X. Wang, and S. Mao, "Deep convolutional neural networks for indoor localization with CSI images," IEEE Trans. Netw. Sci. Eng., vol. 7, no. 1, pp. 316--327, Jan./Mar. 2020.Google ScholarCross Ref
- C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," in Proc. IEEE CVPR'16, Las Vegas, NV, June-July 2016, pp. 2818--2826.Google Scholar
- K. He, X. Zhang, S. Ren, and J. Sun, "Identity mappings in deep residual networks," in Proc. 2016 European Conference on Computer Vision, Amsterdam, The Netherlands, Oct. 2016, pp. 630--645.Google ScholarCross Ref
- C. Szegedy, S. Ioffe, V. Vanhoucke, and A. Alemi, "Inception-v4, inception-resnet and the impact of residual connections on learning," in Proc. AAAI'17, San Francisco, CA, Feb. 2017, pp. 4278--4284.Google Scholar
- Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278--2324, Nov. 1998.Google ScholarCross Ref
- K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proc. IEEE CVPR'16, Las Vegas, NV, June-July 2016, pp. 770--778.Google Scholar
- X. Wang, X. Wang, and S. Mao, "Indoor fingerprinting with bimodal CSI tensors: A deep residual sharing learning approach," IEEE Internet of Things Journal, vol. 8, no. 6, pp. 4498--4513, Mar. 2021.Google ScholarCross Ref
- M. Youssef and A. Agrawala, "The Horus WLAN location determination system," in Proc. ACM MobiSys'05, Seattle, WA, June 2005, pp. 205--218.Google ScholarDigital Library
- X. Wang, Z. Yu, and S. Mao, "Indoor localization using magnetic and light sensors with smartphones: A deep LS™ approach," Springer Mobile Networks and Applications (MONET) J., vol. 25, no. 2, pp. 819--832, Apr. 2020.Google ScholarDigital Library
- Seed Wiki, "ReSpeaker 6-Mic circular array kit for Raspberry Pi," Jan. 2019. [Online]. Available: https://wiki.seeedstudio.com/BIBentrySTDinterwordspacingGoogle Scholar
Index Terms
- Voice Fingerprinting for Indoor Localization with a Single Microphone Array and Deep Learning
Recommendations
Speaker Tracking and Identifying Based on Indoor Localization System and Microphone Array
AINAW '07: Proceedings of the 21st International Conference on Advanced Information Networking and Applications Workshops - Volume 02This paper presents a novel multimodal system to track the participants and identify the active speaker in the smart meeting room. Indoor localization system, Cicada, is used to obtain the location and identity information of the participants. Cicada, ...
Indoor human localization with orientation using WiFi fingerprinting
ICUIMC '14: Proceedings of the 8th International Conference on Ubiquitous Information Management and CommunicationLocalization in indoor environment poses a fundamental challenge in ubiquitous computing compared to its well-established GPS-based outdoor environment counterpart. This study investigated the feasibility of a WiFi-based indoor positioning system to ...
Deep belief networks for fingerprinting indoor localization using ultrawideband technology
With the increasing requirement of localization services in indoor environment, indoor localization techniques have drawn a lot of attention. In recent years, fingerprinting localization techniques have been proved to be effective in indoor localization ...
Comments