WiDet: Wi-Fi based device-free passive person detection with deep convolutional neural networks

Abstract

To achieve device-free person detection, various types of signal features, such as moving statistics and wavelet representations, have been extracted from the Wi-Fi Received Signal Strength Index (RSSI), whose value fluctuates when human subjects move near the Wi-Fi transceivers. However, these features do not work effectively under different deployments of Wi-Fi transceivers because each transceiver has a unique RSSI fluctuation pattern that depends on its specific wireless channel and hardware characteristics. To address this problem, we present WiDet, a system that uses a deep Convolutional Neural Network (CNN) approach for person detection. The CNN takes the 2-dimensional wavelet coefficients as input, and extracts effective and robust detection features automatically. With a large number of internal parameters, the CNN can record and recognize the different RSSI fluctuation patterns from different transceivers. We further apply the data augmentation method to improve the algorithm robustness to wireless interferences and pedestrian speed changes. To take advantage of the wide availability of the existing Wi-Fi devices, we design a collaborative sensing technique that can recognize the subject’s moving directions. To validate the proposed design, we implement a prototype system that consists of three Wi-Fi packet transmitters and one receiver on low-cost off-the-shelf embedded development boards. In a multi-day experiment with a total of 163 walking events, WiDet achieves 95.5% of detection accuracy in detecting pedestrians, which outperforms the moving statistics and the wavelet representation based approaches by 22% and 8%, respectively.

Introduction

Person detection is crucial for many sensitive applications, such as access control, traffic monitoring, and personal identification. Traditional person detection systems rely on cameras or infrared sensors. However, the performance of camera-based systems depend on the lighting conditions and video recordings can raise privacy concerns, and the infrared sensors are affected by the target’s clothing [1] and the ambient temperature [2]. Furthermore, both systems require direct line-of-sight (LoS) with the target, and require dedicated devices. To overcome these limitations, Wi-Fi based person detection systems have been designed [3], [4], [5]. These systems rely on the phenomenon that the Wi-Fi signal strengths fluctuate when a person moves. Since the Wi-Fi based systems can achieve through-the-wall motion detection, and they rely on the widely available Wi-Fi devices in the indoor environments, they are an emerging effective and low-cost technology for person detection.

Mainstream Wi-Fi based device-free person detection systems rely on either the Channel State Information (CSI) or the Received Signal Strength Index (RSSI) measurements. However, currently only a small number of Wi-Fi adapter models support access to CSI, which limits its ability for wide adoption. RSSI measurements are easily accessible in most Wi-Fi devices, but it has been challenging to extract effective and robust features for RSSI because different Wi-Fi adapters generate different RSSI fluctuation patterns due to their unique wireless channel and hardware characteristics. Specifically, the Wi-Fi signal experiences various types of degradations, including path loss, shadowing effect and multi-path effect, which are dependent on the Wi-Fi transceiver deployment location and the wireless channel of the environment. The RSSI measurements on Wi-Fi transceivers are hardware dependent and discrepancies exist even for devices from the same vendors [6]. Many existing person detection techniques rely on hand-crafted signal features, such as wavelet representation [4], [5] and moving statistics [3] based features. However, the detection performance of these features degrades when multiple different Wi-Fi links are used. It will be work intensive to design specialized detection features for each Wi-Fi link.

We apply the machine learning technique to address the challenge of effective feature extraction. Recently, deep learning techniques, Convolutional Network Networks (CNNs) in particular, have achieved remarkable success in recognizing 2-dimensional, image-like data [7]. Using the continuous wavelet transform [8], we convert the 1-dimensional RSSI signal into a 2-dimensional time–frequency domain representation consisting of wavelet coefficients. The advantage of the CNN is that it can learn detection features automatically from data samples. With a large number of internal parameters, the CNN can record and recognize the unique RSSI fluctuation patterns for all the different Wi-Fi transceivers. We designed a CNN architecture that consists of multiple convolutional layers, with each layer consisting of learnable filters that can detect unique signal patterns with different scales. The parameters in the CNN are fine-tuned during the training phase using the back propagation algorithm. The outputs of the stacked convolutional layers are treated as features and are fed into a fully connected layer that conducts classification.

To adequately train a CNN model, a sufficient training data set that includes all the common data variations is needed. In the scenario of device-free person detection, one of the most common type of variations is the change of subject walking speeds. When a person walks at different rates, the durations of the RSSI fluctuations change accordingly. It will be time consuming to collect training data of all walking speeds. Instead, we apply the data augmentation technique to expand the size of the training data set [9]. The basic idea is to warp the data samples with different ratios to mimic data changes caused by the variations of walking speeds. Another type of common variations is the wireless noise that causes the RSSI to vary dramatically. It is known that many Wi-Fi connections are bursty: they shift between poor and good connection quality [10]. To reduce the impact on person detection accuracy, we also generate additional training data by adding random noises that resemble such connection changes. Utilizing these two types of augmented data, we improve the generability of the algorithm without increasing the data collection effort.

The wide deployment of Wi-Fi devices in the indoor environments provides us with an opportunity to monitor a walking directions with multiple sender–receiver pairs. However, the RSSI data collected by different transceiver pairs have different fluctuation patterns, magnitudes and durations. We design a Dynamic Time Warping (DTW) based algorithm that can cope with the cross-device data heterogeneity, and can determine the walking directions of the subjects. To facilitate system deployment, a low-cost, convenient implementation is needed. We build our system on Raspberry Pi development boards, with small USB Wi-Fi adapters attached. We use the open source libraries Aircrack to control the transmission and reception of the customized Wi-Fi packets. We deploy our system in our department building and conduct extensive testing. In a multi-day experiment with 163 walking instances, our deep convolutional neural network-based approach is able to achieve 95.5% of detection accuracy. Our contributions are summarized as follows:

• We proposed a Wi-Fi based device-free person detection system that uses a deep Convolutional Neural Network (CNN) architecture. The CNN can automatically extract effective features from the wavelet coefficients of the Wi-Fi RSSI measurements to conduct person detection.

• We use the Continuous Wavelet Transform to analyze the Wi-Fi RSSI data, and obtain a time–frequency representation of the raw signal. The wavelet transform enhances the signal patterns caused by human motions and suppresses the random noises. The time–frequency representation of the RSSI signal enables more effective feature extraction for the CNN algorithm.

• To improve the system robustness to the wireless signal noises and the changes of subject moving speeds, we applied the data augmentation techniques that generates additional data to better train the CNN classifier. To take advantage of the ubiquitous deployment of Wi-Fi devices in the indoor environment, we designed a collaborative sensing method to determine the walking directions of the subjects using RSSI data collected from multiple transceiver pairs.

• We implemented a prototype system with three transmitters and one receiver on low-cost embedded platforms. In a multi-day experiment with 163 walking events, WiDet achieved 95.5% of detection accuracy, outperforming the moving statistics and the wavelet representation based approaches by 23% and 9%, respectively.