1 Introduction

A crowd is a deformable group of people occupying a particular area. We can obtain crowds information by detecting and calculating the density of people within a specified area, such as the number of people, the trend or speed of the crowd. Automatic crowd understanding has a massive impact on several applications including surveillance and security, situation awareness, crowd management, public space design and etc. For example, the government can decide how to widen existing roads to alleviate traffic congestion based on the location information of high-density populations. In addition, merchants can decide place the advertising labels in the place where can attract the most customers according to the walking track of the crowd on the commercial street. In the real-time environment, if an early warning mechanism is added to the crowd detection system, once a large number of people in a certain area are detected, staff can immediately make evacuation work to avoid injury caused by crowding.

In order to achieve the crowd information automatically, specific equipment and techniques are needed to assist in the counting process. The common methods are cameras, GPS, Wi-Fi and Bluetooth. In general, these methods have the following steps to achieve crowds counting:

  1. 1.

    Obtaining the raw data.

  2. 2.

    Analyzing the relationship between the raw data and the number of people.

  3. 3.

    Displaying the number of people.

Because the equipment and methods used to collect the raw data are different, the above approaches will have different performance in terms of cost, accuracy, and the scope of application. In this paper, we propose a crowd counting and mobility capture system based on Bluetooth. It does not need to deploy devices such as cameras or Radio Frequency Identification (RFID) tag in advance and only needs enough users to carry smartphones with Internet access, which can obtain the crowd information in a certain area. We conducted several experiments in the actual environment to verify the accuracy of the number of people detected by the Bluetooth method, and whether the possibility of walking the trajectory can still be obtained with a small number of users participating.

The rest of the paper is organized as follows. Section 2 describes the related works by other researchers. Section 3 introduces the motivation of developing the system. Section 4 gives details of the system structure and design. Section 5 shows the experiments that we conducted and the evaluation of the results. Section 6 is the conclusions.

2 Related Works

Detecting the crowds to obtain counting number and mobility data is important for today’s urban life. The common method of collecting image data by cameras and then analyzing the number of people by image analysis algorithm has already had mature technologies. But the cost of equipment such as cameras has always been a problem for these methods. Recently, the methods of collecting non-image data to obtain crowd information have gradually increased, especially through Bluetooth.

The original purpose of Bluetooth was to meet the need for small-scale data transmission at short range. So, Bluetooth has features like short communication distance, low speed and low cost. This makes Bluetooth very suitable for some special scenes.

In [1, 2], they installed multiple Bluetooth receiving devices in one room. When a person carrying a Bluetooth device enters the room, each receiver can obtain a Received Signal Strength Indicator (RSSI) value to display the distance to the user. Through the trilateration algorithm [2], the user’s location in the room can be finally obtained. This method can be used as an alternative to GPS localization indoors. Compared to the Wi-Fi localization method, Bluetooth is cheaper and easier to install.

In [3], they proposed to build a Bluetooth Ad Hoc Network, CrowdBlueNet, to collect data from individual smartphones to support crowd management. The network is completely based on Bluetooth devices, so even if the cellular and wireless networks are not accessible, the network will still work, which means that as long as the user is inside the Bluetooth network zone, the crowd information can be obtained. The users can avoid entering areas with high density of people and causing damage.

In [4], they developed a bus query application based on the crowdsourcing model, using Bluetooth as a crowd counting device and location conversion trigger. Let the users’ smartphones provide the number of passengers and traffic conditions instead of the equipment that will be installed on the bus to achieve these proposes. Therefore, reducing the cost of investigating bus operations by using Bluetooth of users’ smartphones.

As the researches we have listed above, most of the application scenarios of Bluetooth are to use the low cost and flexibility of Bluetooth to replace the original high cost and non-movable methods.

3 Motivation

In this paper, we propose a client-server-service-based system that provides users with real-time crowd information such as population density and trajectory in a given area.

Our system is designed based on the following three points.

  1. 1.

    No additional devices are other than users’ smartphones and the server.

  2. 2.

    Reflect the number of people in the area by counting the number of Bluetooth devices.

  3. 3.

    The users are also sensors that provide information about surrounding Bluetooth devices.

Our goal is to achieve a low-cost crowd detection system that is not limited by the location of the equipment installation, which means that the system can detect crowd information in almost any location without the need to install the detection equipment in advance.

4 System Design

4.1 Data Collection Method

According to existing methods for collecting crowd information, we can divide the methods into two groups by different types of raw data: the image data methods and the non-image data methods. For example, using cameras is a typical method which analyzes image data. The raw data obtained by it is images of the video, then utilize human recognition technology to get the crowd information in the image. About the non-image data methods, the raw data of this kind include the sending time, location coordinates, speed, etc. of a device. How the data of the devices are linked to the information of the crowds becomes the key.

Image data methods, such as using closed-circuit television (CCTV) cameras to capture image data, find the crowd area from the image data and convert it into crowd information. Image recognition technology and equipment that can capture high-resolution images make the method highly accurate [5]. But this method relies on the image collecting equipment, so the method is costly. At the same time, the method will cause problems such as occlusion, insufficient light, inflexibility due to the position where the camera is installed. Therefore, it is more suitable for the government to monitor traffic flow and safety, etc.

In the non-image data methods, the GPS-based method has been published by many map service companies [6]. When using the map service applications, the user can get the crowd information from the server, at the same time, his phone as a participating sensor uploads information. GPS chips integrated into the smartphones are cheap now, and this method does not require complicated image information conversion process. However, the GPS signal cannot reach indoors, therefore, it cannot be applied to every place.

Wi-Fi is a recently popular indoor detection method instead of GPS. By analyzing the change of the Wi-Fi signal between multiple access points (APs) or analyzing the Wi-Fi connection data collected by the AP, we can know the flow of people in that area [7]. This method can be used in any place where an AP is deployed. In other words, it is not restricted by indoor and outdoor. However, due to the need to deploy APs, this method is suitable for indoor detection. In the method of analyzing signals, it is also necessary to do a large number of signal samples in advance for reference, so that the method of analyzing signals is relatively complicated.

A Bluetooth method is similar to a device-based method of Wi-Fi. But because the sender and requester of Bluetooth can be converted to each other. There is no need for additional devices like APs, just make sure that the proportion of people carrying Bluetooth devices in the crowd is enough [8]. Although the effective range of Bluetooth is very short, its low cost and wide usage make this method be applied to almost anywhere (Table 1).

Table 1. Comparison of data collection methods
Full size table

We make a comparison of the crowd counting approaches mentioned above. This comparison is not to find out which method is the best but to understand the advantages and disadvantages of each method and the scope of application. Meanwhile, find the most suitable method for our system. Finally, we chose Bluetooth as the method to collect data for our system.

4.2 System Overview

After comparing the above methods, we finally use Bluetooth to counting the number of crowds. According to statistics, in 2018, Bluetooth device shipments have reached 3.9 billion, of which smartphones, tablets and other mobile devices have reached 2.05 billion [9]. This proves that Bluetooth is everywhere in people’s lives. Bluetooth has two main communication protocol technologies, Basic Rate/Enhanced Data Rate (BR/EDR) and Bluetooth Low Energy (BLE). Due to the low power consumption of BLE and the official announcement that 97% of Bluetooth chips will contain BLE mode in 2022, we decided to use BLE mode to scan.

As shown in Fig. 1, the structure of the system is a client-server model. Each smartphone with our app will become a client. They will scan and upload the surrounding Bluetooth devices information while graphically displaying the crowd data obtained by the server to the user. The server receives and analyzes the information from each client, then it returns the data of the crowd in the area where the client is located in real time.

Fig. 1.
figure 1

Structure of the system

Full size image

A user can receive the crowd information by carrying a smartphone with our application installed. The application automatically scans Bluetooth devices and logs location data every 10 s. After scanning, the application sends the data to the server through the Hypertext Transfer Protocol (HTTP) method. The server receives the data from smartphones and stores the information in the database. After analysis, the server sends crowd information back to the smartphones. The smartphones display the crowd information to the users through the graphic interface of our application (Fig. 2).

Fig. 2.
figure 2

Screenshot of the smartphone application

Full size image

4.3 Data Collection and Analysis Method

We define the following roles to make our system easier to understand.

  • The user with a smartphone: After the users installed our application on their smartphones, they can check the crowd information in their area through the application. At the same time, their smartphones become Bluetooth sensors, sending the information of the Bluetooth devices which around them to our server through the network at regular intervals. The rest of this paper will use the smartphone to refer to the user.

  • The Bluetooth device: Bluetooth devices are the devices detected by the smartphones through Bluetooth scanning. It may be a mobile phone, a watch, a headset, etc. Through the counting of Bluetooth devices, we can get the number of the crowds.

  • The server: The server is the computer we set to provide the crowd information for the smartphones. After processing the information sent by the smartphone, the server sends the result back the smartphone, such as the number of people, the movement and location.

When the application on the smartphone starts working, it will perform a 5 s Bluetooth scan, then upload the device information to the server and stop for 5 s before the next scan. In other words, the smartphone sends a message to the server every 10 s. The message includes the timestamp, the location of the smartphone, and the RSSI value and the Media Access Control (MAC) Address of the scanned Bluetooth.

The RSSI value can be converted into the distance between the Bluetooth device and the mobile phone. The MAC Address is a label for connecting to the network. Like the license plate, it is theoretically unique, so we use it to distinguish different Bluetooth devices. If we detect the same Bluetooth device at multiple points on the timeline, we can track the device. Although the location information belongs to the smartphone since the maximum distance set is 15 m, the route of the device still can be tracked or even pre-judged. Furthermore, if the Bluetooth device is detected by more than three or more smartphones at the same time, we can obtain the actual location of the Bluetooth by using the trilateration method [2]. After the analysis, the server sends the result to the smartphones, and our application display the data to the users with the graphical interface.

5 Experiment and Evaluation

5.1 Effective Distance of the Bluetooth Signal

The Bluetooth signal strength is reflected by the RSSI value, which is a negative number. The closer the value is to zero, the closer the distance to the Bluetooth device.

First, we need to confirm the sending range of BLE devices which often be used in our daily lives. We conducted experiments both indoors and outdoors. We set a smartphone and a Bluetooth headset as the sender separately. Both of them support Bluetooth 5.0, which means they can work in BLE mode. We tested the range at 5-meter intervals starting from 0 m up to 30 m. At each range, we measured the RSSI value. Table 2 shows the average result of this experiment.

Table 2. Result of range test
Full size table

According to the results of the above table, we can conclude that although the Bluetooth signal can still be detected at around 30 m, from the RSSI value, it is difficult to distinguish the distance of Bluetooth devices when it is more than 20 m. Therefore, in the following experiments, we set the software acceptable threshold to −75 in an indoor environment and −80 in an outdoor environment. In other words, the farthest Bluetooth device that the mobile phone can detect is 15 m.

5.2 Crowd Counting

In order to verify the feasibility of Bluetooth detection, we conducted the experiments in 4 different scenarios and compared them in pairs. They are the classroom where the professor has a lecture, the crowded metro compartment, the street with the low pedestrian flow and the crowded crossroad. To avoid detecting Bluetooth devices outside the test location, we placed the test point in the middle of the 200 m2 classroom (20 m × 10 m), and we chose the test point in the middle of the compartment, which is 18 m long, at the tail of the subway hich is 18 m long, at the tail of the subway (Fig. 3).

Fig. 3.
figure 3

Scanning result

Full size image

In each experiment, we collected 30 sets of data and took photos at the same time. We manually count the number of people on the photos, then compared with data collected by the smartphone installed our application. About the street and the crossroad, we selected the road of a low-density area and the crossroads with dense crowds in Shimokitazawa. As shown in Table 3, the indoor situation has a higher correlation in the relationship between Bluetooth devices and the number of people. Because almost every student has a notebook or a laptop with a Bluetooth device in that class besides the smartphones. In the other experiments, the relationship between Bluetooth and people is lower, probably because the people being detected is more random. In general, indoor results are more reliable than outdoor results. We consider the reason is that on the one hand, the Bluetooth signal is not easily affected by other factors indoors. On the other hand, indoor flow changes are not as frequent as outdoor.

Table 3. Correlation between bluetooth devices and people
Full size table

The results can show that almost one of two people will use a Bluetooth device. This is helpful for the viability and accuracy of our system.

5.3 Mobility Tracking Implementation

After the feasibility test of Bluetooth scanning, we need to further prove the feasibility of tracking and localization.

According to the principle of the trilateration algorithm [2], to obtain location information of the measured point, we need at least three points with known location information and their respective distances to the measured point. In our system, if three or more mobile phones can detect the same Bluetooth device at the same time, by converting the RSSI value into the distance value, we can obtain the specific location information of the Bluetooth device.

If the same Bluetooth device is continuously detected on a continuous time axis, by analyzing the location information at each time point, we can obtain the route information of the Bluetooth device. It should be noted that our system records the location information of users’ smartphones, not the location information of the detected Bluetooth device. However, due to the RSSI threshold set by us, the detected Bluetooth device will not exceed 15 m to users’ smartphones. If more than three smartphones detect the Bluetooth device at the same time, the actual location of the Bluetooth can also be obtained through trilateration algorithm.

Because the information collected by our system is completed by the participating users’ smartphone, in theory, the more users there are in the same area, the more accurate the crowd information will be, the more Bluetooth devices can be tracked and located. We want to further test which factors affect the accuracy of the data when the number of users is fixed. We control the time, the test area, the user’s walking route overlap rate, and conducted three sets of comparative experiments.

We selected the test area near the train station in Shimokitazawa. We asked 5 participants to walk around the station, each holding a smartphone installed our application. As shown in Table 4, the first two sets were the comparison of route overlapping under the same experiment time. The latter two sets of experiments were the comparison of time under the same area size. The overlapping in Table 4 refers to the percentage of route repetitions of the 5 people (Fig. 4).

Table 4. Environmental parameters of the experiments
Full size table
Fig. 4.
figure 4

Walking route

Full size image

We set the program to record Bluetooth device information every 10 s, which means 6 records per minute. Because there are 5 smartphones, the previous two experiments recorded 450 times respectively, and the third experiment recorded 900 times. Every time a record has information of 0 to multiple Bluetooth devices. We consider Bluetooth devices that have been detected more than 6 times by smartphones as the devices that can be effectively tracked.

From Table 5 we can see that the route overlapping rate will affect the proportion of effective tracking. A smartphone or several smartphones with a low overlapping rate can obtain device information only within its Bluetooth scan range. A single scan area is very limited. However, if the overlapping rate rises, even if the Bluetooth device is outside the scanning range of one smartphone, it is possible to be captured again by other smartphones in the overlapping range, the tracking continues to be effective. On the other hand, through the comparison of the result of Route 2 and Route 3, we found that the increase in measurement time does not necessarily increase the devices ratio of effective tracking. For the time factor, we consider the reason is the crowd have already gone out of our test range. For this situation, even if the detection time is increased within the same range, it does not work.

Table 5. Effective tracking results
Full size table

Tracking not only shows the route of the detected Bluetooth device but also can predict the trend of people flow. With the help of the early warning mechanism, the system can avoid the injury caused by crowds being overcrowded. Tracking also allows us to discover static devices. A static device means the device not carried by a person, such as a desktop computer with Bluetooth. If these devices are counted in the crowd information, the results will be affected. We can remove these static devices by tracking the devices located in a small area for a long time.

The column “Detected only once” in Table 5 is the number of Bluetooth devices that are only detected once by a smartphone. It is difficult to determine if they are mobile devices that can reflect crowd information. Obviously, as the route overlapping rate increases, the number of devices that detected once is reduced.

We analyzed the feasibility of Bluetooth devices localization. From Table 6 we concluded that in the experimental environment we set, the proportion of device that can be located is not high enough. The device localization in our system is based on the number of smartphones that can detect the same Bluetooth device at the same time, which means that the proportion of participants meeting each other needs to be high. In the three sets of experiments above, the route overlapping rate ensures that multiple smartphones can detect the same device, but this does not mean that these smartphones detected the Bluetooth device at the same time. From the column, ‘Times that testers met each other’ of Table 6 we conclude that the five participants are mostly distributed in different locations in the test area. In most cases, only two or fewer smartphones in the detected Bluetooth range are the main reasons for the low proportion of devices that can be located. From the above table, we can know that devices localization is not suitable for the area only has a few smartphones.

Table 6. Localization results
Full size table

6 Conclusion

In this paper, we propose a crowd information system based on Bluetooth scanning. It does not require the addition of equipment such as cameras or routers, so it has the advantage of low cost and flexibility. We verified its accuracy and feasibility through the experiments. Meanwhile, we conducted the experiments on the feasibility of device tracking and localization. It is concluded that the tracking of the device is feasible in the experimental environment we set, but the proportion of devices that can be located is too low.

What we can do is to make every effort to make the application more attractive, to encourage people to use our application, therefore improving the accuracy of the Bluetooth detection population.

As future work, we mainly have two tasks, one is to improve the usability and appeal of mobile software to encourage more users to participate in order to improve the accuracy of our system. The other is to use the tracking and localization function to provide users with crowd information forecasting.