1 Introduction

Driven by the ongoing digitization and automation in the industrial sector, we are currently experiencing a significant increase in the complexity of production plants and manufacturing processes. This growing complexity requires the development of intelligent user interfaces in order to support workers in the regulation and execution of production processes [1]. The term intelligent hereby describes the ability of a user interface to extensively adapt to a usage context, i.e. to a specific user, task and available tools. The development of intelligent user interfaces requires detailed information about the environment and the location of existing dynamic entities. Therefore, a central requirement for the realization of such intelligent user interfaces will be an effective and robust indoor localization of dynamic entities such as machines, vehicles, tools, material boxes and workers. This kind of information can be used to define environmental situations and to regulate production processes. The localization of dynamic entities is already working on a coarsely level today, i.e. based on light barriers or radio-frequency identification (RFID) readers.

However, for the development of future intelligent user interfaces, which aim to support people in the industrial context, more detailed contextual information is necessary than currently available. For the ideal assistance of users, better sensors and a comprehensive data base are needed to exploit the potential of intelligent user interfaces.

While there are a lot of technical systems are available, which offer a large number of different entities to be localized, the localization of people in the working environment poses numerous ethical, legal and social questions [2,3,4]. The localization and identification of users in the environment of such systems is often avoided in order not to conflict with the aforementioned aspects. In many cases, however, this extensive localization would be desirable to provide user interfaces that have some intelligence. For example, an automatic rescheduling of processes could occur as soon as the position of certain employees indicates that a process is not working according to plan. Based on this rescheduling, in the future, assistance systems could automatically order employees to different locations in plants and provide support for the elimination of process deviations. Before these future scenarios can be realized, however, privacy-preserving localization concepts have to be developed which meet both the technical requirements and the interests of employees [5].

In this paper, we take a first step towards this development: we propose a concept for a multi-level localization system that limits the accuracy of the localization to a necessary level depending on the location of a user. To demonstrate the feasibility of our concept, we describe a first implementation that has been carried out and tested in the SmartFactoryOWL [6].

The remainder of the paper is organized as follows: in part two, we will give an overview on privacy aspects and indoor localization technologies. In part three we present our concept for a localization system to locate workers in industrial surroundings in a reasonable manner, while preserving their privacy as best as possible. In part four we discuss the details of the implementation of a first prototype. Finally, we give a summary and an outlook on future work in part five.

2 Related Work

2.1 Workplace Privacy

The continuous digitization and the consequent integration of new technologies in industrial environments allows companies to perform a comprehensive collection and recording of production-related and employee-related data. From an employer’s perspective, this provides significant benefits for optimizing productivity, enhancing security, and safeguarding the interests of a company [7]. But, from an employee’s point of view, this kind of data collection is very likely to be seen as a form of monitoring which opens up various possibilities for performance measurements and other evaluations [8, 9]. These circumstances result in a general conflict with respect to the privacy of employees at the workplace, which has already been investigated and discussed in numerous publications both from legal [8,9,10], ethical [11, 12] and technological perspectives [7, 13]. In this context, privacy can be seen as a sphere of freedom and anonymity in which an individual can move and act freely without having to justify his activities to others.

From a legal perspective, the claim to privacy in the workplace is enshrined in different depths in the legislation of different countries. Thus, the case law in European countries contains a much higher and more clearly defined claim to privacy than, for example, in the US [7]. However, the question arises whether the relevant case-law needs to be adapted to modern circumstances, such as the increasing digitization in industrial environments [8].

In the future, a certain trade-off will be necessary in order to meet both the interests of the employees in terms of privacy at the workplace as well as the requirements of employers for an effectively usable localization of workers in the industrial environment.

2.2 Localization in Industrial Environments

Over the years, numerous localization technologies have been developed with focus on the localization of dynamic entities in industrial environments. Besides the localization of vehicles, boxes, tools and other materials, most of these systems also technically permit the detection of persons in industrial environments [14].

Many different localization technologies have been developed and evaluated and existing localization technologies for the detection and tracking of dynamic entities in the industrial environment have been described and discussed in various publications [15,16,17,18]. According to [16], localization technologies used in the industrial sector can be classified into one of three basic categories: wireless-communication localization technologies or wave propagation localization technologies [14], dead reckoning localization technologies or even motion sensing localization technologies [19] and scene analysis localization technologies [20]:

The group of wireless-communication localization technologies, also known as wave propagation localization technologies, include systems based on different radio technologies such as Wi-Fi, ultra-wideband (UWB) or Bluetooth, as well as infrared technologies and ultrasound technologies. These systems use the characteristics of wave propagation, e.g. the phase or angle of a signal to determine the distance between transmitters and receivers. The systems differ by using active and passive tags as well as by the number of available sensors to determine the position.

The group of dead reckoning localization technologies, or motion sensing localization technologies, on the other hand, include localization systems based on inertial measurement units (IMU), which are usually integrated in mobile devices. These measuring units use the data of various motion sensors such as acceleration sensors and gyroscopes as well as digital compass sensors to determine a localization based on the detected movements.

Finally, the group of localization technologies based on scene analysis methods includes systems that capture the characteristics of an environment via video streams or electromagnetic sensors. Localization is done by performing pattern recognition methods based on comparative data. Even though especially video camera-based systems can offer a wide range of applications for indoor localization, they potentially give employees the impression of permanent supervision by the employer.

2.3 Privacy-Aware Localization in Industrial Environments

From a technical perspective, most of the existing localization technologies for indoor localization in industrial environments are usable for the tracking of employees. While from a technological perspective a high precision of a localization system is desirable, the design of a localization system might be influenced by requirements counting against this high accuracy. Localization of people within an industrial environment is a sensitive topic in terms of privacy and the possibilities of locating employees creates a huge conflict of interests: on the one hand, the information about the position of employees inside an industrial area can be used to optimize production processes, so from a perspective of work organization a high localization accuracy is required. On the other hand, the system has to take into account the need of the employees in terms of not being observed (or surveilled). A user study presented in [21] shows that privacy is a huge concern, when designing localization systems. Users of localization system stated that they “wished to have complete control over the visibility of their location” [21]. That implies from a human-centered design perspective that localization systems have to be designed in a way to communicate to its’ users the current state of observation possibilities.

Given the mentioned conflict and design recommendations from previous user studies, it is remarkable, that only few publications take into account, how to design localization systems for privacy-awareness.

The current state-of-art for the localization of persons in the industrial environment are systems based on tags or identity cards using radio-frequency identification (RFID) technology. These tags or cards have to be actively swiped by user at a particular reader unit to register the user at a certain location. A similar, but technically different, approach is the use of barcodes, that have to be scanned by a user to indicate a location [22]. The mentioned technologies are particularly suitable for the localization of employees in order to record the presence or absence in a production area without determining the exact position. Since they require active gestures, users are possibly aware of being registered at a certain location.

One location system explicitly designed for privacy was the Cricket location-support system, a mobile system based on radio and ultrasonic beacons [23]. The system does not have a central management. Instead, mobile devices determine their own location and users can control, if this information should be shared to other instances. The decentral nature of the system could be used for industrial localization systems as well; however, it requires explicit interactions by the user to share a particular location in a similar way to the use of RFID smartcards or tags.

3 Conceptual Approach

Based on the previously described context, a concept for a privacy-preserving localization of workers in industrial environments should consider at least the following aspects: The accuracy of localization (1), the access to localization data (2), the location of data processing (3), duration for storing localization information (4) and the privacy of workers (5).

In order to limit the accuracy of a localization, our concept for a privacy-aware localization of workers in industrial environments follows a multigranular approach. This means that the accuracy of the localization is technically reduced to a necessary level and adapted to the specific location of a worker. Such an adjustment could theoretically be realized by a software-side limitation of a localization system with high accuracy. With regard to the interests of the employees, however, we propose a hardware-side limitation by implementing multiple levels of localization based on different technologies (multi-level design). While the specific number of localization levels is directly related to the individual usage context, we suggest a multi-level localization system to include at least three levels to cover the relevant requirements for the development of intelligent user interfaces (Fig. 1):

Fig. 1.
figure 1

Example for the different levels of the multi-level localization system.

Full size image

  • Level 1: The first level of localization is intended to capture the presence or absence of employees in a production area without tracking the movements within the environment. This localization can be implemented in the form of an identification process, which is carried out when entering or leaving the specific area.

  • Level 2: The second level of localization is intended to track the movement of workers across large-scale environments in order to detect their presence in specific areas such as the immediate environment of an automated production plant.

  • Level 3: The third level of localization is intended to capture the exact position and the viewing direction of a person in a spatially limited area. The captured data can be used to customize user interfaces in the viewing direction of a worker in order to provide him/her with relevant information.

Regarding the access to the localization information, we propose the communication to be initialized and controlled by the localization systems. In this way, any external access to the raw data will be prevented. Furthermore, we suggest the first stages of data processing to be implemented on side of the localization systems to limit the information density. Additionally, the data retention period should be kept relatively short in order to prevent automatic adaptions of production systems based on outdated data.

4 Prototypical Implementation of a Multi-level-Localization System

In order to evaluate our concept, we implemented a prototypical multi-level localization system based on three different localization technologies: An RFID reader (first level), an UWB real time localization system (second level) and an optical system based on a depth cameras (third level). The system is installed for evaluation purposes inside the SmartFactoryOWL, a demonstration factory for industrial automation and digitization in Lemgo, Germany [6].

4.1 First Level of Localization

For the first level of localization, we used a system based on a RFID reader and personalized tags or identity cards to detect the presence or absence of workers in the production area. The system was implemented based on a Raspberry Pi (RPi) and a RFID breakout board (MFRC522), which is connected to the RPi via the general-purpose input/output (GPIO) pins. The RFID reader is located at the entry to the manufacturing area. The system was integrated into the local area network of the SmartFactoryOWL to communicate with a central logistics system (see Subsect. 4.4) (Fig. 2).

Fig. 2.
figure 2

The prototypical RFID reader (left) and the associated tags and identity cards (right).

Full size image

4.2 Second Level of Localization

For the second level of localization, we used an Ubisense UWB real-time localization system which is installed inside the SmartFactoryOWL. It consists of eight sensors, which are mounted under the roof of the production area (Fig. 3 left), and multiple active tags (Fig. 3 right). The data from the eight sensors are transferred via one specific root sensor to a Linux-based webserver, where the location messages are decrypted and stored within an SQL database. This database is queried by an RPi (we reuse the RPi from the first level of localization) and the latest data is retrieved and provided as a data stream to the central logistics system (see Subsect. 4.4).

Fig. 3.
figure 3

A sensor of the UWB-system (left) and the associated tags (right).

Full size image

4.3 Third Level of Localization

According to our concept, the third level of localization is used to capture the position and the viewing direction of a user in a spatial limited area. The system is implemented using a Microsoft Kinect V2 depth-camera and a face-tracking algorithm provided by the Microsoft Face Basics API. The camera is mounted on top of a display at one of the demonstrators, which shows the current state of the factory. Depending on the current user/users (e.g. shop floor worker, management) information is displayed. The face-tracking can be used to enable an interaction with the data visualizations, e.g. accordion panels can be expanded by looking at them.

In order to identify the persons to be captured in the viewing area of the camera and to distinguish them from other persons not to be captured by the overall system, it was necessary to transfer the collected position data into the coordinate system of the UWB system used for the second level of localization system in order to make them comparable. For this purpose, the exact position of the camera in the coordinate system of the UWB system was determined as part of a test measurement by means of a UWB tag and offset with the position data. On this basis, a comparison of the positions of the UWB tags with the positions of the captured faces can achieve a unique assignment (Fig. 4).

Fig. 4.
figure 4

Area of the third level localization in the SmartFactoryOWL (left). The localization is done by using a Kinect depth camera system on top of a display (right).

Full size image

4.4 Integration and Central Logistic System (ISIPlus®)

In order to create a realistic industrial scenario, we used a commercial logistic software tool (ISIPlus®) from ISI-Automation GmbH & Co. KGFootnote 1 that aggregates all localization data. The logistic system collects and visualizes the positioning data from the different localization levels. The communication between the localization systems and the ISIPlus® system was implemented via individual TCP socket connections (see Fig. 5). In order to restrict any external access to the localization information, the communication channels were initialized by the localization systems. We used a general data structure in order to handle the incoming data.

Fig. 5.
figure 5

Overview of the communication of the overall system.

Full size image

Figure 6 shows the “localization manager” – a prototypical user interface with a table-based and a graphical-based visualization of the present users detected by the system, their positions and their current localization level. Using this interface, people can be located with a necessary accuracy. Internally, the data can be used in the ISIPlus® system to optimize logistic processes or production planning.

Fig. 6.
figure 6

Prototypical user interface (localization manager) of the ISIPlus® system at the demonstrator in the SmartFactoryOWL.

Full size image

5 Summary and Outlook

In this paper, we presented our concept and a first prototype implementation for a multi-level localization system for intelligent user interfaces. We showed the feasibility of such a system that restricts the localization accuracy according to the position of a user in order to ensure workplace privacy on one hand and localization opportunities for the implementation of intelligent user interfaces on the other hand.

In addition to the already existing user interface, we will implement several other potential applications in a prototypical way in a next step and evaluate the overall system within user studies to get insight about its usability and perceived user experience. In addition, the system is to be integrated in the future as a demonstrator in the leadership of the SmartFactoryOWL, through which experts from research and industry, as well as other interested persons on the subject of industrial digitization and automation can inform. On the basis of a long-term written survey of this specialist audience, further findings are to be collected for further optimization of the concept.