Metadata based need-to-know view in large-scale video surveillance systems

Abstract

Large-scale video surveillance systems are increasingly seen as the answer to problems concerning public safety, law enforcement, and situational awareness in public places. However, the unauthorized use of personal information derived from video data can be harmful. To preserve privacy, it is important to understand what type of personal information is contained in video surveillance data, and how much of that information is essential for an observer to achieve her authorized purpose. The purpose of the observer is described in terms similar to the information extracted by video surveillance systems, so they can be compared. This helps identify what type of information is better suited to control the flow of information to multiple observers, without compromising the privacy of the individuals. This paper presents a privacy-aware and need-to-know access control framework built on fine-grained data properties, extracted from surveillance data, which must conform to the explicitly defined purpose of the observers.

Introduction

Large-scale video surveillance systems (VSS) deploy a large number of video cameras at many public places to observe different events and objects of interest. Video cameras are excellent multi-sensors so different observers (an observer is either a monitoring human, a program, or a system) may use the same video feed to accomplish different tasks such as smart police patrolling, public safety management, traffic operations management, infrastructure maintenance, or in case of recent pandemic social-distance inspection, etc. (Baran et al., 2016). Most of the above-mentioned services require continuous monitoring of public places because the information that they are interested in is not pre-defined. No one usually knows the exact form that an unwanted incident may take or when a particular event will occur, for instance, a traffic-monitoring observer (TM) does not know when and where a vehicle will be speeding, or a patrolling officer (PO) is generally unaware of where and when two individuals may start a Fight. To record a particular activity when it happens, and then use that specific piece of data according to the particular requirements of the observer, data needs to be collected and analyzed continuously.

This huge amount of VSS data congregated at a mass-scale from various public places contains a lot of information about general individuals, their routine activities, and associations, which may be inferred as personal. Therefore, although VSS provides many benefits it is also very invasive as it continuously records citizens’ activities, which may pose a threat to their freedom and privacy if misused by observers (Van den Hoven et al., 2019).

Countries across the globe have introduced legislation for the installation and operation of VSS to preserve their citizens’ privacy, whenever a VSS collects or processes any personal data (Rajpoot and Jensen, 2015). Legislations like the General Data Protection Regulations (GDPR) and California Consumer Privacy Act (Garlie, 2020) require VSS owners (either public or private) to have a valid legal basis for its deployment. It also requires owners to state an explicit purpose for their data usage, and confirmation that video data will not be subject to secondary use, i.e., it will only be used for the consented primary purpose. Most data-protection legislation allows informed individual consent as a legal basis for recording personal data, which reduces legal uncertainty. However, due to the continuous presence of VSS in the public space, it is generally not possible to obtain consent from every individual every time a camera records them. Therefore, VSS deployment often refers to “public interest” as a legal basis, as different public administrative services and authorities use the data (broadly) for multiple purposes like public safety, traffic management, etc. Individuals do not have a right to erasure and data portability under this legal base, but they do retain a right to object in some cases. Hence, VSS data collected under “public interest” supports different sorts of purposes that are beneficial for citizens, but they have fewer rights and are often expected to trust observers with their data. Nonetheless, at times, observers with individuals’ information have misused it by exploiting ambiguous purposes and indefinite legal basis, causing a high number of privacy invasion incidents of voyeurism, blackmail, profiling, etc. (Campbell, 2019; Snowden, 2019; Froomkin, 2015).

To preserve citizens’ privacy, it is imperative to limit observers’ views only to the relevant parts of VSS data that is helpful for their job. There are two likely approaches to achieve this, (1) limit the time and location of cameras that the observer has complete access to and (2) limit what can be seen in the individual frames in the (content of the) video stream. In the first case, observers are restricted based on the VSS physical context of time and space (Rajpoot and Jense, 2015). For example, a PO can only view cameras (live stream or recording), which are installed within a mile radius of her ‘location’, or if the ‘timestamp’ of the requested camera-recording lies within her ‘duty-hours’. In this case, observers are allowed to view all the cameras within their allowed physical context irrespective of whether they need to view more or fewer cameras to complete their tasks. This limits the exposure of the observer, as they have access to a limited number of cameras, both location and timewise. The second approach is to limit the observer's view based on the high-level semantic information obtained from the content of the video stream. For example, a TM can be limited to only view frames of video streams that detect ‘speeding vehicles’, or PO can be restricted to view video streams that detect 'face' similar to that of a person-of-interest. In this case, observers do not have indefinite access to some cameras’ feed (as the first approach), as they will be restricted to view a specific portion of recordings (from any location or time) that have similar semantic content that they are ‘allowed’ to view. The second approach based on semantic content is more flexible and observer-oriented than just the physical context of time and space. However, observers are still exposed to some irrelevant data as even within the allowed duration of recording; it can still have information that is inapt for the observer. For instance, TM is allowed to view a recording where the camera caught a ‘speeding’ (event of interest) ‘vehicle’ (object of interest), here TM is only interested in viewing the ‘license plate’ of a ‘vehicle’ to find owner's information via automatic number plate recognition (ANPR). Yet, TM can view all the persons (drivers and passengers) in that vehicle, and all the other vehicles and persons on the road, at the 'time' or 'location' when only one of those vehicles was caught in a violation. It is important to note here that these two approaches are not mutually exclusive and can be combined in different ways creating a hybrid approach to further limit the exposure of irrelevant information. Ideally, TM should only be allowed to view the ‘license plate’ of the 'vehicle' from the camera installed at a specific 'location' from 'time' when 'traffic violation' was detected. Alternatively, the PO can only view 'humans' with specific descriptive or biometric features at the time of an ‘accident’. The hybrid approach considering both physical and semantic context can help limit the exposure of information to the observers depending upon their requirements while preserving privacy.

Thus, for VSS to enforce a dynamic need-to-know view for different observers, it is essential to understand their data requirements, i.e., what is it that they need to view in order to complete a particular task, referred to as the ‘purpose’ of the observer. The purpose should explicitly state two points: first, contextual (physical or semantic) properties that are required to limit the sequence or duration (series of frames) of the recording for viewing. Second, the type of information required from the selected content/video sequence that is required by the observer to complete his authorized task. Here, we assume that all observers are authorized and have a valid legal base to support their purposes. It is also imperative that the user's purposes should be described in a way similar to the type of information that is derived from the VSS data, so they can be mapped or compared against each other. This way, part of the surveillance data (‘events’ or ‘objects’ identified in the content) can be used to enforce restrictions on the other part of the data (objects of interest), and irrelevant data can be hidden (Sultan and Jensen, 2021). In this paper, we propose to extend Attributes Enhanced Role-Based Access Control (AERBAC) model with content-based restrictions in a way that allows observers to have explicit access to the content relevant to their purposes without compromising privacy. The proposed model will take into account both the physical and semantic context to control the amount of information from video-recordings based on the ‘purpose’ of the observer. This will limit the possibility of prejudiced interpretation by observers and enforce fine-grained need-to-know permissions while allowing multiple observers to achieve their authorized purposes, without compromising individuals’ privacy (Eckhoff and Wagner, 2018). The rest of this paper is organized as follows. Section 2 presents a smart-city video surveillance example to observe VSS privacy requirements and is used to illustrate important points throughout the paper. Section 3 investigates VSS data properties and how can they help access control mechanisms to limit the exposure of information to the observer. Section 4 will present and analyze the extended AERBAC model to enforce a context-based need-to-know view for observers. Finally, the last section will survey the related work followed by a conclusion.