Elsevier

Expert Systems with Applications

Volume 42, Issue 24, 30 December 2015, Pages 9449-9467
Expert Systems with Applications

Automatic surveillance in transportation hubs: No longer just about catching the bad guy

https://doi.org/10.1016/j.eswa.2015.08.001Get rights and content

Highlights

  • Surveillance and video analytics systems are typically only used for security.

  • We explore how emerging video analytics can support security and operations needs.

  • A conceptual framework combining security and operations is proposed.

  • An analysis of the proposed framework as applied to an airport is presented.

  • We show how many technologies have dual applications, and the benefits this brings.

Abstract

As critical infrastructure such as transportation hubs continue to grow in complexity, greater importance is placed on monitoring these facilities to ensure their secure and efficient operation. In order to achieve these goals, technology continues to evolve in response to the needs of various infrastructure. To date, however, the focus of technology for surveillance has been primarily concerned with security, and little attention has been placed on assisting operations and monitoring performance in real-time. Consequently, solutions have emerged to provide real-time measurements of queues and crowding in spaces, but have been installed as system add-ons (rather than making better use of existing infrastructure), resulting in expensive infrastructure outlay for the owner/operator, and an overload of surveillance systems which in itself creates further complexity. Given many critical infrastructure already have camera networks installed, it is much more desirable to better utilise these networks to address operational monitoring as well as security needs.

Recently, a growing number of approaches have been proposed to monitor operational aspects such as pedestrian throughput, crowd size and dwell times. In this paper, we explore how these techniques relate to and complement the more commonly seen security analytics, and demonstrate the value that can be added by operational analytics by demonstrating their performance on airport surveillance data. We explore how multiple analytics and systems can be combined to better leverage the large amount of data that is available, and we discuss the applicability and resulting benefits of the proposed framework for the ongoing operation of airports and airport networks.

Introduction

The scale of challenges facing society in providing more advanced critical infrastructure, and in particular transportation hubs, is substantial. An increased importance has been placed (and will continue to be placed) on transportation hubs to accommodate increased demand as cities continue to expand, and global air travel becomes more accessible. Consequently, the complexity of these hubs is increasing; airports are a perfect example where a multitude of factors are continually in play, including technological advancements, changes in regulations, and the interaction of multiple stakeholders including (but not limited to) government agencies, airport operators, airlines, security contractors, commercial operators, and of course, the travelling public (Ashford, Coutu, & Beasley, 2013). With this increase in demand and complexity, it can be argued that it is becoming more important to monitor the operational performance of the system. This is especially true for many privatised airports who often depend on non-aviation revenue sources to a greater extent than traditional aviation revenue.

Unfortunately, with millions of passengers passing through these hubs on a daily basis, the infrastructure itself become prime targets for terrorist activities (Tsai, Rathi, Keikintveld, Ordonez, & Tambe, 2009). Recent examples of such attacks on transportation hubs include the London train bombings in 2005, the Glasgow Airport car bombing in 2007, the Domodedovo International Airport bombing in 2011; the Peshawar Airport attack in 2012 and the Jinnah International Airport attack in 2014. With this constant threat in place, it is extremely important to ensure transportation hubs are safe and secure for all involved.

Over the past decade, surveillance cameras have become commonplace in public locations, including transportation hubs (Adrem, Dell’orto, Lennerman, 2007, Wells, Allard, Wilson, Welsh, Farrington, 2009). This is a direct result of an increased focus on public safety and security, but can also be attributed to the reduced cost of cameras and their associated computing infrastructure which help to keep overall security costs down (Adrem et al., 2007). The increase in surveillance cameras has given rise to an increase in computer systems to manage, and in many cases to analyse the incoming camera feeds. At present, these feeds are typically monitored by staff, and detecting events as they happen is very challenging due to the sheer amount of data being presented to each operator.

To assist human operators in monitoring large CCTV networks, there has been a significant increase in computer vision research and development, to create algorithms to analyse and extract information from the CCTV feeds. These developments have tended to focus on security related tasks such as object/person tracking, perimeter surveillance, motion segmentation, abnormal event detection and recognition, and biometrics (e.g face, iris, fingerprint) for person identification (Fookes et al., 2010). Many of these algorithms have begun to be implemented within video management and analytic systems, giving commercial video analytic packages a wide range of (primarily) security capabilities.

Operational analytics however, such as crowd counting and queue monitoring, have received less attention and while commercial systems do exist to perform these tasks, they are fewer in number and often require specially placed cameras making them difficult to integrate with existing systems. This is despite the comparatively poor performance of security based systems, which are prone to missed detections and false alarms.

Furthermore, the integration of these emerging techniques into airports or other critical infrastructure has received limited attention. When discussing the implementation of surveillance techniques, existing research has focussed on how a single surveillance task may function within a piece of infrastructure in isolation (i.e. Li, Wu, Karanam, and Radke, 2014 considers person re-detection in an airport environment; while Arroyo, Yebes, Bergasa, Daza, and Almazán, 2015 consider suspicious behaviour detection in a shopping mall). Similarly, when considering the system wide implications of video surveillance, the literature has focussed on aspects such as the data and networking requirements of such large scale systems (Ajiboye, Birch, Chatwin, Young, 2015, Chang, Wang, Wang, Liu, Ho, 2012); interfaces to retrieve and display results (Ye, Liao, Dong, Zeng, & Zhong, 2015); or the needs of researchers and developers to aid in the development of such techniques (Nazare, dos Santos, Ferreira, & Robson Schwartz, 2014).

Within this paper we propose an automated surveillance framework for both operational and security tasks for on-site and across-site monitoring. Whilst ensuring that a wide range of possible surveillance technologies are included in this framework, we specifically discuss the overlap between video analytics for security and operational analytics for operational monitoring which can be exploited to make better use of CCTV networks in public spaces. We present an overview of intelligent surveillance techniques for security and operational tasks, and show that although security has long been the focus of surveillance deployments, the operational video analytics currently in development are in many ways, more appropriate for deployment.

We show, on airport surveillance data, how recent approaches can be used and combined to extract measures of operational performance such as crowd sizes, processing rates and dwell times. The performance of these approaches, as well as their strengths and weaknesses from a real-world standpoint (i.e. deployment requirements and challenges) are discussed, and we explore how these techniques can be used in tandem with other statistical modelling approaches to provide better situational awareness. To demonstrate how such a combined security and operations framework could benefit a transport hub, we develop a case study around airport security, incident response and level of service monitoring to demonstrate the potential of video analytics as a solution to both these needs.

The remainder of this paper is structured as follows: Section 2 presents an overview of intelligent surveillance and provides an outline of the current abilities of security analytics; Section 3 presents our proposed surveillance framework, and an overview of operational analytics and how they can be applied to a transport environment; Section 4 presents two case studies examining how our proposed framework could be applied to an airport environment; and Section 5 concludes the paper.

Section snippets

Automatic surveillance: a history in security

Surveillance systems are an essential and integral component in transportation networks, public places, and other critical infrastructure where it is necessary to monitor activities, threats, and to prevent or investigate criminal or other unwanted activity.

An increased focus on security coupled with falling costs of hardware has seen an increase in the number of CCTV management products available. Some systems (such as Iomniscient,1 BlueEye Video,2

The emerging role of automatic surveillance in operations management

A major limitation of existing surveillance systems is their focus on security at the expense of critical business performance data and operational information that can be extracted from the same system. Often, cameras placed to look for security threats in public areas (such as a fight or disturbance) are also able to observe an operational processes such as a queue, doorway or meeting place. These cameras can also be used to extract additional information of interest, such as the number of

Case study: the future for automatic surveillance in airports

In this section, we assess the modern airport as a case study, and detail how the automatic surveillance framework presented in Section 3 could be perceived to provide a secure, responsive and efficient airport environment. In Section 4.1 we discuss the continued support for traditional airport and aircraft security, and highlight the emergence of capabilities which can support timely and effective incident response management. In Section 4.2 we look at new capabilities for monitoring airport

Conclusion

In this paper, we have explored how intelligent surveillance is becoming increasingly focussed on operational analytics, and the implications of this for transport hubs and other large infrastructure. We have outlined two particular areas of operational analytics: crowd counting (including counting everyone in a scene, measuring pedestrian throughput and estimating queue size) and dwell time estimation; and have shown how valuable operational information can be obtained using these techniques.

Acknowledgements

This research was supported by the Australian Research Council’s Linkage Project “Airports of the Future” (LP0990135). The authors also acknowledge the contribution made by the many aviation industry stakeholders also involved in this project.

References (161)

  • AnantharajahK. et al.

    Local inter-session variability modelling for object classification

    2014 IEEE Winter conference on applications of computer vision (WACV)

    (2014)
  • AshfordN. et al.

    Airport operations

    (2013)
  • AuvinetE. et al.

    Left-luggage detection using homographies and simple heuristics

    IEEE international workshop on PETS, New York, June 18, 2006

    (2006)
  • BakS. et al.

    Person re-identification using haar-based and DCD-based signature

    Advanced video and signal based surveillance (AVSS)

    (2010)
  • BakS. et al.

    Person re-identification by pose priors

    IS&T/SPIE electronic imaging

    (2015)
  • BandiniS. et al.

    A qualitative evaluation of technologies and techniques for data collection on pedestrians and crowded situations

    Proceedings of the 2007 Summer computation simulation conference

    (2007)
  • BarandiaranJ. et al.

    Real-time people counting using multiple lines

    (2008)
  • BazzaniL. et al.

    Symmetry-driven accumulation of local features for human characterization and re-identification

    Computer Vision and Image Understanding

    (2013)
  • BazzaniL. et al.

    Joint individual-group modeling for tracking

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2015)
  • BellottoN. et al.

    A distributed camera system for multi-resolution surveillance

    International conference on distributed smart cameras

    (2009)
  • Ben ShitritH. et al.

    Multi-commodity network flow for tracking multiple people

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2014)
  • BerclazJ. et al.

    Multiple object tracking using k-shortest paths optimization

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2011)
  • BlackM. et al.

    A framework for the robust estimation of optical flow

    Fourth international conference on computer vision

    (1993)
  • BleiD. et al.

    Probabilistic topic models

    IEEE Signal Processing Magazine

    (2010)
  • BleiD.M. et al.

    Latent dirichlet allocation

    Journal of Machine Learning Research

    (2003)
  • BoehmK. et al.

    Enhancing situation awareness in real time geospatial visualization

    Americas conference on information systems (AMCIS 2005)

    (2005)
  • BondiE. et al.

    Real-time people counting from depth imagery of crowded environments

    2014 11th IEEE international conference on advanced video and signal based surveillance (AVSS)

    (2014)
  • BoswellD. et al.

    Air, fire and ice: Fire & security challenges unique to airports

    Fire & Security Today

    (2007)
  • BreitensteinM.D. et al.

    Online multi person tracking-by-detection from a single, uncalibrated camera

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2011)
  • BrulinM. et al.

    Video surveillance traffic analysis using scene geometry

    Pacific rim symposium on image and video technologies

    (2010)
  • CancelaB. et al.

    Open-world person re-identification by multi-label assignment inference

    (2014)
  • ChanA. et al.

    Privacy preserving crowd monitoring: Counting people without people models or tracking

    IEEE Conference on Computer Vision and Pattern Recognition

    (2008)
  • ChanA.B. et al.

    Analysis of crowded scenes using holistic properties

    Performance evaluation of tracking and surveillance workshop

    (2009)
  • ChangR.-I. et al.

    Effective distributed service architecture for ubiquitous video surveillance

    Information Systems Frontiers

    (2012)
  • ChenT.-H.

    An automatic bi-directional passing-people counting method based on color image processing

    International Carnahan conference on security technology

    (2003)
  • ChenT.-H. et al.

    An intelligent people-flow counting method for passing through a gate

    IEEE conference on robotics, automation and mechatronics

    (2006)
  • CorreiaA. et al.

    Evaluating level of service at airport passenger terminals

    Transportation Research Record: Journal of the Transportation Research Board

    (2004)
  • DalalN. et al.

    Histograms of oriented gradients for human detection

  • DanelljanM. et al.

    Adaptive color attributes for real-time visual tracking

    2014 IEEE conference on computer vision and pattern recognition (CVPR)

    (2014)
  • D’AngeloA. et al.

    Color based soft biometry for Hooligans detection

    IEEE international symposium on circuits and systems (ISCAS)

    (2010)
  • DantchevaA. et al.

    Bag of soft biometrics for person identification: New trends and challenges

    Multimedia Tools and Applications

    (2011)
  • DaugmanJ.

    How iris recognition works

    IEEE Transactions on Circuits and Systems for Video Technology

    (2004)
  • DaviesA. et al.

    Crowd monitoring using image processing

    Electronics & Communication Engineering Journal

    (1995)
  • DengL. et al.

    Deep learning: Methods and applications

    Foundations and Trends in Signal Processing

    (2014)
  • DenmanS. et al.

    Determining operational measures from multi-camera surveillance systems using soft biometrics

    8th IEEE international conference on advanced video and signal-based surveillance (AVSS)

    (2011)
  • DenmanS. et al.

    A multi-class tracker using a scalable condensation filter

    Advanced video and signal based surveillance

    (2006)
  • DenmanS. et al.

    Soft-Biometrics: unconstrained authentication in a surveillance environment

    Digital Image Computing: Techniques and Applications

    (2009)
  • DenmanS. et al.

    Multi-view intelligent vehicle surveillance system

    Advanced video and signal based surveillance

    (2006)
  • DenmanS. et al.

    Large scale monitoring of crowds and building utilisation: A new database and distributed approach

    12th IEEE international conference on advanced video and signal based surveillance (AVSS)

    (2015)
  • DenmanS. et al.

    Improved simultaneous computation of motion detection and optical flow for object tracking

    Digital image computing: Techniques and applications

    (2009)
  • Cited by (16)

    • Affective analytics of demonstration sites

      2019, Engineering Applications of Artificial Intelligence
      Citation Excerpt :

      Haghani and Sarvi (2016) examined decision-making processes. Seok et al. (2012), Wagner and Agrawal (2014), Denman et al. (2015), Filip et al. (2016a, b), Yogameena and Nagananthini (2017), Li et al. (2017) and analyzed intelligent and recommender systems. The above sensor networks, analytics, modeling, and decision making, intelligent and recommender systems can be used either alone or combined in different ways.

    • A review on classifying abnormal behavior in crowd scene

      2019, Journal of Visual Communication and Image Representation
      Citation Excerpt :

      The general flow of detecting abnormalities in the surveillance video includes monitoring the activities, identifying the crowd features such as motion pattern, trajectory and density from the visual images. It is continued with the detection of any irregular activities in the video scene [10]. Assuming a normal event occurs frequently while an abnormal event rarely occurs, an outlier is usually determined as abnormalities.

    • Combining keypoint-based and segment-based features for counting people in crowded scenes

      2016, Information Sciences
      Citation Excerpt :

      It has been widely applied in fields such as the monitoring of public security, gathering of business intelligence, scheduling of public transportation, resource management, and optimization. The estimation of the number of people is also applied in other surveillance applications, such as detection of congestion, delay, unsafe situation, or other behavioral analysis of crowds [11,13]. Hence, many methods based on computer vision techniques have been proposed to perform this task automatically.

    • On realistic target coverage by autonomous drones

      2019, ACM Transactions on Sensor Networks
    View all citing articles on Scopus
    View full text