Data adaptive functional outlier detection: Analysis of the Paris bike sharing system data

Abstract

Bike sharing systems (BSSs) have become an increasingly popular means of sustainable transportation, and have been implemented in many cities worldwide. Our approach contributes to the identification of abnormal patterns by applying real-time occupancy data from Paris. In particular, we propose a novel functional outlier detection algorithm based on a two-step approach: In the first stage, a clean dataset is obtained based on the combined effect of two extreme statistics calculated from random sampling; in the second stage, a multiple testing approach based on the clean dataset is proposed, in which the false discovery rate (FDR) control procedure is used to adaptively choose the thresholds for the hypothesis tests. Extensive numerical simulations were conducted to compare the outlier detection performance with those of other state-of-art methods. The proposed approach is then applied to the Paris Vélib’ bike sharing system dataset to identify abnormal patterns that are of particular interest to BSS operators for identifying system inefficiencies and update policies.

Introduction

With the development of positioning devices, Internet-of-Things (IoT) modules, and sensor networks, massive numbers of transportation data that contain considerable undiscovered knowledge have been collected. The exploration of traffic monitoring data can provide deep insight into public transportation facility planning, route optimization, and surveillance purposes. For example, traffic volume data are useful for estimating the design–hour volumes and predicting the travel times [18], and traffic planners trace vehicle the trajectories to optimize the locations of charging stations. There have been a wide variety of research streams in the processing of large datasets in the transportation sector. Zhu et al. [43] predicted the future state of road traffic using a clustering method combined with a recurrent convolutional neural network, Deb and Liew [10] proposed an algorithm for identifying and correcting noisy categorical attribute values in large traffic accident datasets, with a probability measure inferred from statistics within the dataset and Teng et al. developed a multi-step forecasting method for online car-hailing demand [34].

This research was motivated by the real-world application of discovering patterns in the usage of stations in the Vélib’ bicycle sharing system (BSS) in Paris, France. Vélib’ BBS has approximately 14,500 bicycles at 1,230 rental stations located within the Paris metropolitan area [15]. Users are allowed to retrieve and return bikes to any available station, and staff members redistribute bicycles among rental stations overnight. The data contain hourly occupancy information in terms of the available bicycles and docks at each station. In this context, the task of outlier detection is to discover occupancy profiles that differ substantially from or are inconsistent with the remaining set. For instance, in the case of an unexpected system disruption, operators can identify the potential root causes after identifying the outlying trajectory [30]. Outlier detection can also contribute to the adjustment of pricing policies and improve the redistribution efficiency of bikes over stations.

The bike rental occupancy data are continuously measured for all rental stations in real time, which may result in high-dimensional data given a long observation period. Consequently, functional data analysis (FDA) has become a popular statistical tool that assumes that measurement vectors are realizations of functions defined on some continuous domains. Modeling data into functions is a viable way to recover the true nature of the underlying data generation process, and the smoothness characteristic of the functions is robust to measurement noise. Functional outlier detection is important for several reasons. Outliers pose significant challenges in obtaining coherent statistical analyses, resulting in biased estimations, misleading inferences, and poor predictions [29]. However, they may also carry important information regarding the nature of the underlying data-generation process; hence, the identified outliers require further investigation.

In terms of outlier detection methods for functional data, previous studies have extended outlier detection methods for multivariate data to functional settings; however, they encounter some apparent drawbacks [12]. Given that functional data are intrinsically infinite-dimensional, outlier detection methods for multivariate data are affected by the curse of dimensionality; thus, plotting methods are difficult to visually assess. There are also a limited number of methods that explicitly address the detection of functional outliers. One of the most popular methods is based on graphical approaches utilizing the concept of functional depth, as in [12], [32]. This provides a center-outward ordering score of the set of curves, where curves with a higher depth are close to the center of the functional distribution, whereas functional outliers are far from the center of the data with a significantly lower depth. Another type of approach is based on statistical measures, which include the robust principal component analysis method developed by Hyndman and Ullah [22], the high-breakdown mean function estimator created by Ren et al. [29], and the successive likelihood ratio test and smoothed bootstrapping developed by Febrero et al. [11].

In this study, we propose a novel functional outlier detection method for identifying outlying usage patterns in BSS data in Paris, France. Given the infinite-dimensional nature of functional data, this method is based on a functional principal component analysis (FPCA) to provide a low-dimensional representation of the original data. The outlier detection process is formulated as a statistical hypothesis testing procedure that has competitive power for detecting outliers with controlled sizes. Because the method used to select a clean set is based on the joint effect of the maximum and minimum statistics, the proposed algorithm is called the max–min functional outlier detection algorithm (MM-FOD). The code used to reproduce the results of this study is available at  https://github.com/LC8736/MM-FOD.git.

To summarize, the main contributions of this study are as follows:

• We propose a novel two-stage functional outlier detection approach, which can avoid the masking and swamping effect in outlier detection. This algorithm is summarized in Algorithm 1.

• A novel clean set construction method is proposed based on the joint effect of the maximum and minimum statistics, which is easy to implement and enjoys a competitive accuracy in comparison to the existing benchmark methods.

• The FDR control procedure is used to adaptively determine the threshold of the multiple hypothesis testing procedure in the second stage, which controls the false positive rate within the specified significance level.

• The proposed outlier detection method is applied to the Vélib’ BSS data in Paris, and the identified outliers provides system operators with additional insight to improve the efficiency of the system.

The remainder of this paper is organized as follows. In Section 2, we briefly review the preliminaries of functional outlier detection. The proposed MM-FOD algorithm is presented in detail in Section 3. The simulation study and real-world case study are provided in Sections 4 Simulation study, 5 Analysis of bike-sharing systems, respectively, and Section 6 provides some concluding remarks regarding this research.