Novel Bayesian inference on optimal parameters of support vector machines and its application to industrial survey data classification

Abstract

Engineering Asset Management (EAM) is a recently attractive discipline and it aims to address valuable contributions of asset management to organization׳s success. As of today, there is no specific method to evaluate performance of EAM standards. This paper aims to fill this gap and rank performance of asset management automatically after conducting survey, instead of evaluating questionnaires, analyzing results and ranking performances with a tedious process. Hence, it is necessary to develop intelligent data classification to simplify the whole procedure. Among many supervised learning methods, support vector machine attracts much attention for binary classification problems and its extension, namely multiple support vector machines, is able to solve multiclass classification problems. It is crucial to find optimal parameters of support vector machines prior to their use for prediction of unknown testing data sets. In this paper, novel Bayesian inference on optimal parameters of support vector machines is proposed. Firstly, a state space model is constructed to find the relationship between parameters of support vector machines and guess cross-validation accuracy. Here, the guess cross-validation accuracy aims to prevent support vector machines from overfitting. Secondly, particle filter is introduced to iteratively find posterior probability density functions of the parameters of support vector machines. Then, optimal parameters of support vector machines can be found from the posterior probability density functions. Ultimately, survey data collected from industry are used to validate the effectiveness of the proposed Bayesian inference method. Comparisons with some randomly selected parameters are conducted to highlight the superiority of the proposed method. The results show that the proposed Bayesian inference method can result in both high training and testing accuracies.

Introduction

Engineering Asset Management as a discipline addresses valuable contributions of asset management to organization׳s success [1]. Good asset management is becoming an expected practice in mature organizations all over the world. PAS 55:2008, which is the first publicly available specification for optimized management of physical assets, was developed by a consortium of 50 organizations from 15 different industry sectors in 10 countries. Given the popularity of PAS 55, after consultation with industry and professional bodies around the world, the specification was put forward in 2009 to the International Standards Organization as the basis for a new ISO standard for asset management. This was approved and the resulting ISO 55000 family of standards have been developed with 31 participating countries [2]. A EAM certificate provides recognized credibility in good practice and corporate governance, and a robust platform for developing further improvements.

A number of utility service providers have obtained asset management certificates through hiring well-known consultancy companies to perform auditing and performance assessment on EAM. However, many small and medium-sized enterprises (SMEs) cannot afford to hire renowned consultancy companies to guide them in obtaining required certificates and provide more professional suggestions to optimize asset management. Therefore, one purpose of the current research in EAM is to build an intelligent system so that it can automatically classify different performance levels of a particular company and then identify the most suitable practice in EAM for that company after benchmarking with information and performances given by other companies. For the other, PAS-55 only lists general guidelines in what elements are required to be accomplished so as to obtain the certificate in EAM. There is no specific method to evaluate the standard implementations and measure performance for managing assets. This paper aims to fill this gap and rank performance levels of asset management automatically and rapidly after conducting survey, instead of evaluating questionnaires, analyzing results, and ranking their performances with a tedious process. To achieve this goal, a novel Bayesian inference method for finding optimal parameters of support vector machines is proposed in this paper. The major reason why support vector machines are adopted is that it has many unique advantages in solving small samples, nonlinear and high-dimensional pattern recognition problems [3], [4], [5]. Moreover, after optimizing support vector machines, this newly method not only has accomplished the requirements for ranking performance levels, but also can improve effectiveness and efficiency of analyzing and measuring performance levels in a simplified and low costing way. Moreover, predicted performance levels can be provided to other small and medium-sized enterprises (SMEs) and industries for benchmarking and proceed further survey and research.

The novelties of the proposed Bayesian inference method are summarized as follows. Firstly, a state space model is constructed to establish the relationship between parameters of support vector machines and guess cross-validation accuracy. Here, the guess cross-validation aims to alleviate the overfitting problem in the training process of support vector machines. Secondly, particle filter is introduced to iteratively obtain posterior probability density functions of parameters of support vector machines. According to our literature review, the particle filter for one-dimensional optimization [6], wind farm layout design [7], slurry pump prognosis [8], bearing fault diagnosis [9], etc., have been reported. However, its use for optimization of supervised learning methods, particularly support vector machines, is very limited and seldom reported. The contents reported in this paper could be used to clarify how the particle filter is able to find optimal parameters of support vector machines. Moreover, because support vector machine is just one kind of supervised learning methods, the proposed Bayesian inference on optimal parameters of support vector machines can be extended to optimize parameters of other supervised learning methods.

The rest of this paper is outlined as follows. Fundamental theories related to the proposed Bayesian inference method are simply reviewed in Section 2. The novel Bayesian inference method for finding optimal parameters of support vector machines for multiclass classification problems is proposed in Section 3. Industrial survey data are analyzed in Section 4 to demonstrate the effectiveness of the proposed Bayesian inference method, and comparisons with some randomly selected parameters are conducted. Conclusions are drawn at last.