A latent information function to extend domain attributes to improve the accuracy of small-data-set forecasting
Introduction
The manufacturing environment is now changing rapidly, cutting product life cycles and causing increasingly intense competition among enterprises. How to control a manufacturing system effectively and efficiently is thus very important for manufacturing firms, especially in the early stages of such systems [1]. This is because businesses can raise their competitive ability if managers can quickly discover problems in manufacturing processes and take appropriate actions [2]. For this reason, suitable forecasting techniques are needed to boost managerial efficiency.
However, few observations are usually available in the early stages of manufacturing systems, so it is difficult to find robust results using prediction methods that depend on large data sets, such as multivariate analysis, time series models, and data mining techniques [3]. The purpose of this research is thus to establish a forecasting model based on small data sets to help engineers and decision makers to make better predictions under non-deterministic conditions.
Academically, the related uncertainty problems can be divided into roughly three categories: stochastic phenomena [4], cognitive uncertainties [5], and insufficient information [6]. The problems related to the early stages of a manufacturing system are due to the insufficient information caused by a limited sample size, which cannot completely reflect the whole features of a population [7], [8]. To overcome this, virtual sample generation (VSG) techniques are adopted to provide more stable learning methods and create robust and precise models.
In the literature, the prior knowledge obtained from a given small training set is used to create virtual samples to improve the learning results [9], [10], [11]. Li et al. [12] developed a Functional Virtual Population to expand the domain of the system attributes and generate virtual samples for scheduling problems in small data set. A bootstrap procedure was then proposed to enhance statistical inference in simulation experiments by generating bootstrap samples for training [13], [14]. Other VSG algorithms have emerged in recent years, based on the principles of information diffusion [15] derived from fuzzy theory, and the related approaches have been used in many fields, such as medicine, management and manufacturing [16], [17], [18], [19], [20], [21].
However, the generation of virtual samples is usually not directly applied to time series data; because the developing trends of such data are closely related to the order of observations, and it is hard to maintain the appropriate relations among the virtual data that is produced, meaning that the approach cannot effectively improve the performance of model learning. Fig. 1 provides a simple illustration of this issue. If we create virtual samples and get the trend line, shown as the dotted line in the figure, we can see that there is a significant accumulative difference between the dotted line of virtual samples and the solid line of real data. To minimize this difference, the virtual samples have to be located close to the real trend line, and this weakens the effectiveness of this approach.
This study thus proposes a Latent Information (LI) function to analyze data characteristics and extract information to assist knowledge acquisition with small data sets. The approach can acquire extra information by analyzing the data features to extend the domain attributes. To verify the effectiveness of the proposed method, this study employs the Synthetic Control Chart Time Series (SCCTS) dataset from the Knowledge Discovery Database and the monthly average price of aluminum for cash buyers from the London Metal Exchange (LME) to implement the experimental analysis. The experimental results show that the LI function is an appropriate technique for small-sample learning, because it can improve forecasting accuracy.
The remainder of this paper is organized as follows. In Section 2, the concept of the LI function is introduced. In Section 3, a demonstration of the procedure and experimental results are given. Finally, the conclusions are presented in Section 4.
Section snippets
Methodology
The samples collected in the early stages of manufacturing systems are usually not adequate for effective model learning, and past research has found that increasing quantities of information can gradually lead to more stable forecasting results. Correspondingly, the gathering of time series data can be treated as a successive and incremental data collection procedure, where the amount of incoming data will rise and the information gradually be updated. Therefore, an LI function based on
Experimental studies
In this section, we employ one artificial process data set and one real dataset, the Synthetic Control Chart Time Series dataset and the aluminum price dataset, respectively, to demonstrate the use of the LI function. The detailed experimental process is described in the following sub-sections.
Conclusions and discussion
In order to control operating costs in an effective manner, enterprises require appropriate forecasting technology, especially in the early stages of manufacturing systems. However, during these early stages the sample sizes are restricted by considerations of cost and time, and thus only insufficient information can be used to acquire knowledge, and traditional forecasting methods often fail to produce useful results. Therefore, it is very important to develop better small-data-set learning
Acknowledgments
This research is partially supported by the National Science Council of Taiwan under grant NSC 101-22188-E-033-004-.
Che-Jung Chang received his PhD degree in management science from National Cheng Kung University, Taiwan in 2011. He is currently an assistant professor in the Department of Business Administration at Chung Yuan Christian University, Taiwan. His recent research interests include grey system theory, production management and small-data-set learning. His articles have appeared in Omega, Applied Mathematical Modelling, Computers & Industrial Engineering and Journal of Grey System.
References (27)
- et al.
An improved grey-based approach for early manufacturing data forecasting
Comput. Ind. Eng.
(2009) - et al.
A neural network-based framework for the reconstruction of incomplete data sets
Neurocomputing
(2010) - et al.
Effective feature selection scheme using mutual information
Neurocomputing
(2005) Adaptive nonparametric density estimation with missing observations
J. Stat. Plan. Inference
(2013)Learning from hints in neural networks
J. Complexity
(1990)- et al.
Utilize bootstrap in small data set learning for pilot run modeling of manufacturing systems
Expert Syst. Appl.
(2008) - et al.
A diffusion-neural-network for learning from small samples
Int. J. Approx. Reason.
(2004) - et al.
Using virtual sample generation to build up management knowledge in the early manufacturing stages
Eur. J. Oper. Res.
(2006) - et al.
Utilization of virtual samples to facilitate cancer identification for DNA microarray data in the early stages of an investigation
Inform. Sci.
(2009) - et al.
Forecasting short-term electricity consumption using the adaptive grey-based approach-an Asian case
Omega-Int. J. Manage S
(2012)
Sales forecasting practices: results from a United States survey
Int. J. Forecast.
A New Reliability Prediction Model in Manufacturing Systems
IEEE Trans. Reliab.
Statistics: Theory and Methods
Cited by (24)
A transferred hybrid surrogate model integrating Gaussian membership virtual sample generation for small sample prediction: Applications in metal tube bending
2024, Engineering Applications of Artificial IntelligenceSmall-sample continual learning classification method with vaccine to update memory cells based on the artificial immune system
2022, BioSystemsCitation Excerpt :However, obtaining sufficient training samples is expensive and sometimes difficult to complete, such as aircraft failure data at runtime. The machine learning problem with few training sample data is called small sample learning problem (Raudys, 2006; Zhu et al., 2016; Chang et al., 2014a). In practice, the sample threshold of small sample problem is usually set at 30 (Cadini et al., 2019; Fukunada et al., 1990).
Dimensionality reduction for multi-criteria problems: An application to the decommissioning of oil and gas installations
2020, Expert Systems with ApplicationsA novel and effective nonlinear interpolation virtual sample generation method for enhancing energy prediction and analysis on small data problem: A case study of Ethylene industry
2018, EnergyCitation Excerpt :In other words, if one wants to build an accurate and reliable data-driven model, sufficient data and a good distribution assumption are two necessary conditions [1]. In the references of [2] and [3], small data problems refer to the case where the number of samples is less than 50 concerning engineering applications or less than 30 regarding academic researches. The whole features of a population are hard to completely be revealed by the small data because of the insufficient information [4].
A PSO based virtual sample generation method for small sample sets: Applications to regression datasets
2017, Engineering Applications of Artificial IntelligenceCitation Excerpt :Such algorithms are data-dependent, i.e., sufficient data and a good distribution assumption are the two necessary conditions to ensure a more accurate model in the applications of classification and regression. Small sample sets problems (Zhu et al., 2016; Chang et al., 2014a, 2015; Li et al., 2012, 2014; Li and Lin, 2013) refer to the case of small amount of samples, where the number of samples is less than 50 in respect to engineering applications or less than 30 in regard to academic researches. Such small number of sample sets cannot completely reveal the whole features of a population due to the insufficient information (Li and Fang, 2009).
Che-Jung Chang received his PhD degree in management science from National Cheng Kung University, Taiwan in 2011. He is currently an assistant professor in the Department of Business Administration at Chung Yuan Christian University, Taiwan. His recent research interests include grey system theory, production management and small-data-set learning. His articles have appeared in Omega, Applied Mathematical Modelling, Computers & Industrial Engineering and Journal of Grey System.
Der-Chiang Li is a distinguished professor at the Department of Industrial and Information Management, National Cheng Kung University, Taiwan. He received his PhD degree at the Department of Industrial Engineering at Lamar University, Beaumont, Texas, USA, in 1985. As a research professor, his current interests focus on learning with small data sets. His articles have appeared in Decision Support Systems, Information Sciences, European Journal of Operational Research, Computer & Operations Research, International Journal of Production Research, and other publications.
Wen-Li Dai is an associate professor at the Department of Information Management at Tainan University of Technology. He received his PhD degree from National Cheng Kung University. His primary research interests focus on the information system management, supply chain management and operations research. His research has been published in International Journal of Production Research, Expert Systems with Applications, and Web Journal of Chinese Management Review.
Chien-Chih Chen Chien-Chih Chen received his PhD degree from the Department of Industrial and Information Management at National Cheng Kung University, Taiwan in 2011. His current research interests are in the area of forecasting and data mining. His research has been published in International Journal of Production Research and Expert Systems with Applications.