A parametrized approach for linear regression of interval data

Abstract

Interval symbolic data is a complex data type that can often be obtained by summarizing large datasets. All existing linear regression approaches for interval data use certain fixed reference points to model intervals, such as midpoints, ranges and lower and upper bounds. This is a limitation, because different datasets might be better represented by different reference points. In this paper, we propose a new method for extracting knowledge from interval data. Our parametrized approach automatically extracts the best reference points from the regressor variables. These reference points are then used to build two linear regressions: one for the lower bounds of the response variable and another for its upper bounds. Before the regressions are applied, we compute a criterion to verify the mathematical coherence of predicted values. Mathematical coherence means that the upper bounds are greater than the lower bounds. If the criterion shows that the coherence is not guaranteed, we suggest the use of a novel interval Box-Cox transformation of the response variable. Experimental evaluations with synthetic and real interval datasets illustrate the advantages and the usefulness of the proposed method to perform interval linear regression.

Introduction

Linear regression is related to the construction of models that explore linear dependency between variables. Two types of variables are involved: the response (or dependent) variable and the regressor (or independent) variables. The main goal is to define a linear model which explains the response based on regressors. It can be used to predict unknown or unobservable values of the response variable based on the regressor variables’ values [1], [2], [3], [4].

Symbolic Data Analysis (SDA) [5], [6] defines a way of extracting knowledge from complex data types, called symbolic data, which represent higher level units, such as classes or concepts. In order to take into account the variability within each unit member, they can be described by intervals, distributions, sets of categories or numbers, which can sometimes be weighted. The first step in SDA is to build a symbolic data table where the rows are higher level units and the columns can take symbolic values. The second step is to study and extract new knowledge from these new kinds of data using extensions of Computer Statistics and Data Mining to symbolic data. SDA gives answers to big data and complex data challenges, because big data can be reduced and summarized by classes [7].

In the literature of linear regression applied to interval data, there are many works which do not assume any distribution for the residuals. Billard and Diday [8] proposed the Center Method (CM), which builds a linear regression model using the midpoints of response and regressor intervals and predicts response bounds using the regressor variable bounds. Later, Billard and Diday [9] discussed the MinMax Method (MinMax), which defines two models, one for each response bound, with lower bounds depending on the regressor variables’ lower bounds and the response upper bounds depending on the regressor variables’ upper bounds. Lima Neto and De Carvalho [10] introduced the Center and Range Method (CRM), which also proposes two linear models: one for the midpoints and another for the ranges. Lima Neto and De Carvalho [11] later extended CRM to include positive constraints to the coefficients of interval ranges, introducing the Constrained Center and Range Method (CCRM) which guarantees the mathematical coherence of predicted values, where the predicted upper bounds are greater than or equal to their respective lower bounds. Wang et al. [12] proposed a linear model which uses all interval points, named Complete Information Method (CIM). By using Moore’s linear combination [13], CIM also guarantees the mathematical coherence of its predictions.

Other methods assume certain properties for the datasets. Domingues et al. [14] introduced an interval linear regression which is robust against outliers and builds two models, which have symmetric errors for centers and ranges. In another work, Lima Neto et al. [15] suggested the representation of intervals as bivariate vectors. They use a bivariate symbolic regression for interval data which is based on generalized linear models. Souza et al. [16] proposed multi-class logistic regression models that employ different interval representations. Fagundes et al. [17] proposed a robust regression model for intervals which is resistant to outliers. A method based on kernel functions was introduced by Fagundes et al. [18]. They suggested the use of non-parametric functions for interval centers and ranges. Giordani [19] proposed a Lasso-IR method using one regression model for the interval centers and another for the ranges. The models’ coefficients are related by a degree of diversity which is a parameter of the method. The sum of the squared errors is minimized by Least Absolute Shrinkage and Selection Operator (Lasso) [20], which also includes a limit for the sum of absolute coefficient values. Lima Neto and De Carvalho [21] extended classical nonlinear regression to build an interval nonlinear regression model. They use some optimization algorithms to build models with the best accuracy and prediction precision.

Due to the complex nature of interval data, the formulation of regression models is not trivial. Usually, the methods proposed in the literature and discussed above choose certain reference points or parameters from intervals to build the models, such as midpoint and range or lower bound and upper bound. The problem with this kind of approach is that there are differences in the information contained in intervals from different datasets.

If a method always fixes the same reference points to build its models, it might have a poor performance with datasets that would be better modeled by another set of reference points. Therefore, this paper proposes the novel Parametrized Method (PM) for interval linear regression modeling. In this new approach, the intervals of regressor variables are parametrized through the parametric equation of the straight line. Two models are proposed for the estimation of response bounds. PM discovers, automatically, the set of reference points from regressor variables to build the regression models. This is an improvement and a generalization over existing methods.

In addition to being able to choose the reference points that better represent the intervals, a criterion is proposed to verify the mathematical coherence of the model’s predictions, before building the regression. A novel strategy, through interval transformations applied to the response variable, guarantees the mathematical coherence.

The rest of the paper is organized as follows: Section 2 reviews regression methods available for SDA; Section 3 shows the construction of PM, explaining the model fitting and the least squares solution; Section 4 introduces a procedure for PM models which guarantees the mathematical coherence; Section 5 provides experimental evaluation of regression methods, using synthetic and real interval data and Section 6 presents some concluding remarks.