Least median of squares and regression through the origin

doi:10.1016/j.csda.2005.01.005

Computational Statistics & Data Analysis

Volume 50, Issue 6, 10 March 2006, Pages 1391-1397

https://doi.org/10.1016/j.csda.2005.01.005 Get rights and content

Abstract

An exact algorithm is provided for finding the least median of squares (LMS) line for a bivariate regression with no intercept term. It is shown that the popular Program for RObust reGRESSion (PROGRESS) routine will not, in general, find the LMS slope when the intercept is suppressed. A Microsoft Excel workbook that provides the code in Visual Basic is made available at http://www.wabash.edu/econexcel/LMSOrigin.

Introduction

Rousseeuw (1984) introduced least median of squares (LMS) as a robust regression procedure. Instead of minimizing the sum of squared residuals, coefficients are chosen so as to minimize the median of the squared residuals. Unlike conventional least squares (LS), there is no closed-form solution with which to easily calculate the LMS line since the median is an order or rank statistic. A general non-linear optimization algorithm performs poorly because the median of squared residuals surface is so bumpy that merely local minima are often incorrectly reported as the solution.

Although a closed-form solution does not exist and brute force optimization is not reliable, several algorithms are available for fitting the LMS line (or hyperplane). Perhaps the most popular approach is called Program for RObust reGRESSion (PROGRESS). The program itself is explained in Rousseeuw and Leroy (1987) and the most recent version is available at http://www.agoras.ua.ac.be/. Several software packages, such as SAS/IML (version 6.12 or greater), have an LMS routine based on PROGRESS.

This paper focuses on the special problem of finding the LMS fitted line through the origin in the bivariate case. The next section presents the model and defines the LMS line. Section 3 shows that the PROGRESS algorithm gives an incorrect solution, in general, when the intercept is restricted to zero. Section 4 presents an analytical, exact method for finding the minimum median squared residual for the bivariate, zero intercept case. Finally, a simple example is provided to illustrate the algorithm and show why PROGRESS fails in the zero-intercept case.

Section snippets

The model

Suppose that observed values of $y$ are generated according to the model $y_{i} = μ x_{i} + ε_{i}$ . Given a realization of n $(x_{i}, y_{i})$ points, the problem is to find the ‘best’ choice for the slope of a straight line that passes through the origin, $\hat{y} = mx$ .

One choice for the ‘best’ line is the line that minimizes the median of the individual squares of the deviations, or residuals. Given this objective, one chooses the value of the slope, $m$ , to minimize the median value of the squared residuals: $d_{i}^{2} (m) = {(y_{i} - {\hat{y}}_{i})}^{2} = {(y_{i} - {mx}_{i})}^{2},$

PROGRESS and LMS with a zero intercept

The PROGRESS algorithm is based on sampling subsets of points from the data in order to generate candidate LMS estimates. The size of each subset is determined by the number of coefficients to be estimated. For the table above, there is one coefficient to be estimated and five observations so there are five subsets, each containing one data point. For each of the $n$ subsets, the slope is computed. Using this slope, the squared deviation of each of the data points is calculated and the median of

An exact algorithm for LMS with a zero intercept in the bivariate case

The central idea of this algorithm that provides an exact solution to the LMS problem is the observation that there are a finite number of slopes, bounded by the number of pairs of points, which could provide the minimum median deviation. In most cases not all of the $O (n^{2})$ points need to be checked to determine the optimal slope.

For a given $x$ , $y$ pair, the square of the deviation of the fitted line from the actual data point is given by $d^{2} (m) = {(y - \hat{y})}^{2} = (y - mx)^{2}$ . As a function of the slope, m, this

An example

Fig. 2 shows an example of this algorithm using the five data points from Table 1. The slope varies from $m_{\min} = 1.4$ due to observation 5 to $m_{\max} = 3$ due to observation 3.

Each parabola is labeled by the data point that relates to that parabola, obs 1, obs 2, obs 3, obs 4 and obs 5. The median squared residual for a given slope, $m$ , is the median, or middle, one of the $y$ values of the 5 parabolas. The thick line follows the median, or 3rd, deviation in this example of 5 data points. The vertical line

Conclusion

When applying least median of squares, coefficients are chosen so as to minimize the median of the squared residuals. Because the median is not sensitive to extreme values, it can outperform conventional least squares when data are contaminated. This paper makes two contributions to the LMS literature:

(1)
PROGRESS, the standard algorithm for fitting the LMS estimator, does not find the true LMS fit when the intercept is suppressed. Any computations based on the estimated slope (such as regression

Acknowledgements

The authors thank Michael Axtell, Frank Howland, and anonymous referees for suggestions and criticisms.

References (2)

P.J. Rousseeuw
Least median of squares regression
J. Amer. Statist. Assoc.
(1984)
P.J. Rousseeuw et al.
Robust Regression and Outlier Detection
(1987)

Cited by (17)

Novel Method for Accurately Assessing Pull-up Artifacts in STR Analysis
2021, Forensic Science International: Genetics
OSIRIS is a mathematically-based software tool for Short Tandem Repeat (STR) and DNA fragment analysis (https://www.ncbi.nlm.nih.gov/osiris/). As part of its routine sample analyses, OSIRIS computes unique quality metrics that can be used for sample quality assessment. A common artifact of STR analysis is cross-channel pull-up or pull-down (negative pull-up). This occurs because of the spectral overlap between the dyes used with the marker set, and the failure of the color deconvolution matrix to isolate the colors in the dye set adequately. This paper describes a mathematical method for analyzing and quantifying the pull-up patterns across sample channels and effectively identifying and correcting the pull-up artifacts, as implemented in OSIRIS. Unlike approaches to pull-up that require a training set composed of previous samples, the algorithm described here uses a mathematical model of the underlying causes of pull-up. It is based solely on the information intrinsic to the sample it is analyzing and therefore incorporates the effects of the ambient conditions and the specific procedures used in creating the sample. These conditions are the physical determinants of the level of pull-up in the sample and are not likely to be represented in a training set consisting of past samples.
CFD modelling of diffusive-reactive transport of ozone gas in rice grains
2019, Biosystems Engineering
The search for alternatives to the use of chemical products to control insect pests in stored grains has stimulated the development of new techniques that allow the maintenance and preservation of grain quality without posing risks to people and the environment. One of these alternatives is the use of ozone gas (O₃) as a fumigant, mainly due to its oxidising and biocidal characteristics. To investigate the transport mechanisms involved in the flow of the O₃ gas through rice grains, the CFD (Computational Fluid Dynamics) analysis was used, and from this evaluation, it was possible to predict the decomposition reaction constant and the O₃ effective diffusion coefficient. In the experiment, grains were submitted to fumigation process in a prototype adapted from a diaphragm cell. Data on the ozone gas concentration were collected from a monitoring point immediately above the grain layer every 10 min. Parallel to the experimental procedure, the modelling of the O₃ gas flow using the CFD technique was performed. The adjustment parameters input to the CFD model were the effective diffusion and the decomposition reaction constant of the O₃ gas in the rice grains. The estimated diffusivity value (1.0 × 10⁻⁶ m² s⁻¹), and decomposition reaction constant (0.00167 s⁻¹) are of the same order of magnitude of several other gases for agricultural grains.
Computing the least quartile difference estimator in the plane
2007, Computational Statistics and Data Analysis
A common problem in linear regression is that largely aberrant values can strongly influence the results. The least quartile difference (LQD) regression estimator is highly robust, since it can resist up to almost $50 %$ largely deviant data values without becoming extremely biased. Additionally, it shows good behavior on Gaussian data—in contrast to many other robust regression methods. However, the LQD is not widely used yet due to the high computational effort needed when using common algorithms. It is shown that it is possible to compute the LQD estimator for n bivariate data points in expected running time $O (n^{2} \log n)$ or deterministic running time $O (n^{2} \log^{2} n)$ . Additionally, two easy to implement algorithms with slightly inferior time bounds are presented. All of these algorithms are also applicable to least quantile of squares and least median of squares regression through the origin, improving the known time bounds to expected time $O (n \log n)$ and deterministic time $O (n \log^{2} n)$ . The proposed algorithms improve on known results of existing LQD algorithms and hence increase the practical relevance of the LQD estimator.
A Nonlinear Transformation Methods Using Covid-19 Data in the Kurdistan Region
2022, Proceedings of the 2nd 2022 International Conference on Computer Science and Software Engineering, CSASE 2022
New Robust Tensor PCA via Affine Transformations and L2,1 Norms for Exact Tubal Low-Rank Recovery from Highly Corrupted and Correlated Images in Signal Processing
2022, Mathematical Problems in Engineering
New Robust Regression Method for Outliers and Heavy Sparse Noise Detection via Affine Transformation for Head Pose Estimation and Image Reconstruction in Highly Complex and Correlated Data: Applications in Signal Processing
2022, Mathematical Problems in Engineering