1 Introduction

During the mirrors manufacture, the phase of optical test allows to identify optical defects which could be corrected to obtain the images of optical systems sharper. These defects are considered as deformations that affect the response of the optical system resulting into image degradation. One of the optical tests performed in mirrors is known as Ronchi test [8]. This method was proposed by the Italian physicist Vasco Ronchi in 1922 when he published about the formation of fringe or interference patterns, a.k.a. ronchigrams [8]. These interference patterns are generated by placing a grid near the center of curvature of a mirror, which are related to optical aberrations (i.e. errors in the shape of the surface). The study of phase shift is one of the procedures used for ronchigrams analysis, Zernike polynomials are generally used to quantify these aberrations reflected by the mirror under test. This technique has been used in concave mirrors [1], spherical surfaces using a LCD screen to show vertical and horizontal grids [2, 3, 6], with interferometry, linear polarization modulation [10] and for optical tests in flat mirrors [4]. However, for the correct interpretation of these ronchigrams, it is required precision of the measurement tools, to process information of images or electromagnetic signals, and application of knowledge from mathematics, physics and engineering [7]. For ronchigram image analysis, several methods have been developed to detect aberrations by applying image processing techniques into space and frequency domains [10]. The procedure to obtain the ronchigram images, according to the Ronchi test, is performed by sampling the wavefront of the response of the optical system, in such a way that benchmarks can be compared with those resulting from the test. However, the ronchigram image acquisition requires a specific device and the subsequent image processing techniques for analysis, that is also related with the type of camera, lighting and kind of algorithms or procedures.

This paper is organized as follows: Sect. 2 explains the details of the proposed method to detect optical defects based on BoVWs and SVMs, Sect. 3 describes the experimental evaluation, including the data set description, the experimental setup and results. Finally, in Sect. 4 we present the main conclusions and future work.

2 Methodology

The proposed approach used a methodology which starts with a SIFT-based Bag of Visual Words (BoVW) image representation and Support Vector Machines (SVM) for automatic classification of Ronchigram images. During classification process, a SVM model was trained, applying linear kernel functions, different variations of regularization factor and local feature extraction methods using Scale-Invariant Feature Transform (SIFT) algorithm, together with global image representation of “Bag of Words” (BoVW). In this way, to select the results with better performance of the classifier and choose the implementation with the solution of parameters more efficient, this algorithm is integrated as MARC (Module of Analysis of Ronchigrams) module, into the ANGMAR - Image Processing V1.0 software of SAPRULL [9].

2.1 Ronchigram Image Acquisition

The Optical Center at Universidad de los Llanos produces optical components such as a simple magnifying glass or even an objective mirror of an astronomical reflecting telescope. In order to evaluate the quality of the lenses and mirrors an assisted system is used. This system consists of a Computer-Assisted Ronchi Interferometer (CARI) module and a Ronchigram Computer Analysis (RCA) module. The practical arrangement identifies three primordial elements: (i) white light source, (ii) periodic grid (4 lines/mm) and the digital camera (Exmor R®CMOS sensor, type 1/2.3 (7.82 mm), 20.4 MP). Figure 1 shows the system used to acquire ronchigrams images [9].

Fig. 1.
figure 1

Source: [9].

Computer-aided system for Ronchi test - SAPRULL. 1. rail (1 m), 2. power source (12 V), 3. NEMA motors, 4. circuit set - long moving platform, 5. white light source, portable laser, 6. camera (CMOS), grid and slit - short motion platform, 7. mirror and bracket.

Full size image

2.2 Local Image Feature Extraction (SIFT)

In this stage, the detection of interest points and their description by a local feature representation of each ronchigram image was developed. The implementation used SIFT points and features (Scale Invariant Feature Transform), that is an algorithm to detect and represent local features of a given image. An important aspect of this approach is that it generates a large number of features that densely cover the image across the entire range of scales and locations. An image of size 256 \(\times \) 256 pixels will give rise to a certain number of stable characteristics (this number depends on the image content and the variation of existing parameters in the image), see Fig. 2.

Fig. 2.
figure 2

Local feature extraction of a Ronchigram image using SIFT. Source: Authors. (Color figure online)

Full size image

Extraction and description of interest points by SIFT for the image gives rise to different results as seen in the right ronchigram of Fig. 2, where 60 key points (points of interest) were extracted. The center Ronchigram has 292 points of interest and one on the left has 82 key points. These key points represent features that allow the creation of the visual dictionary of visual words in representation of the images of each class, then, to be compared and based on the euclidean distance of their characteristic vectors can find coincidence images that allow their clustering using the K-mean algorithm for dictionary construction.

2.3 Image Representation via Bag of Visual Words (BoVW)

The Ronchigram Images were represented using the Histogram of Bag of Visual Words approach, which starts with: (i) the local features extraction using the Scale Invariant Feature Transform (SIFT), to build (ii) the visual dictionary (vocabulary) using the clustering algorithm known as K-means, in order to represent the ronchigram images as a histogram of the frequency of bag of visual words of the dictionary. Figure 3 depicts the BoVW approach for Ronchigram image representation. The vocabulary construction of the BoVW was done using the K-means clustering algorithm. Each cluster represents a visual word of the visual vocabulary that represents, according to the occurrence, the visual content distribution in the image. The BoVW method stats with the detection and description of keypoints for each image, using SIFT descriptors. The vocabulary was built with 50.000 keypoints with a length of 1.024 visual words.

Fig. 3.
figure 3

Ronchigram image representation using BoVW [12].

Full size image

2.4 Image Classification via Support Vector Machine (SVM)

For the automatic classification of ronchigram images from the BoVW Image Representation, a Support Vector Machine (SVM) was trained. SVM is one of the common and successful methods used in supervised machine learning in several tasks, which is used either for regression and classification tasks. This method uses the “kernel trick”, which allows the computation of the original data representation into a high dimensional space. This high dimensional space is known as “feature space” ‘,’ where data is expected to be linearly separable by hyperplanes learned during training process for the classification task, independently of the kernel function applied in training. For instance, SVM has been successfully applied in regression applications like analysis of data, to determinate fraud or not-fraud in banking transactions [5] or in Multi-class classification to recognize hand-written digits from 0 to 9 [11]. Here, the original set of ronchigrams is obtained from the Ronchi Test applied to 14 parabolic mirrors, which were selected with alterations on their surfaces (optical aberrations). The original data set of 2.554 Ronchigrams and 5 classes, was divided into 70% for training set and 30% for testing set, using stratified sampling per class (Fig. 4).

Fig. 4.
figure 4

Source: Authors.

Ronchigrams of the library.

Full size image

3 Experimental Evaluation

3.1 Ronchigram Image Dataset Description

The data set was obtained from mirrors samples without defects and mirrors that present borders and/or center defects. For each sample a digitized image was obtained by the Ronchi test according to the process described in Sect. 2.1. In Fig. 5, we can appreciate the variations of the number of fringes when the ronchigram was acquired for different places with respect to the optical focus (0.5 cm, 1.0 cm, 1.5 cm, 2.0 cm). This behavior is validate by the expert in optics aberrations from the optical laboratory of the Universidad de los Llanos. These patterns highlight the importance of the relation between optical aberrations at different focal lengths for optical aberration differentiation. Table 1 describes the data set distribution per class for training and testing of the SVM classifier.

Fig. 5.
figure 5

Source: Authors.

Ronchigrama with movement inside of the focus, (a) 0.5 cm, (b) 1.0 cm, (c) 1.5 cm, (d) 2.0 cm.

Full size image
Table 1. Ronchigram image data set. Source: Authors
Full size table

3.2 Experimental Setup

Once the data set of ronchigram images was acquired according to description of Subsect. 2.1, the SVM classifier was trained using 5-fold cross-validation in training to obtain the best parameters. The kernel function used was linear because it is simplest and obtain best classification performance.

The performance measures of precision \(Pr=TP/(TP+FP)\), recall (Rc \(Rc=TP/(TP+FN)\) and accuracy \(Acc=(TP+TN)/(TP+TN+FP+FN)\), were used to evaluate the ronchigrams classification. Where true-positive samples (TP) are those samples in test data set that are correctly classified in the corresponding class, false-positive samples (FP) are those samples that were incorrectly classified in the class but actually are from other classes, true-negative samples (TN) are those samples that are not the class of interest and were correctly classified as other class, and false-negative samples (FN) are those samples that were incorrectly classified as other classes different of the class of interest. In addition, the Receiver operating characteristic (ROC) curve was calculated to analyze the classification performance per class.

3.3 Results

The best combination of SVM parameters was obtained by grid-search in training data set using: 5-fold cross-validation, exploring between “Linear” and “Radial” Kernels and using a range of 10 different values for the regularization parameter C between 0.0001 and 0.02 values. The best parameters were for“Linear” Kernel and \(C=0.001\) achieving a classification performance of: precision (0.71), recall (0.68), accuracy (0.69) and execution time (1.013 s). The execution time varies according to the number of parameters that are initialized for validation and also of the CPU characteristics where the program was executed. Figure 2 shows three example images of ronchigrams with the points of interest (red circles) detected using SIFT algorithm detection and description for local features. The main advantage of these kind of local feature detector and description is that are invariant to scale, rotation, and size. Particularly, similar ronchigram image patterns reveals similar points of interest with the same local feature representation, which is useful to detect similar optical defects for the posterior classification process.

Figure 6 presents two confusion matrices of the classification performance from the trained SVM model over the test set of ronchigram images. The ice colorbar represents that dark blue colors are higher values whereas white colors are lower values. In this case, the values of the right confusion matrix correspond to the accuracy performance measure and left matrix the number of samples. This figure shows that each confusion matrix are presenting good performance of the SVM models with a diagonal values closed to the optimal. Using the accuracy measure, the best results of SVM classification in test data set found was for \(C=0.001\) achieving an average accuracy of 0.69. Finally, ROC curve is shown in Fig. 7 in order to analyze the classification performance per each class which is higher for most of classes.

Fig. 6.
figure 6

Confusion matrices of the multi-class classification performance. (Color figure online)

Full size image
Fig. 7.
figure 7

ROC curve of the multi-class classification performance.

Full size image

4 Conclusions and Future Work

The proposed approach presented in this paper for automatic classification of optical defects from Ronchigram images using SIFT-based Bag of Visual Words and Support Vector Machines was successfully applied achieving high classification performances. Our approach includes the image acquisition process where samples of each class corresponds to optical defects of mirrors used during the Ronchi test validated by an expert. The ronchigrams vary in terms of number of fringes but the defect is keeped. However, images acquired from different distances did not alter the classification performance results. The used of SVM models requires the systematic exploration of hyperparameters through a grid search using cross-validation for each kind of kernel evaluated. This could be a time consuming task, which could suggest the exploration of alternative methods with less number of parameters with similar classification performance. The most robust results were obtained when both measures, precision and recall, were took into account. Future work includes to increase the number of image per each optical defect to train better models. In fact, data-driven and state-of-the-art methods based on convolutional neural networks will be evaluated.