A microcontroller based machine vision approach for tomato grading and sorting using SVM classifier

https://doi.org/10.1016/j.micpro.2020.103090Get rights and content

Abstract

Advances in computer vision have led to the development of promising solutions for challenging problems in agriculture. Fruit grading and sorting are complex problems which require a great deal of human expertise. In this paper, we propose a non-destructive system for sorting and grading tomatoes, which is confounding even for expert human sorters. The proposed system performs classifications of tomatoes in three stages with digital images of samples captured in an experimental setup deployed using microcontroller. In the first stage, a binary classification is performed to discriminate tomatoes from other species using a species vector constructed from these images. In the second stage, the tomatoes are classified into ripe and unripe categories based on the color attribute. Then, the defects in the fruits are identified using Gabor wavelet transform to segment the infected regions of these images. The third stage identifies three types of defects namely the black spots, cankers and Melanose, based on a defect vector constructed from additional color and geometric features. Due to the complexity involved in solving such a non-linear problem, the proposed system is implemented as a cascade of two support vector machine classifiers. The performance of this system is assessed with the accuracy, specificity, sensitivity and precision metrics. The experimental results and comparative analyses with similar methods testify the efficacy of the proposed system over existing systems on the sorting and grading of tomatoes.

The results obtained from each of the three classification stages i.e. Tomato/Non-Tomato, Good/Defective and the type of defect in the case of defective are communicated to the microcontroller to enable the respective motor, so that the given fruit is classified and collected in the respective bin.

Introduction

Agriculture sector in India provides unswerving employment to more than 50% of the workforce, contributing 16% of the Gross Domestic Product (GDP). The choice of crop cultivation is governed by cropping patterns, physical, economic and social factors. India is one of the largest producers of tomato with great prospects for export. The recent study on the tomato value chain by the National Horticultural Research & Development Foundation gives detailed accounts on the issues on tomato cultivation across the country. Finally, this study advocates the establishment of processing units for sorting and grading to reduce loss in the supply chain. The general labor-intensive manual sorting and grading process is subjective and not very accurate. Evolution of modern imaging and machine vision techniques have introduced non-destructive sorting and grading without human interventions.

In agricultural science, digital image processing based systems are commonly deployed in the detection of defects, sorting and grading the produce etc. Novel image acquisition devices and diverse facets of digital image processing such as segmentation, morphological and chromatic analyses, coupled with machine vision algorithms have increased the speed and accuracy of the quality evaluation and disease identification in agricultural products. Particularly, automatic detection of infections in plants with digital imaging techniques facilitates early detection and management of diseases. The earliest work on the use of nuclear magnetic resonance imaging to investigate the distribution and relaxation time of water in tomato dates back to 1989 [17]. In succession, new non-invasive imaging techniques for quality evaluation, defect discrimination and grading have been proposed.

Many research works [13], [24], [34], [45], [50] demonstrating the effectiveness of computer vision based non-destructive approaches for accurate fruit and vegetable grading in food industry and precision agriculture have been reported in the recent years.

Jhawar [20] presented a nearest neighbor and linear regression based pattern recognition based method for determining the maturity of oranges with a single color image of a fruit. Multispectral images were used in the evaluation of pomegranate [23] fruit quality based on the quantification of Total Soluble Solids (TSS), pH and firmness attributes.

An automatic system for grading tomato employing computer vision techniques proposed by Arakeri [5], comprises both software and hardware components. The hardware component captures the image and moves it to the respective bin after grading without human intervention. A non-destructive method for counting the number of fruits, detection of overlap among fruits, classification of harvestable fruits in coffee plants based on linear estimations was proposed by Ramos et al. [39]. An Artificial neural network (ANN) classifier for sorting ripe and unripe mangoes, implemented by Yossy et al. [51] reports an accuracy of 94% in.

An automatic mechanism for sorting and quality assessment of three varieties of apple cultivars was proposed by Sofu et al. [44]. In this method, two industrial cameras fixed on a conveyor belt captured four images of a single apple in real-time and the classifier algorithm sorted it based on color, size and weight. Further, this method also detected defective apples affected by stain, scab and rot from the images.

Inspired by the well-established demonstrations of machine vision based automatic grading and sorting systems for different kinds of fruits, we advocate a similar, but novel approach for tomatoes. Tomatoes are highly edible and vulnerable to damages when compared to citrus fruits and pomegranates. Classification of tomatoes with respect to quality and diseases is challenging and it involves a great degree of nonlinearity attributed to its physical characteristics. The differences between a ripe and unripe tomato are very subtle even to the Human Visual System (HVS). An accurate classification of ripe and unripe tomatoes with digital images is purely non-linear. Support Vector Machine (SVM) conceptualized by Cortes and Vapnik [10] has been successfully employed in the implementation of non-linear systems characterized by complex decision boundaries. Abe [1] demonstrated the effectiveness of SVM in various problems on pattern classification.

Hence, this paper employs a SVM classifier for sorting ripe and unripe fruits and a variant of SVM called Multiclass Support Vector Machine (MSVM) for defect detection. Elhariri et al. [12] have employed MSVM for determining the ripeness of tomatoes. Detailed experiments with tomato images, captured with an exclusively established image acquisition system and the performance metrics signify the efficacy of the proposed system in species discrimination, ripeness detection and disease classification.

The organization of the paper is as below. A detailed review on the literature covering classifiers for fruit sorting and grading is presented in Section 2. The underlying concepts in the deployment of the proposed system are discussed in Section 3. The proposed system and the workflow are described in Section 4. Quantitative results of the experimental works and interpretations are given in Section 5. The paper is concluded with the scope for further research, in Section 6.

Section snippets

Related works

Assurance of quality is vital for fruit consumption and export in the consumer-centric agriculture industry. Major agriculture industries resort to modern methods for enforcing quality standards of their deliverables, catering to local populations and international regulatory authorities. Unlike the conventional destructive sampling based methods, the new methods based on digital imaging are non-destructive and comparatively faster. This section presents a detailed review of various machine

Materials and methods

In this section, the experimental setup and the methodologies employed in deploying the proposed system are described. The system break-down structure consists of the image acquisition, image enhancement, feature prediction, segmentation and classification processes, detailed in the forthcoming subsections. The schematic diagram of the proposed system is given in Fig. 1.

Proposed system

Following the image acquisition process described in Section 3.1, the classification process is performed. This section presents the implementation of the proposed classification system, the workflow of which is shown in Fig. 4. The three stages of the proposed system are as described below.

  • i.

    Species discrimination: This process is performed initially to distinguish the tomatoes from other species. It is performed by employing the basic binary SVM classifier. Feature vectors called species

Experimental results and discussion

The experimental results obtained with our dataset are presented in this section. In this research, the ripeness levels of tomatoes are assessed and three disease types are identified. We have captured the images of 266 tomato fruits with including ripe, unripe, and black spot, canker and melanose infected fruits. Table 2, represents the two subsets of training and testing images are constructed from these images at the proportion of 50:50. The features apposite for each stage of the system are

Conclusion

In this paper we have presented a novel non-destructive system for detecting the ripeness levels and defects due to diseases in tomato fruits, using digital image processing techniques with minimal number of features compared to existing systems. The experimental works are conducted with exclusive image acquisition mechanism and software implementation of classifiers. The selection of color, shape and texture factors for construction of feature vectors in this research is bound by strong

Declaration of Competing Interest

No conflict of interest.

Acknowledgment

The authors wish to thank the Principal and the Management of PSG College of Technology for providing facilities to develop our research work in the Digital Signal Processing Laboratory of the department of Instrumentation and Control Systems Engineering. The authors wish to thank Tamil Nadu Agricultural University (TNAU) for providing samples to carry out the research work.

S. Dhakshina Kumar is working as an Assistant Professor in the Department of Electronics and Communication Engineering in University College of Engineering Ramanathapuram. He has 9 years of teaching experience. He did B.E. in Electronics and Communication Engineering from Panimalar Engineering College, Chennai and M.E. in Control Systems from PSG College of Technology Coimbatore. Currently, He is doing research in the field of Image processing.

References (54)

  • P. Moallem et al.

    Computer vision-based apple grading for golden delicious apples based on surface features

    Inf. Process. Agric.

    (2017)
  • G. Muhammad

    Date fruits classification using texture descriptors and shape-size features

    Eng. Appl. Artif. Intell.

    (2015)
  • L.M. Oo et al.

    A simple and efficient method for automatic strawberry shape and size estimation and classification

    Biosyst. Eng.

    (2018)
  • P.J. Ramos et al.

    Automatic fruit count on coffee branches using computer vision

    Comput. Electron. Agric.

    (2017)
  • F.S.A. Sa'ad et al.

    Shape and weight grading of mangoes using visible imaging

    Comput. Electron. Agric.

    (2015)
  • S. Sabzi et al.

    A new approach for visual identification of orange varieties using neural networks and metaheuristic algorithms

    Inf. Process. Agric.

    (2018)
  • M. Sharif et al.

    Detection and classification of citrus diseases in agriculture based on optimized weighted segmentation and feature selection

    Comput. Electron. Agric.

    (2018)
  • M.M. Sofu et al.

    Design of an automatic apple sorting system using machine vision

    Comput. Electron. Agric.

    (2016)
  • D.W. Sun

    Inspecting pizza topping percentage and distribution by a computer vision method

    J. Food Eng.

    (2000)
  • P. Wan et al.

    A methodology for fresh tomato maturity detection using computer vision

    Comput. Electron. Agric.

    (2018)
  • F. Wang et al.

    An automatic sorting system for fresh white button mushrooms based on image processing

    Comput. Electron. Agric.

    (2018)
  • E.H. Yossy et al.

    Mango fruit sortation system using neural network and computer vision

    Proc. Comput. Sci.

    (2017)
  • Y. Zhang et al.

    Non-destructive recognition and classification of citrus fruit blemishes based on ant colony optimized spectral information

    Postharvest Biol. Technol.

    (2018)
  • Y. Zhang et al.

    Fruit classification using computer vision and feedforward neural network

    J. Food Eng.

    (2014)
  • B. Zhu et al.

    Gabor feature-based apple quality inspection using kernel principal component analysis

    J. Food Eng.

    (2007)
  • S. Abe
    (2005)
  • H. Alimohamadi et al.

    Detecting skin defect of fruits using optimal Gabor wavelet filter

  • Cited by (81)

    View all citing articles on Scopus

    S. Dhakshina Kumar is working as an Assistant Professor in the Department of Electronics and Communication Engineering in University College of Engineering Ramanathapuram. He has 9 years of teaching experience. He did B.E. in Electronics and Communication Engineering from Panimalar Engineering College, Chennai and M.E. in Control Systems from PSG College of Technology Coimbatore. Currently, He is doing research in the field of Image processing.

    Dr. S. Esakkirajan is working as a Professor in PSG College OF Technology Coimbatore, in the Department of Instrumentation and Control Systems Engineering. He has 14 years of teaching experience. He did B.Tech. in Instrumentation from Cochin University of Science and Technology, M.E. in Applied Electronics from PSG College of Technology and Ph.D. in Information and Communication Engineering from Anna University, Chennai. He has published two books (i) Fundamentals of Relational Database Management Systems with Dr. S. Sumathi and (ii) digital Image Processing with Dr. S. Jayaraman and Dr. T. Veerakumar.

    Dr. S. Bama obtained her Bachelor degree from Institution of Engineers, India (2002), in the field of Electronics and Communication Engineering and M.E. in Communication Systems (2004) from Anna University Chennai. She obtained Ph.D. from Anna University, Chennai (2018) in the Faculty of Information and Communication Engineering with specialization in Image Processing. She is currently working as Associate Professor, in the Department of Electronics and Communication Engineering at Kalasalingam Academy of Research and Education, Krishnankoil, India. Her research interest includes developing algorithm in Medical Image Processing, and Applications of Deep Transfer Learning in the field of Medical Image Processing. She has around 13 years of Teaching Experience and 2 years in Industry.

    B. Keerthiveena is a research scholar in the Department of Instrumentation and Control Systems Engineering, PSG College of technology, Tamil Nadu, India. She received her B.E. degree in Electrical and Electronics Engineering and M.E. degree in Control Systems from Anna University, Tamil Nadu, India, in 2015 and 2017 respectively. Her research interest includes medical image analysis, optimization techniques and machine learning.

    View full text