Automatic recognition of biological particles in microscopic images

https://doi.org/10.1016/j.patrec.2006.06.010Get rights and content

Abstract

A simple and general-purpose system to recognize biological particles is presented. It is composed of four stages: First (if necessary) promising locations in the image are detected and small regions containing interesting samples are extracted using a feature finder. Second, differential invariants of the brightness are computed at multiple scales of resolution. Third, after point-wise non-linear mappings to a higher dimensional feature space, this information is averaged over the whole region thus producing a vector of features for each sample that is invariant with respect to rotation and translation. Fourth, each sample is classified using a classifier obtained from a mixture-of-Gaussians generative model.

This system was developed to classify 12 categories of particles found in human urine; it achieves a 93.2% correct classification rate in this application. It was subsequently trained and tested on a challenging set of images of airborne pollen grains where it achieved an 83% correct classification rate for the three categories found during one month of observation. Pollen classification is challenging even for human experts and this performance is considered good.

Introduction

Microscopic image analysis is used in many fields of technology and physics. In a typical application the input image may have a resolution of 1024 × 1024 pixels, while objects of interest have a resolution of just 50 × 50 pixels. In all these applications, detection and classification are difficult because of poor resolution and strong variability of objects of interest, and because the background can be very noisy and highly variable. In this work, we aim to develop a general recognition system that is able to deal with the variability between the classes and also with the variability within each class. Moreover, our approach is segmentation free and allows easy addition of new classes.

Section 3 describes the two datasets used to train and test the system. The first is a collection of image patches centered around corpuscles found in microscopic urinalysis. These particles can be classified into 12 categories. The second dataset is a collection of airborne pollen images. In this dataset a recognition system has first to detect pollen grains, then, to classify them into their correct genus. We have 27 different pollen categories to detect, only the eight most numerous classes were then used to test the classifier. We considered this second dataset not only because automatic pollen recognition is still an unsolved problem and interesting application, but also because we wanted to assess the ability of the classifier to adapt to other classes of particles. In Section 4, we present a detector that is able to extract points of interest by looking at the differences of Gaussian images, and then to generate patches which might contain a particle of interest. In Section 5, we describe the key point of the system: a new set of very informative and straightforward features based on the local invariants defined by Mohr and Schmid (1997). A simple processing step then allows us to extract from the input image most of the information contained in the particle in the foreground, so that no segmentation is needed. Section 6 gives an overview at the mixture of Gaussians classifier we have used in our experiments.

In both datasets, we have to classify images with poor resolution; in addition, we are considering many different classes of biological particles that sometimes look very similar to each other. In addition to improving and automating the microscopic analysis of urine particles and the recognition of airborne pollen grains, the system we have developed can be applied to many other kinds of biological particles found in microscopic analysis. Indeed, this system has shown very good performance on a large variety of corpuscles as reported by several experiments in Section 7, and also by other recent experiments done on another set of images: a collection of small underwater animals.1

Section snippets

Previous work

The approach formulated by Dahmen et al. (2000) for the classification of red blood cells is general and segmentation-free, as is the system we are describing in this paper. They aim to classify three different kinds of red blood cells (namely, stomatocyte, echinocyte, discocyte) in a dataset of gray-scale images with resolution 128 × 128 pixels. Their idea of extracting features that are invariant with respect to shift, rotation and scale is powerful because it allows us to cluster cells that

Datasets

Our first dataset is a collection of particles found in microscopic urinalysis. Because of the technique used to acquire images in urinalysis, and the low concentration of particles in the specimen, the only challenging task in this application is classification. Thus the images in this dataset are already small image patches centered around a corpuscle. Some of these samples are shown in Fig. 1. They are (columns from left to right): bacteria, white blood cell clumps, yeasts, crystals, hyaline

Detection

Detection is the first stage of a recognition system. Given an input image with a lot of particles, it aims to find key points which have at least a little probability of being a particle of interest. Once one of these points is detected, a patch is centered on it, and then it is passed to the classifier in the next stage. It is extremely important to detect all the objects of interest especially when they are rarely found in images and their number is low, as typically happens in pollen

Feature extraction

In order to recognize an object in an image, it must be represented by some kind of features that should express the characteristics and the information contained in the image. These features should be invariant with respect to shift and rotation of the particle, and should be robust against small changes in scale as well. In this way, we are able to cluster images belonging to a certain class because we acquire the same features independent of the position and orientation of the cell. We

Classification

Given a patch with one centered object, a feature vector is extracted from the image. In training, the system learns how to distinguish among features of images belonging to different categories. In the test phase, a decision is made according to the test image feature. If the object in the image is considered a particle of interest, then its class is also identified; otherwise, it is discarded because the feature does not match any training model.

We use a Bayesian classifier that models the

Detection

Detection is performed only on images of the pollen dataset (see Fig. 2). We evaluate the performance by measuring how well our automatic detector agrees with the set of reference labels. A detection occurs if the algorithm indicates the presence of an object at a location where a pollen exists according to the reference list. Similarly, a false alarm occurs if the algorithm indicates the presence of an object at a location where no pollen exists according to the reference list. We say that

Summary and conclusions

Microscopic analysis of biological particles aims to find and to classify particles. These objects often have variable shape and texture, and very low contrast and resolution. Moreover, the background can be highly variable as well. Recognition on these images is very challenging.

In this work, we have defined a new kind of feature based on “local jets” (Mohr and Schmid, 1997). These are able to extract information from a patch centered on the object of interest without any segmentation.

Acknowledgements

The authors are very grateful to C. Poultney and H.M. Kim for their precious comments.

Research described in this article was supported by Philip Morris USA Inc., Philip Morris International and the Southern California Environmental Health Sciences Center (NIEHS grant number 5P30 ES07048). P. Taylor gratefully acknowledges support from a Boswell Fellowship, which is a joint position intended to foster cooperation between the California Institute of Technology and the Huntington Medical Research

References (14)

There are more references available in the full text version of this article.

Cited by (103)

  • A review on various methods for recognition of urine particles using digital microscopic images of urine sediments

    2021, Biomedical Signal Processing and Control
    Citation Excerpt :

    It included 10 articles based on machine learning techniques for automated urinalysis as follows. Ranzato et al. [16] developed a system for identifying biological particles in microscopic images. The system has four stages.

  • Image analysis-based quantification of fungal sporulation by automatic conidia counting and gray value correlation

    2021, MethodsX
    Citation Excerpt :

    The present method can be performed with low cost, conventional laboratory equipment and the open-source software ImageJ that enables the adaptability of the method by other scientists. Furthermore, the developed method could be combined with established methods for shape and pattern recognition, e.g. those presented by Nielsen et al. [6] or Ranzato et al. [10] and might be adapted for other fungi or microscopic particles as well. It will also prove useful to quantify spores produced in solid state fermentation on technical scale and provide a reliable option for quality control of these bioprocesses.

View all citing articles on Scopus
View full text