Automated atlas-based segmentation of NISSL-stained mouse brain sections using supervised learning

Abstract

The problem of segmentation of mouse brain images into anatomical structures is an important stage of practically every analytical procedure for these images. The present study suggests a new approach to automated segmentation of anatomical structures in the images of NISSL-stained histological sections of mouse brain. The segmentation algorithm is based on the method of supervised learning using the existing anatomical labeling of the corresponding sections from a specialized mouse brain atlas. A mouse brain section to be segmented into anatomical structures is preliminarily associated with a section from the mouse brain atlas displaying the maximum similarity. The image of this section is then preprocessed in order to enhance its quality and to make it as close to the corresponding atlas image as possible. An efficient algorithm of luminance equalization, an extension of the well-known Retinex algorithm is proposed. A random forest is trained on pixel feature vectors constructed based on the atlas section images and the corresponding class labels associated with anatomical structures extracted from the atlas anatomical labeling. The trained classifier is then applied to classify pixels of an experimental section into anatomical structures. A new combination of features based on superpixels and location priors is suggested. Accuracy of the obtained result is increased by using Markov random field. Procedures of luminance equalization and subsequent segmentation into anatomical structures have been tested on real experimental sections.