3D Orientation Estimation of Pharmaceutical Minitablets with Convolutional Neural Network

Abstract

We present a Convolutional Neural Network for 3D orientation estimation of pharmaceutical minitablets, i.e., round tablets with diameter less than 3 mm. The network inputs a single grayscale image with the minitablet positioned approximately in the center and predicts a 3D unit orientation vector that fully describes the 3D orientation of the imaged minitablet. We trained the network on synthetic images, generated by rendering CAD models of minitablets at realistic conditions by varying the orientation, scale, camera distance, position within the imaging plane, and surface properties. No manual 3D orientation labeling of training images was therefore required. We evaluated the accuracy of the approach on both synthetic and real images. The real images were acquired during pharmaceutical film coating processes. Accuracies of ${1.388}^{\circ }$ and ${2.657}^{\circ }$ were achieved on synthetic and real image datasets, respectively. We tested two different minitablet shapes. Obtained results indicate that good performance can be obtained on a real image datasets despite training the network on synthetic data only. The estimated 3D orientations provide means for further automated analysis of the images, which we demonstrated by measuring an important coating process parameter (coating thickness) during the minitablet coating process. Although tested only for minitablets, the 3D orientation estimation approach should perform well also for other symmetrical shapes.