We present a method for recognition of agglomerates in images acquired during the coating process of pharmaceutical pellets. The pellets in the images are not perfectly dispersed and it is often hard to differentiate between a random group of primary particles and a real agglomerate. The method utilizes a clustering-based image segmentation for candidate region detection and a convolutional neural network for classification of detected pellets to primary particles or agglomerates. We validated the performance of the method on real images of pharmaceutical pellets acquired during the coating process and achieved 93 % classification accuracy.