The Drosophila embryonic gene expression images provide important spatio-temporal expression information for understanding the mechanisms of Drosophila embryogenesis. Automatic annotation of these images is an imperative but challenging task. Unlike the auto-annotation for nature images, the labels (terms from a controlled vocabulary) are assigned to genes rather than images. Each gene corresponds to a set of images, and different genes are associated with different numbers of images and labels, thus conventional machine learning methods are not applicable in such a scenario. In this study, we treat this task as a multi-instance multi-label (MIML) problem, and propose a hierarchical MIML learning framework, called HMIML. We implement HMIML at image-level and gene-level, respectively, both using convolutional neural networks. Especially, an image stitching strategy is employed to get a combined image representation at gene-level. Experimental results on the FlyExpress database show that HMIML enhances annotation accuracy on all developmental stages compared with the existing methods.