The blood smear analysis provides vital information and forms the basis to diagnose most of the diseases. With recent developments, deep learning methods can analyze the microscopic blood sample using image processing and classification tasks with less human effort and increased accuracy. In this work, embarking upon domain-specific feature extraction and active learning, we propose a compact, yet efficient feature fusion ensemble-based architecture for WBC sub-class classification and WBC disease identification which can automate and speed up the process of blood smear analysis with the help of digital image slide scanners.The proposed architecture is a three-stage multi-channel architecture with shallow feature inputs, deep feature extractors, and classification stages, respectively. The trainable parameters are quite less in our architecture when compared to deeper networks like ResNet 152, VGG19, which are the State of the Art (SOTA) methods [1], [2], [3]. However, labeling large medical data sets has been challenging and very costly. To mitigate huge labeling requirements and costs, Active Learning is employed to train this architecture and demonstrated much higher accuracy with quite less labeled data than SOTA. The proposed approach is shown to be quite general and yields better performance in terms of accuracy in WBC classification and Disease identification as well, with much fewer labeled samples for training, when compared with recent approaches employing deeper models.