Verification of dispensed pharmaceutical packages is of paramount importance to prescription dispensing. Due to lack of identification peripherals like bar codes or RFID tags on blister packages, image-based solutions have been utilized. Earlier two-stage solutions require more resources for implementation and training, in addition to more computational time. In contrast, this paper presents an end-to-end trained solution, called Fast Rotated Occluded Rectangular (Fast ROR) pattern recognition architecture, composed of modules of: rotational rectangular detection, affine transformation, and image recognition. In particular, the features used to localize the package and the features used for identification are the same and extracted by a common feature extractor. As a result, the overall architecture is more compact with only one training set needed and more efficient computation time. Comparison experiments have been conducted on the proposed end-to-end FOR and on a representative two-stage HBIN [1] solution: Targeting a pool of 230 types of pharmaceutical packages, 30 paired front and back handheld images for each type were taken and randomly partitioned with 4:1 for training and testing, FOR (vs. HBIN) uses 41.79M (120.32M) network parameters, with a training time of 17 hours (119 hours), and a testing speed of 22.2 fps (10fps). The identification results in terms of F1-score by FOR (vs. HBIN) is 100% (98.67%) in familiar environment, whereas 94.30% (91.27%) when the identification is conducted in new environment.