In the clinically widely used rating scale (MDS-UPDRS), the pronation-supination movement task of hands is required for assessment of bradykinesia, which is a typical clinical symptom of Parkinson’s disease (PD). Due to inter-rater variability in the task rating process, objective automated rating models are critically needed. Still, the performance of such models would be limited if prior knowledge of the clinical rating principles is not adequately accounted for. Therefore, we propose a clinically guided method which fully exploits the MDS-UPDRS rating principles to achieve consistent and accurate automated rating. First, a multi-scale framework which employs two graph convolutional networks (GCNs) as two streams is developed to extract transient and persistent features related to these rating principles. In particular, abnormal transient features are detected through a specialized multiple-instance-learning GCN. Moreover, the multiple-instance-learning GCN is equipped with an accumulation-aware ordered multiple-instance pooling module, which estimates sample-level severity by accounting for both the intra-instance intrinsic severity order and the inter-instance accumulation effects. Besides, an instance context encoding module is designed to combine the phase information in the pronation-supination movement cycles with instances’ motion features. This facilitates the differentiation between PD-induced halts and natural periodic halts. Our method demonstrated excellent performance on both a large clinical dataset and an additional independent test dataset. Our proposed scheme only requires consumer-level cameras, and therefore exhibits high potential for large-scale applications in PD telemedicine.