Prostate cancer is the second most common cancer among men worldwide. For prognosis and treatment, pathologists assign International Society of Urological Pathology (ISUP) grades to Whole Slide Imaging (WSI) of sampled prostate tissues to express the severity of the cancer. The manual approach suffers from human error and person-to-person variations. Existing approaches that use machine learning for automatic grading of WSI images includes using expensive datasets in which different regions of the slide are annotated by pathologists to show different levels of cancer. However, most of the existing realworld datasets contain weak labels; i.e. each slide is labeled with just one grade number. In this work, we present a self-supervised learning method to automatically grade prostate slides using datasets of weakly labeled slides. Our algorithm achieves a 5-way classification accuracy of 54.1% which which surpasses the performance of 41.5% using state of the art multiple-instance labeling methods.