Abstract
Purpose
Knee osteoarthritis (OA) is a common disease that impairs knee function and causes pain. Radiologists usually review knee X-ray images and grade the severity of the impairments according to the Kellgren–Lawrence grading scheme. However, this approach becomes inefficient in hospitals with high throughput as it is time-consuming, tedious and also subjective. This paper introduces a model for automatic diagnosis of knee OA based on an end-to-end deep learning method.
Method
In order to process the input images with location and classification simultaneously, we use Faster R-CNN as baseline, which consists of region proposal network (RPN) and Fast R-CNN. The RPN is trained to generate region proposals, which contain knee joint and then be used by Fast R-CNN for classification. Due to the localized classification via CNNs, the useless information in X-ray images can be filtered and we can extract clinically relevant features. For the further improvement in the model’s performance, we use a novel loss function whose weighting scheme allows us to address the class imbalance. Besides, larger anchors are used to overcome the problem that anchors don’t match the object when increasing the input size of X-ray images.
Result
The performance of the proposed model is thoroughly assessed using various measures. The results show that our adjusted model outperforms the Faster R-CNN, achieving a mean average precision nearly 0.82 with a sensitivity above 78% and a specificity above 94%. It takes 0.33 s to test each image, which achieves a better trade-off between accuracy and speed.
Conclusion
The proposed end-to-end fully automatic model which is computationally efficient has the potential to achieve the real automatic diagnosis of knee OA and be used as computer-aided diagnosis tools in clinical applications.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Zhang Y, Jordan JM (2010) Epidemiology of osteoarthritis. Clin Geriatr Med 26(3):355–369
Cross M, Smith E, Hoy D, Nolte S, Ackerman I, Fransen M, Bridgett L, Williams S, Guillemin F, Hill CL (2014) The global burden of hip and knee osteoarthritis: estimates from the global burden of disease 2010 study. Ann Rheum Dis 73(7):1323–1330
Karsdal MA, Michaelis M, Ladel C, Siebuhr AS, Bihlet A, Andersen JR, Guehring H, Christiansen C, Bayjensen AC, Kraus VB (2016) Disease-modifying treatments for osteoarthritis (DMOADs) of the knee and hip: lessons learned from failures and opportunities for the future. Osteoarthr Cartil 24(12):2013–2021
Braun HJ, Gold GE (2012) Diagnosis of osteoarthritis: imaging. Bone 51(2):278–288
Yoo TK, Kim DW, Choi SB, Park JS (2016) Simple scoring system and artificial neural network for knee osteoarthritis risk prediction: a cross-sectional study. PloS ONE 11(2):e0148724
Orlov N, Shamir L, Macura T, Johnston J, Eckley DM, Goldberg IG (2008) WND-CHARM: multi-purpose image classification using compound image transforms. Pattern Recognit Lett 29(11):1684–1693
Oka H, Muraki S, Akune T, Mabuchi A, Suzuki T, Yoshida H, Yamamoto S, Nakamura K, Yoshimura N, Kawaguchi H (2008) Fully automatic quantification of knee osteoarthritis severity on plain radiographs. Osteoarthr Cartil 16(11):1300–1306
Juefei-Xu F, Naresh Boddeti V , Savvides M (2017) Local binary convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 19–28
Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Laak JAVD, Ginneken BV, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88
Antony J, McGuinness K, O’Connor NE, Moran K (2016) Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks. In: 2016 23rd international conference on pattern recognition (ICPR). IEEE, pp 1195–1200
Antony J, McGuinness K, Moran K, O’Connor NE (2017) Automatic detection of knee joints and quantification of knee osteoarthritis severity using convolutional neural networks. In: International conference on machine learning and data mining in pattern recognition. Springer, pp 376–390
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440
Tiulpin A, Thevenot J, Rahtu E, Lehenkari P, Saarakkala S (2018) Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach. Sci Rep 8(1):1727
Abedin J, Antony J, McGuinness K, Moran K, O’Connor NE, Rebholz-Schuhmann D, Newell J (2019) Predicting knee osteoarthritis severity: comparative modeling based on patient’s data and plain X-ray images. Sci Rep 9(1):5761
Lin TY, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988
Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587
He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916
Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448
Neubeck A, Van Gool L (2006) Efficient non-maximum suppression. In: 18th international conference on pattern recognition (ICPR’06), vol 3. IEEE, pp 850–855
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788
Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263–7271
Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. arXiv preprint arXiv:1804.02767
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: single shot multibox detector. In: European conference on computer vision. Springer, pp 21–37
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Acknowledgements
This work is supported by the Scientific and Technological Innovation Action Plan of the Science and Technology Commission of Shanghai Municipality under Grant Number 19511121200.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liu, B., Luo, J. & Huang, H. Toward automatic quantification of knee osteoarthritis severity using improved Faster R-CNN. Int J CARS 15, 457–466 (2020). https://doi.org/10.1007/s11548-019-02096-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-019-02096-9