Abstract
Knee osteoarthritis is a major diarthrodial joint disorder with profound global socioeconomic impact. Diagnostic imaging using magnetic resonance image can produce morphometric biomarkers to investigate the epidemiology of knee osteoarthritis in clinical trials, which is critical to attain early detection and develop effective regenerative treatment/therapy. With tremendous increase in image data size, manual segmentation as the standard practice becomes largely unsuitable. This review aims to provide an in-depth insight about a broad collection of classical and deep learning segmentation techniques used in knee osteoarthritis research. Specifically, this is the first review that covers both bone and cartilage segmentation models in recognition that knee osteoarthritis is a “whole joint” disease, as well as highlights on diagnostic values of deep learning in emerging knee osteoarthritis research. Besides, we have collected useful deep learning reviews to serve as source of reference to ease future development of deep learning models in this field. Lastly, we highlight on the diagnostic value of deep learning as key future computer-aided diagnosis applications to conclude this review.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ababneh SY, Prescott JW, Gurcan MN (2011) Automatic graph-cut based segmentation of bones from knee magnetic resonance images for osteoarthritis research. Med Image Anal 15:438–448. https://doi.org/10.1016/j.media.2011.01.007
Abdelsamea MM, Gnecco G, Gaber MM, Elyan E (2015) On the relationship between variational level set-based and SOM-based active contours. Comput Intell Neurosci 2015:109029. https://doi.org/10.1155/2015/109029
Adams R, Bischof L (1994) Seeded region growing. IEEE Trans Pattern Anal Mach Intell 16:641–647. https://doi.org/10.1109/34.295913
Ahmet S, Songül A (2020) Knee meniscus segmentation and tear detection from MRI: a review current medical. Imaging 16:2–15. https://doi.org/10.2174/1573405614666181017122109
Ahn C, Bui TD, Lee Y-W, Shin J, Park H (2016) Fully automated, level set-based segmentation for knee MRIs using an adaptive force function and template: data from the osteoarthritis initiative. BioMed Eng 15:99. https://doi.org/10.1186/s12938-016-0225-7
Akhtar S, Poh CL, Kitney RI (2007) An MRI derived articular cartilage visualization framework. Osteoarthr Cartil 15:1070–1085. https://doi.org/10.1016/j.joca.2007.03.009
Almajalid R, Shan J, Du Y, Zhang M (2019a) Identification of knee cartilage changing pattern. Appl Sci 9:1–14
Almajalid R, Shan J, Zhang M, Stonis G, Zhang M (2019b) Knee bone segmentation on three-dimensional MRI. In: IEEE 18th international conference on machine learning and applications (ICMLA), 16–19 Dec. 2019, pp 1725–1730. https://doi.org/10.1109/ICMLA.2019.00280
Ambellan F, Tack A, Ehlke M, Zachow S (2019) Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: data from the osteoarthritis initiative. Med Image Anal 52:109–118. https://doi.org/10.1016/j.media.2018.11.009
Antony J, McGuinness K, Moran K, O’Connor N Automatic detection of knee joints and quantification of knee osteoarthritis severity using convolutional neural networks. In: International conference on machine learning and data mining in pattern recognition, 2017. Lecture Notes in Computer Science. Springer, Cham. https://doi.org/10.1007/978-3-319-62416-7_27
Aprovitola A, Gallo L (2016) Knee bone segmentation from MRI: a classification and literature review. Biocybern Biomed Eng 36:437–449. https://doi.org/10.1016/j.bbe.2015.12.007
Ashinsky BG et al (2015) Machine learning classification of OARSI-scored human articular cartilage using magnetic resonance imaging. Osteoarthr Cartil 23:1704–1712. https://doi.org/10.1016/j.joca.2015.05.028
Ashinsky BG et al (2017) Predicting early symptomatic osteoarthritis in the human knee using machine learning classification of magnetic resonance images from the osteoarthritis initiative. J Orthop Res 35:2243–2250. https://doi.org/10.1002/jor.23519
Bae KT, Shim H, Tao C, Chang S, Wang JH, Boudreau R, Kwoh CK (2009) Intra- and inter-observer reproducibility of volume measurement of knee cartilage segmented from the OAI MR image set using a novel semi-automated segmentation method. Osteoarthr Cartil 17:1589–1597. https://doi.org/10.1016/j.joca.2009.06.003
Barr AJ, Campbell TM, Hopkinson D, Kingsbury SR, Bowes MA, Conaghan PG (2015) A systematic review of the relationship between subchondral bone features, pain and structural pathology in peripheral joint osteoarthritis. Arthritis Res Ther 17:228. https://doi.org/10.1186/s13075-015-0735-x
Bien N et al (2018) Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet. PLOS Med 15:e1002699. https://doi.org/10.1371/journal.pmed.1002699
Bonaretti S, Gold GE, Beaupre GS (2020) pyKNEEr: an image analysis workflow for open and reproducible research on femoral knee cartilage. PLoS ONE 15:1–19. https://doi.org/10.1371/journal.pone.0226501
Bourgeat P, Fripp J, Stanwell P, Ramadan S, Ourselin S (2007) MR image segmentation of the knee bone using phase information. Med Image Anal 11:325–335. https://doi.org/10.1016/j.media.2007.03.003
Bowers ME, Trinh N, Tung GA, Crisco JJ, Kimia BB, Fleming BC (2008) Quantitative MR imaging using “LiveWire” to measure tibiofemoral articular cartilage thickness. Osteoarthr Cartil 16:1167–1173. https://doi.org/10.1016/j.joca.2008.03.005
Boykov Y, Funka-Lea G (2006) Graph cuts and efficient N-D image segmentation. Int J Comput Vision 70:109–131. https://doi.org/10.1007/s11263-006-7934-5
Boykov YY, Jolly M (2001) Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In: Proceedings eighth IEEE international conference on computer vision. ICCV 2001, 7–14 July. vol 101, pp 105–112. https://doi.org/10.1109/ICCV.2001.937505
Brem MH et al (2009) Magnetic resonance image segmentation using semi-automated software for quantification of knee articular cartilage—initial evaluation of a technique for paired scans. Skelet Radiol 38:505–511. https://doi.org/10.1007/s00256-009-0658-1
Carballido-Gamio J, Majumdar S (2011) Atlas-based knee cartilage assessment. Magn Reson Med 66:575–581. https://doi.org/10.1002/mrm.22836
Carballido-Gamio J, Bauer JS, Keh-Yang L, Krause S, Majumdar S (2005) Combined image processing techniques for characterization of MRI cartilage of the knee. In: IEEE engineering in medicine and biology 27th annual conference, 17–18 Jan. 2006, pp 3043–3046. https://doi.org/10.1109/IEMBS.2005.1617116
Caselles V, Catté F, Coll T, Dibos F (1993) A geometric model for active contours in image processing. Numer Math 66:1–31. https://doi.org/10.1007/BF01385685
Caselles V, Kimmel R, Sapiro G (1997) Geodesic active contours. Int J Comput Vis 22:61–79. https://doi.org/10.1023/A:1007979827043
Cashman PMM, Kitney RI, Gariba MA, Carter ME (2002) Automated techniques for visualization and mapping of articular cartilage in MR images of the osteoarthritic knee: a base technique for the assessment of microdamage and submicro damage. IEEE Trans Nanobiosci 99:42–51. https://doi.org/10.1109/TNB.2002.806916
Chang GH, Felson DT, Qiu S, Capellini TD, Kolachalama VB (2018) Predicting bilateral knee pain from MR imaging using deep neural networks. bioRxiv:463497 https://doi.org/10.1101/463497
Charlier E et al (2016) Insights on molecular mechanisms of chondrocytes death in osteoarthritis. Int J Mol Sci 17:2146. https://doi.org/10.3390/ijms17122146
Cheng R et al (2020) Fully automated patellofemoral MRI segmentation using holistically nested networks: implications for evaluating patellofemoral osteoarthritis, pain, injury, pathology, and adolescent development. Magn Reson Med 83:139–153. https://doi.org/10.1002/mrm.27920
Cohen LD (1991) On active contour models and balloons. CVGIP: Image Underst 53:211–218. https://doi.org/10.1016/1049-9660(91)90028-N
Collins JE et al (2016) Semiquantitative imaging biomarkers of knee osteoarthritis progression: data from the foundation for the national institutes of health osteoarthritis biomarkers consortium. Arthritis Rheumatol 68:2422–2431. https://doi.org/10.1002/art.39731
Cootes TF, Taylor CJ (1992) Active shape models—‘Smart Snakes’. In: Hogg D, Boyle R (eds) BMVC92, London. Springer, London, pp 266–275
Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23:681–685. https://doi.org/10.1109/34.927467
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297. https://doi.org/10.1007/BF00994018
Dalvi R, Abugharbieh R, Wilson D, Wilson DR (2007) Multi-contrast MR for enhanced bone imaging and segmentation. In: IEEE 29th engineering in medicine and biology society, Lyon, France, 22–26 Aug. 2007. IEEE, pp 5620–5623. https://doi.org/10.1109/IEMBS.2007.4353621
Dam E, Lillholm M, Marques J, Nielsen M (2015) Automatic segmentation of high- and low-field knee MRIs using knee image quantification with data from the osteoarthritis initiative. J Med Imaging 2:024001
Dargan S, Kumar M, Ayyagari MR, Kumar G (2019) A survey of deep learning and its applications: a new paradigm to machine learning. Arch Comput Methods Eng. https://doi.org/10.1007/s11831-019-09344-w
Davies-Tuck ML et al (2010) Development of bone marrow lesions is associated with adverse effects on knee cartilage while resolution is associated with improvement—a potential target for prevention of knee osteoarthritis: a longitudinal study. Arthritis Res Therapy 12:1–10. https://doi.org/10.1186/ar2911
Dodin P, Pelletier J, Martel-Pelletier J, Abram F (2010) Automatic human knee cartilage segmentation from 3-D magnetic resonance images. IEEE Trans Biomed Eng 57:2699–2711. https://doi.org/10.1109/TBME.2010.2058112
Dodin P, Martel-Pelletier J, Pelletier J-P, Abram F (2011) A fully automated human knee 3D MRI bone segmentation using the ray casting technique. Med Biol Eng Comput 49:1413–1424. https://doi.org/10.1007/s11517-011-0838-8
Duryea J et al (2007) Novel fast semi-automated software to segment cartilage for knee MR acquisitions. Osteoarthr Cartil 15:487–492. https://doi.org/10.1016/j.joca.2006.11.002
Duryea J et al (2014) Local area cartilage segmentation: a semiautomated novel method of measuring cartilage loss in knee osteoarthritis. Arthritis Care Res 66:1560–1565. https://doi.org/10.1002/acr.22332
Duryea J, Cheng C, Schaefer LF, Smith S, Madore B (2016) Integration of accelerated MRI and post-processing software: a promising method for studies of knee osteoarthritis. Osteoarthr Cartil 24:1905–1909. https://doi.org/10.1016/j.joca.2016.06.001
Ebrahimkhani S, Jaward MH, Cicuttini FM, Dharmaratne A, Wang Y, de Herrera AGS (2020) A review on segmentation of knee articular cartilage: from conventional methods towards deep learning. Artif Intell Med. https://doi.org/10.1016/j.artmed.2020.101851
Eckstein F, Peterfy C (2016) A 20 years of progress and future of quantitative magnetic resonance imaging (qMRI) of cartilage and articular tissues—personal perspective. Seminars Arthritis Rheumatism 45:639–647. https://doi.org/10.1016/j.semarthrit.2015.11.005
Eckstein F, Wirth W (2011) Quantitative cartilage imaging in knee osteoarthritis. Arthritis 2011:1–19. https://doi.org/10.1155/2011/475684
Eckstein F et al (2006) Proposal for a nomenclature for Magnetic Resonance Imaging based measures of articular cartilage in osteoarthritis. Osteoarthr Cartil 14:974–983. https://doi.org/10.1016/j.joca.2006.03.005
Eckstein F et al (2015) Brief report: cartilage thickness change as an imaging biomarker of knee osteoarthritis progression: data from the foundation for the national institutes of health osteoarthritis biomarkers consortium. Arthritis Rheumatol 67:3184–3189. https://doi.org/10.1002/art.39324
Englund M (2010) The role of biomechanics in the initiation and progression of OA of the knee Best Practice & Research. Clin Rheumatol 24:39–46. https://doi.org/10.1016/j.berh.2009.08.008
Fabian B, Tiziano R, Pletscher M (2015) Distal femur segmentation on MR images using random forests. In: Medical Image Analysis Laboratory. pp 1–6
Falcao AX, Udupa JK, Miyazawa FK (2000) An ultra-fast user-steered image segmentation paradigm: live wire on the fly. IEEE Trans Med Imaging 19:55–62. https://doi.org/10.1109/42.832960
Favero M, Ramonda R, Goldring MB, Goldring SR, Punzi L (2015) Early knee osteoarthritis. RMD Open 1:e000062. https://doi.org/10.1136/rmdopen-2015-000062
Folkesson J, Dam EB, Olsen OF, Pettersen PC, Christiansen C (2007) Segmenting articular cartilage automatically using a voxel classification approach. IEEE Trans Med Imaging 26:106–115. https://doi.org/10.1109/TMI.2006.886808
Fripp J, Crozier S, Warfield SK, Ourselin S (2007) Automatic segmentation of the bone and extraction of the bone–cartilage interface from magnetic resonance images of the knee. Phys Med Biol 52:1617–1631. https://doi.org/10.1088/0031-9155/52/6/005
Gan H-S, Sayuti K (2016) Comparison of improved semi-automated segmentation technique with manual segmentation: data from the osteoarthritis initiative. Am J Appl Sci 13:1068–1075. https://doi.org/10.3844/ajassp.2016.1068.1075
Gan H-S, Tan T-S, Wong L-X, Tham W-K, Sayuti KA, Abdul Karim AH, bin Abdul Kadir MR (2014a) Interactive knee cartilage extraction using efficient segmentation software: data from the osteoarthritis initiative. Bio-Med Mater Eng 24:3145–3157. https://doi.org/10.3233/BME-141137
Gan HS, Tan T, Karim AHA, Sayuti KA, Kadir MRA (2014b) Interactive medical image segmentation with seed precomputation system: data from the osteoarthritis initiative. In: IEEE conference on biomedical engineering and sciences (IECBES), 8–10 Dec 2014. pp 315-318. https://doi.org/10.1109/IECBES.2014.7047510
Gan HS, Tan T, Karim AHA, Sayuti KA, Kadir MRA (2014c) Multilabel graph based approach for knee cartilage segmentation: Data from the osteoarthritis initiative. In: IEEE conference on biomedical engineering and sciences (IECBES), 8–10 Dec 2014, pp 210–213. https://doi.org/10.1109/IECBES.2014.7047487
Gan H-S, Karim AHA, Sayuti KA, Tan T-S, Kadir MRA (2016) Analysis of parameters’ effects in semi-automated knee cartilage segmentation model: data from the osteoarthritis initiative. AIP Conf Proc 1775:030052. https://doi.org/10.1063/1.4965172
Gan H-S, Sayuti KA, Karim AHA (2017) Investigation of random walks knee cartilage segmentation model using inter-observer reproducibility: data from the osteoarthritis initiative. Bio-Med Mater Eng 28:75–85. https://doi.org/10.3233/BME-171658
Gan H, Rosidi RM, Hamidur H, Sayuti KA, Ramlee MH, Karim AHA, Salam BAA (2018) Binary seeds auto generation model for knee cartilage segmentation. In: International conference on intelligent and advanced system (ICIAS), 13–14 Aug. 2018, pp 1–5. https://doi.org/10.1109/ICIAS.2018.8540570
Gan H-S, Sayuti KA, Ramlee MH, Lee Y-S, Wan Mahmud WMH, Abdul Karim AH (2019) Unifying the seeds auto-generation (SAGE) with knee cartilage segmentation framework: data from the osteoarthritis initiative. Int J Comput Assist Radiol Surg 14:755–762. https://doi.org/10.1007/s11548-019-01936-y
Gandhamal A, Talbar S, Gajre S, Razak R, Hani AFM, Kumar D (2017) Fully automated subchondral bone segmentation from knee MR images: data from the osteoarthritis Initiative. Comput Biol Med 88:110–125. https://doi.org/10.1016/j.compbiomed.2017.07.008
Goceri E (2018) Formulas behind deep learning success. In: International conference on applied analysis and mathematical modeling, Istanbul, Turkey, 2018. p 156
Goceri E (2019) Challenges and recent solutions for image segmentation in the era of deep learning. In: 2019 ninth international conference on image processing theory, tools and applications (IPTA), 6–9 Nov 2019. pp 1–6. https://doi.org/10.1109/IPTA.2019.8936087
Goceri E, Goceri N (2017) Deep learning in medical image analysis: recent advances and future trends. In: 11th international conference on computer graphics. Visualization, computer vision and image processing, Lisbon, Portugal, 2017, pp 305–310
Goldring MB, Goldring SR (2010) Articular cartilage and subchondral bone in the pathogenesis of osteoarthritis. Ann N Y Acad Sci 1192:230–237. https://doi.org/10.1111/j.1749-6632.2009.05240.x
González G, Escalante-Ramírez B (2013) Knee cartilage segmentation using active shape models and contrast enhancement from magnetic resonance images vol 8922. IX International seminar on medical information processing and analysis. SPIE
González G, Escalante-Ramírez B (2014) Knee cartilage segmentation using active shape models and local binary patterns, vol 9138. SPIE Photonics Europe. SPIE
Górriz M, Antony J, McGuinness K, Giró-i-Nieto X, O’Connor NE (2019) Assessing knee OA severity with CNN attention-based end-to-end architectures. Paper presented at the Proceedings of The 2nd international conference on medical imaging with deep learning
Gougoutas AJ, Wheaton AJ, Borthakur A, Shapiro EM, Kneeland JB, Udupa JK, Reddy R (2004) Cartilage volume quantification via live wire segmentation. Acad Radiol 11:1389–1395. https://doi.org/10.1016/j.acra.2004.09.003
Grady L (2006) Random walks for image segmentation. IEEE Trans Pattern Anal Mach Intell 28:1768–1783. https://doi.org/10.1109/TPAMI.2006.233
Greenspan H, Bv G, Summers RM (2016) Guest Editorial Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique. IEEE Trans Med Imaging. 35:1153–1159. https://doi.org/10.1109/TMI.2016.2553401
Guo Y, Jiang J, Hao S, Zhan S (2011) Distribution-based active contour model for medical image segmentation. In: International conference on image and graphics, 12–15 Aug 2011, pp 61–65. https://doi.org/10.1109/ICIG.2011.11
Hafezi-Nejad N et al (2017) Prediction of medial tibiofemoral compartment joint space loss progression using volumetric cartilage measurements: data from the FNIH OA biomarkers consortium. Eur Radiol 27:464–473. https://doi.org/10.1007/s00330-016-4393-4
Heimann T, Meinzer H-P (2009) Statistical shape models for 3D medical image segmentation: a review. Med Image Anal 13:543–563. https://doi.org/10.1016/j.media.2009.05.004
Heimann T, Morrison BJ, Styner MA, Niethammer M, Warfield SK (2010) Segmentation of knee images: a grand challenge. In: Proceedings MICCAI workshop on medical image analysis for the clinic, 2010. pp 207–214
Hesamian MH, Jia W, He X, Kennedy P (2019) Deep learning techniques for medical image segmentation: achievements and challenges. J Digit Imaging 32:582–596. https://doi.org/10.1007/s10278-019-00227-x
Hiligsmann M et al (2013) Health economics in the field of osteoarthritis: An Expert's consensus paper from the European Society for Clinical and Economic Aspects of Osteoporosis and Osteoarthritis (ESCEO) Seminars in Arthritis and Rheumatism 43:303–313. https://doi.org/10.1016/j.semarthrit.2013.07.003
Hunter DJ et al (2006) Increase in bone marrow lesions associated with cartilage loss: a longitudinal magnetic resonance imaging study of knee osteoarthritis. Arthritis Rheum 54:1529–1535. https://doi.org/10.1002/art.21789
Iranpour-Boroujeni T, Watanabe A, Bashtar R, Yoshioka H, Duryea J (2011) Quantification of cartilage loss in local regions of knee joints using semi-automated-segmentation software: analysis of longitudinal data from the Osteoarthritis Initiative (OAI). Osteoarthr Cartil 19:309–314. https://doi.org/10.1016/j.joca.2010.12.002
Jianbo S, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22:888–905. https://doi.org/10.1109/34.868688
Jolliffe IT (2002) Principal component analysis. Springer series in statistics, vol 2. Springer, New York
Kapoor M, Martel-Pelletier J, Lajeunesse D, Pelletier J-P, Fahmi H (2011) Role of proinflammatory cytokines in the pathophysiology of osteoarthritis. Nat Rev Rheumatol 7:33–42. https://doi.org/10.1038/nrrheum.2010.196
Kashyap S, Oguz I, Zhang H, Sonka M (2016) Automated segmentation of knee MRI using hierarchical classifiers and just enough interaction based learning: data from osteoarthritis initiative. In: Medical image computing and computer assisted intervention—MICCAI 2016, Athens, Greece, 2016. Springer, Berlin, pp 344–351. https://doi.org/10.1007/978-3-319-46723-8_40
Kashyap S, Zhang H, Rao K, Sonka M (2018) Learning-based cost functions for 3-D and 4-D multi-surface multi-object segmentation of knee MRI: data from the osteoarthritis initiative. IEEE Trans Med Imaging 37:1103–1113. https://doi.org/10.1109/TMI.2017.2781541
Kass M, Witkin A, Terzopoulos D (1988) Snakes: Active contour models. Int J Comput Vis 1:321–331. https://doi.org/10.1007/BF00133570
Khan A, Sohail A, Zahoora U, Qureshi AS (2020) A survey of the recent architectures of deep convolutional neural networks. Artif Intell Rev. https://doi.org/10.1007/s10462-020-09825-6
Kumar D, Gandhamal A, Talbar S, Hani AFM (2018) Knee articular cartilage segmentation from MR images: a review. ACM Comput Surv. https://doi.org/10.1145/3230631
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444. https://doi.org/10.1038/nature14539
Lee J-S, Chung Y-N (2005) Integrating edge detection and thresholding approaches to segmenting femora and patellae from magnetic resonance images. Biomed Eng Appl Basis Commun 17:1–11. https://doi.org/10.4015/S1016237205000020
Lee S, Park SH, Shim H, Yun ID, Lee SU (2011) Optimization of local shape and appearance probabilities for segmentation of knee cartilage in 3-D MR images. Comput Vis Image Understanding 115:1710–1720. https://doi.org/10.1016/j.cviu.2011.05.014
Lee J-G, Gumus S, Moon CH, Kwoh CK, Bae KT (2014) Fully automated segmentation of cartilage from the MR images of knee using a multi-atlas and local structural analysis method. Med Phys 41:092303. https://doi.org/10.1118/1.4893533
Lee H, Hong H, Kim J (2018) BCD-NET: a novel method for cartilage segmentation of knee MRI via deep segmentation networks with bone-cartilage-complex modeling. In: IEEE 15th international symposium on biomedical imaging (ISBI 2018), 4–7 April 2018, pp 1538–1541. https://doi.org/10.1109/ISBI.2018.8363866
Li G, Yin J, Gao J, Cheng TS, Pavlos NJ, Zhang C, Zheng MH (2013) Subchondral bone in osteoarthritis: insight into risk factors and microstructural changes. Arthritis Res Ther 15:223. https://doi.org/10.1186/ar4405
Lim J, Kim J, Cheon S (2019) A deep neural network-based method for early detection of osteoarthritis using statistical data. Int J Environ Res Public Health 16:1281. https://doi.org/10.3390/ijerph16071281
Litjens G et al (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88. https://doi.org/10.1016/j.media.2017.07.005
Liu Q, Wang Q, Zhang L, Gao Y, Shen D Multi-atlas context forests for knee MR Image segmentation. In: International workshop on machine learning in medical imaging, Munich, Germany, 2015. Springer International Publishing, pp 186–193
Liu F, Zhou Z, Jang H, Samsonov A, Zhao G, Kijowski R (2018) Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging. Magn Reson Med 79:2379–2391. https://doi.org/10.1002/mrm.26841
Liu F et al (2018) Deep learning approach for evaluating knee MR images: achieving high diagnostic performance for cartilage lesion detection. Radiology 289:160–169. https://doi.org/10.1148/radiol.2018172986
Liukkonen MK, Mononen ME, Klets O, Arokoski JP, Saarakkala S, Korhonen RK (2017) Simulation of subject-specific progression of knee osteoarthritis and comparison to experimental follow-up data: data from the osteoarthritis initiative. Sci Rep 7:9177. https://doi.org/10.1038/s41598-017-09013-7
Liukkonen MK, Mononen ME, Tanska P, Saarakkala S, Nieminen MT, Korhonen RK (2017) Application of a semi-automatic cartilage segmentation method for biomechanical modeling of the knee joint. Comput Methods Biomech Biomed Eng 20:1453–1463. https://doi.org/10.1080/10255842.2017.1375477
Loeser RF, Goldring SR, Scanzello CR, Goldring MB (2012) Osteoarthritis: a disease of the joint as an organ. Arthritis Rheum 64:1697–1707. https://doi.org/10.1002/art.34453
Lorigo LM, Faugeras O, Grimson WEL, Keriven R, Kikinis R (1998) Segmentation of bone in clinical knee MRI using texture-based geodesic active contours. In: Medical image computing and computer assisted intervention—MICCAI 1998. Springer, Berlin, pp 1195–1204
Lundervold AS, Lundervold A (2019) An overview of deep learning in medical imaging focusing on MRI. Zeitschrift für Medizinische Physik 29:102–127. https://doi.org/10.1016/j.zemedi.2018.11.002
Lynch J, Zaim S, Zhao J, Stork A, Peterfy C, Genant H (2000) Cartilage segmentation of 3D MRI scans of the osteoarthritic knee combining user knowledge and active contours vol 3979. Medical Imaging 2000. SPIE
MacQueen JB (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of the fifth Berkeley symposium on mathematical statistics and probability. California: University of California Press, pp 281–297
Maier A, Syben C, Lasser T, Riess C (2019) A gentle introduction to deep learning in medical image processing. Zeitschrift für Medizinische Physik 29:86–101. https://doi.org/10.1016/j.zemedi.2018.12.003
Maldonado M, Nam J (2013) The role of changes in extracellular matrix of cartilage in the presence of inflammation on the pathology of osteoarthritis. Biomed Res Int 2013:284873–284873. https://doi.org/10.1155/2013/284873
Malladi R, Sethian JA, Vemuri BC (1995) Shape modeling with front propagation: a level set approach. IEEE Trans Pattern Anal Mach Intell 17:158–175. https://doi.org/10.1109/34.368173
Man GS, Mologhianu G (2014) Osteoarthritis pathogenesis—a complex process that involves the entire joint J Med. Life 7:37–41
Mononen ME, Tanska P, Isaksson H, Korhonen RK (2016) A novel method to simulate the progression of collagen degeneration of cartilage in the knee: data from the osteoarthritis initiative. Sci Rep 6:21415. https://doi.org/10.1038/srep21415
Mononen ME, Liukkonen MK, Korhonen RK (2019) Utilizing atlas-based modeling to predict knee joint cartilage degeneration: data from the osteoarthritis initiative. Ann Biomed Eng 47:813–825. https://doi.org/10.1007/s10439-018-02184-y
Mortensen EN, Barrett WA (1998) Interactive segmentation with intelligent scissors. Gr Models Image Process 60:349–384. https://doi.org/10.1006/gmip.1998.0480
Neogi T (2012) Clinical significance of bone changes in osteoarthritis. Ther Adv Musculoskeletal Dis 4:259–267. https://doi.org/10.1177/1759720X12437354
Neogi T et al (2009) Cartilage loss occurs in the same subregions as subchondral bone attrition: a within-knee subregion-matched approach from the multicenter osteoarthritis study. Arthritis Care Res 61:1539–1544. https://doi.org/10.1002/art.24824
Neogi T et al (2013) Magnetic resonance imaging-based three-dimensional bone shape of the knee predicts onset of knee osteoarthritis: data from the osteoarthritis initiative. Arthritis Rheum 65:2048–2058. https://doi.org/10.1002/art.37987
Norman B, Pedoia V, Majumdar S (2018) Use of 2D U-Net convolutional neural networks for automated cartilage and meniscus segmentation of knee MR imaging data to determine relaxometry and morphometry. Radiology 288:177–185. https://doi.org/10.1148/radiol.2018172322
Norman B, Pedoia V, Noworolski A, Link T, Majumdar S (2018) Applying densely connected convolutional neural networks for staging osteoarthritis severity from plain radiographs. J Digit Imaging 32:471–477. https://doi.org/10.1007/s10278-018-0098-3
Öztürk CN, Albayrak S (2016) Automatic segmentation of cartilage in high-field magnetic resonance images of the knee joint with an improved voxel-classification-driven region-growing algorithm using vicinity-correlated subsampling. Comput Biol Med 72:90–107. https://doi.org/10.1016/j.compbiomed.2016.03.011
Pakin SK, Tamez-Pena J, Totterman S, Parker K (2002) Segmentation, surface extraction, and thickness computation of articular cartilage, vol 4684. Medical Imaging 2002. SPIE
Palazzo C, Nguyen C, Lefevre-Colau M-M, Rannou F, Poiraudeau S (2016) Risk factors and burden of osteoarthritis. Ann Phys Rehabil Med 59:134–138. https://doi.org/10.1016/j.rehab.2016.01.006
Panfilov E, Tiulpin A, Klein S, Nieminen MT, Saarakkala S (2019) Improving robustness of deep learning based knee MRI segmentation: mixup and adversarial domain adaptation. In: IEEE International conference on computer vision workshop (ICCVW), Seoul, Korea, pp 450–459
Pang J, Li P, Qiu M, Chen W, Qiao L (2015) Automatic articular cartilage segmentation based on pattern recognition from knee MRI images. J Digit Imaging 28:695–703. https://doi.org/10.1007/s10278-015-9780-x
Paranjape CS et al (2019) A new stress test for knee joint cartilage. Sci Rep 9:2283. https://doi.org/10.1038/s41598-018-38104-2
Park SH et al (2009) Fully automatic 3-D segmentation of knee bone compartments by iterative local branch-and-mincut on MR images from osteoarthritis initiative (OAI). In: IEEE 16th international conference on image processing (ICIP), 7–10 Nov. 2009. pp 3381–3384. https://doi.org/10.1109/ICIP.2009.5413874
Pedoia V, Majumdar S, Link TM (2016) Segmentation of joint and musculoskeletal tissue in the study of arthritis. Magn Reson Mater Phys Biol Med 29:207–221. https://doi.org/10.1007/s10334-016-0532-9
Pedoia V, Lee J, Norman B, Link T, Majumdar S (2019) Diagnosing osteoarthritis from T2 Maps using deep learning: an analysis of the entire osteoarthritis initiative baseline cohort. Osteoarthr Cartil 27:1002–1010. https://doi.org/10.1016/j.joca.2019.02.800
Pelletier J-P et al (2007) Risk factors associated with the loss of cartilage volume on weight-bearing areas in knee osteoarthritis patients assessed by quantitative magnetic resonance imaging: a longitudinal study. Arthritis Res Ther 9:R74. https://doi.org/10.1186/ar2272
Peterfy CG, Schneider E, Nevitt M (2008) The osteoarthritis initiative: report on the design rationale for the magnetic resonance imaging protocol for the knee. Osteoarthr Cartil 16:1433–1441. https://doi.org/10.1016/j.joca.2008.06.016
Peuna A et al (2018) Variable angle gray level co-occurrence matrix analysis of T2 relaxation time maps reveals degenerative changes of cartilage in knee osteoarthritis: Oulu knee osteoarthritis study. J Magn Reson Imaging 47:1316–1327. https://doi.org/10.1002/jmri.25881
Prasoon A, Petersen K, Igel C, Lauze F, Dam E, Nielsen M (2013) Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. In: Medical image computing and computer assisted intervention—MICCAI 2013. Springer, Berlin, pp 246–253
Quinlan JR (1986) Induction of decision trees. Mach Learn 1:81–106. https://doi.org/10.1007/BF00116251
Raghu M, Schmidt E (2020) A survey of deep learning for scientific discovery. arXiv Preprint 0:1–48
Raj A, Vishwanathan S, Ajani B, Krishnan K, Agarwal H (2018) Automatic knee cartilage segmentation using fully volumetric convolutional neural networks for evaluation of osteoarthritis. In: IEEE 15th international symposium on biomedical imaging (ISBI 2018), 4–7 April 2018, pp 851–854. https://doi.org/10.1109/ISBI.2018.8363705
Rini C, Perumal B, Rajasekaran MP (2020) Automatic knee joint segmentation using Douglas-Rachford splitting method. Multimed Tools Appl 79:6599–6621. https://doi.org/10.1007/s11042-019-08303-8
Rish I (2001) An empirical study of the naive Bayes classifier. In: IJCAI 2001 workshop on empirical methods in artificial intelligence, vol 22, pp 41–46
Riza S, Marlinawati D, Fahmi M (2019) COMSeg technique for MRI knee cartilage segmentation. Int Rev Appl Sci Eng 10:1–9. https://doi.org/10.1556/1848.2019.0018
Roemer FW et al (2010) A comparison of dedicated 1.0 T extremity MRI vs large-bore 1.5 T MRI for semiquantitative whole organ assessment of osteoarthritis: the MOST study. Osteoarthr Cartil. 18:168–174. https://doi.org/10.1016/j.joca.2009.08.017
Rohlfing T, Brandt R, Menzel R, Russakoff DB, Maurer CR (2005) Quo vadis, atlas-based segmentation? In: Suri JS, Wilson DL, Laxminarayan S (eds) Handbook of biomedical image analysis: Volume III: registration models. Springer US, Boston, MA, pp 435–486. https://doi.org/10.1007/0-306-48608-3_11
Rokach L (2010) Ensemble-based classifiers. Artif Intell Rev 33:1–39. https://doi.org/10.1007/s10462-009-9124-7
Roughley PJ, Mort JS (2014) The role of aggrecan in normal and osteoarthritic cartilage Journal of Experimental Orthopaedics 1:8. https://doi.org/10.1186/s40634-014-0008-7
Schaefer LF et al (2017) Quantitative measurement of medial femoral knee cartilage volume - analysis of the OA Biomarkers Consortium FNIH Study cohort Osteoarthritis and cartilage 25:1107–1113. https://doi.org/10.1016/j.joca.2017.01.010
Schmid J, Magnenat-Thalmann N (2008) MRI bone segmentation using deformable models and shape priors. In: Medical image computing and computer assisted intervention—MICCAI 2008, 2008. Springer, Berlin, pp 119–126
Seim H, Kainmueller D, Lamecker H, Bindernagel M, Malinowski J, Zachow S (2010) Model-based auto-segmentation of knee bones and cartilage in MRI data Proc Med Image Anal
Sengupta S et al (2020) A review of deep learning with special emphasis on architectures, applications and recent trends Knowledge-Based Systems 194:105596 https://doi.org/10.1016/j.knosys.2020.105596
Serre T (2019) Deep Learning: The Good, the Bad, and the Ugly Annual Review of Vision Science 5:399–426 https://doi.org/10.1146/annurev-vision-091718-014951
Shah RF, Martinez AM, Pedoia V, Majumdar S, Vail TP, Bini SA (2019) Variation in the Thickness of Knee Cartilage. The Use of a Novel Machine Learning Algorithm for Cartilage Segmentation of Magnetic Resonance Images The Journal of Arthroplasty 34:2210–2215. https://doi.org/10.1016/j.arth.2019.07.022
Shamir L, Orlov N, Eckley DM, Macura T, Johnston J, Goldberg IG (2008) Wndchrm – an open source utility for biological image analysis Source Code for Biology and Medicine 3:13. https://doi.org/10.1186/1751-0473-3-13
Shan L, Charles C, Niethammer M (2012a) Automatic Atlas-based Three-label Cartilage Segmentation from MR Knee Images Proc Workshop Math Methods Biomed Image Analysis:241–246. https://doi.org/10.1109/mmbia.2012.6164757
Shan L, Charles C, Niethammer M Automatic multi-atlas-based cartilage segmentation from knee MR images. In: IEEE 9th International Symposium on Biomedical Imaging (ISBI 2012), 2–5 May 2012 2012b. pp 1028–1031. https://doi.org/10.1109/ISBI.2012.6235733
Shan L, Zach C, Charles C, Niethammer M (2014) Automatic atlas-based three-label cartilage segmentation from MR knee images. Med Image Anal 18:1233–1246. https://doi.org/10.1016/j.media.2014.05.008
Sharif B, Garner R, Hennessy D, Sanmartin C, Flanagan WM, Marshall DA (2017) Productivity costs of work loss associated with osteoarthritis in Canada from 2010 to 2031. Osteoarthr Cartil 25:249–258. https://doi.org/10.1016/j.joca.2016.09.011
Sharma AR, Jagga S, Lee S-S, Nam J-S (2013) Interplay between cartilage and subchondral bone contributing to pathogenesis of osteoarthritis. Int J Mol Sci 14:19805–19830. https://doi.org/10.3390/ijms141019805
Shen D, Wu G, Suk H-I (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19:221–248. https://doi.org/10.1146/annurev-bioeng-071516-044442
Shim H, Chang S, Tao C, Wang J-H, Kwoh CK, Bae KT (2009) Knee cartilage: efficient and reproducible segmentation on high-spatial-resolution MR images with the semiautomated graph-cut algorithm method. Radiology 251:548–556. https://doi.org/10.1148/radiol.2512081332
Shim H, Kwoh CK, Yun ID, Lee SU, Bae K (2009b) Simultaneous 3D segmentation of three bone compartments on high resolution knee MR images from osteoarthritis initiative (OAI) using graph cuts, vol 7259. SPIE Medical Imaging. SPIE
Shrestha A, Mahmood A (2019) Review of deep learning algorithms and architectures. IEEE Access 7:53040–53065. https://doi.org/10.1109/ACCESS.2019.2912200
Singh S, Wang L, Gupta S, Goli H, Padmanabhan P, Gulyas B (2020) 3D deep learning on medical images: a review. arXiv Preprint:1–13
Smith JJ, Sorensen AG, Thrall JH (2003) Biomarkers in imaging: realizing radiology’s future. Radiology 227:633–638. https://doi.org/10.1148/radiol.2273020518
Sokolove J, Lepus CM (2013) Role of inflammation in the pathogenesis of osteoarthritis: latest findings and interpretations. Ther Adv Musculoskeletal Dis 5:77–94. https://doi.org/10.1177/1759720X12467868
Solloway S, Hutchinson CE, Waterton JC, Taylor CJ (1997) The use of active shape models for making thickness measurements of articular cartilage from MR images. Magn Reson Med 37:943–952. https://doi.org/10.1002/mrm.1910370620
Stammberger T, Eckstein F, Michaelis M, Englmeier K-H, Reiser M (1999) Interobserver reproducibility of quantitative cartilage measurements: comparison of B-spline snakes and manual segmentation. Magn Reson Imaging 17:1033–1042. https://doi.org/10.1016/S0730-725X(99)00040-5
Stewart HL, Kawcak CE (2018) The importance of subchondral bone in the pathophysiology of osteoarthritis. Front Vet Sci 5:178–178. https://doi.org/10.3389/fvets.2018.00178
Tack A, Zachow S (2019) Accurate automated volumetry of cartilage of the knee using convolutional neural networks: data from the osteoarthritis initiative. In: IEEE 16th international symposium on biomedical imaging (ISBI 2019), Venice, Italy. IEEE, pp 40–43. https://doi.org/10.1109/ISBI.2019.8759201
Tamez-Peña JG, Farber J, González PC, Schreyer E, Schneider E, Totterman S (2012) Unsupervised segmentation and quantification of anatomical knee features: data from the osteoarthritis initiative. IEEE Trans Biomed Eng 59:1177–1186. https://doi.org/10.1109/TBME.2012.2186612
Tan C, Yan Z, Zhang S, Li K, Metaxas DN (2019) collaborative multi-agent learning for MR knee articular cartilage segmentation. In: Shen D et al (eds) Medical image computing and computer assisted intervention—MICCAI 2019, Shenzhen, China, 2019. Springer Berlin Heidelberg, pp 282–290
Tang J, Millington S, Acton ST, Crandall J, Hurwitz S (2006) Surface extraction and thickness measurement of the articular cartilage from MR images using directional gradient vector flow snakes. IEEE Trans Biomed Eng 53:896–907. https://doi.org/10.1109/TBME.2006.872816
Thaha R, Jogi SP, Rajan S, Mahajan V, Venugopal VK, Mehndiratta A, Singh A (2020) Modified radial-search algorithm for segmentation of tibiofemoral cartilage in MR images of patients with subchondral lesion. Int J Comput Assist Radiol Surg 15:403–413. https://doi.org/10.1007/s11548-020-02116-z
Thengade A, Rajurkar A (2019) A comprehensive survey of articular cartilage segmentation methods on knee MRI. Int J Adv Sci Technol 27:148–159
Thomas KA et al (2020) Automated classification of radiographic knee osteoarthritis severity using deep neural networks radiology. Artif Intell 2:e190065. https://doi.org/10.1148/ryai.2020190065
Tiulpin A, Saarakkala S (2019) Automatic grading of individual knee osteoarthritis features in plain radiographs using deep convolutional neural networks. arXiv Preprint:1–14
Tiulpin A, Thevenot J, Rahtu E, Lehenkari P, Saarakkala S (2018) Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach. Sci Rep 8:1727. https://doi.org/10.1038/s41598-018-20132-7
Tiulpin A et al (2019) Multimodal machine learning-based knee osteoarthritis progression prediction from plain radiographs and clinical data. Sci Rep 9:20038. https://doi.org/10.1038/s41598-019-56527-3
Vilimek D, Kubicek J, Penhaker M, Oczka D, Augustynek M, Cerny M (2019) Current automatic methods for knee cartilage segmentation: a review. In: 8th European workshop on visual information processing (EUVIP), 28–31 Oct. 2019, pp 117–122. https://doi.org/10.1109/EUVIP47703.2019.8946132
Vina ER, Kwoh CK (2018) Epidemiology of osteoarthritis: literature update. Curr Opin Rheumatol 30:160–167. https://doi.org/10.1097/BOR.0000000000000479
Wang Q, Wu D, Lu L, Liu M, Boyer KL, Zhou SK (2014) Semantic context forests for learning-based knee cartilage segmentation in 3D MR images. In: International conference on medical image computing and computer-assisted intervention: MICCAI Cham. Medical Computer Vision. Large data in medical imaging. Springer International Publishing, pp 105–115
Warner SC, Valdes AM (2016) The genetics of osteoarthritis: a review. J Funct Morphol Kinesiol 1:140–153
Waterton JC et al (2000) Diurnal variation in the femoral articular cartilage of the knee in young adult humans. Magn Reson Med 43:126–132. https://doi.org/10.1002/(sici)1522-2594(200001)43:1%3c126::aid-mrm15%3e3.0.co;2-#
Williams TG et al (2010a) Automatic segmentation of bones and inter-image anatomical correspondence by volumetric statistical modelling of knee MRI. In: IEEE International symposium on biomedical imaging: from nano to macro, 14–17 April 2010, pp 432–435. https://doi.org/10.1109/ISBI.2010.5490316
Williams TG et al (2010b) Measurement and visualisation of focal cartilage thickness change by MRI in a study of knee osteoarthritis using a novel image analysis tool. Br J Radiol 83:940–948. https://doi.org/10.1259/bjr/68875123
Wluka AE, Stuckey S, Snaddon J, Cicuttini FM (2002) The determinants of change in tibial cartilage volume in osteoarthritic knees. Arthritis Rheum 46:2065–2072. https://doi.org/10.1002/art.10460
Wluka AE et al (2009) Bone marrow lesions predict increase in knee cartilage defects and loss of cartilage volume in middle-aged women without knee pain over 2 years. Ann Rheum Dis 68:850–855. https://doi.org/10.1136/ard.2008.092221
Wu Z, Leahy R (1993) An optimal graph theoretic approach to data clustering: theory and its application to image segmentation. IEEE Trans Pattern Anal Mach Intell 15:1101–1113. https://doi.org/10.1109/34.244673
Xu Z, Niethammer M (2019) DeepAtlas: joint semi-supervised learning of image registration and segmentation. In: Shen D et al (eds) Medical image computing and computer assisted intervention—MICCAI 2019, Cham, 2019. Springer, Berlin, pp 420–429
Yin Y, Zhang X, Williams R, Wu X, Anderson DD, Sonka M (2010) LOGISMOS-layered optimal graph image segmentation of multiple objects and surfaces: cartilage segmentation in the knee joint. IEEE Trans Med Imaging 29:2023–2037. https://doi.org/10.1109/TMI.2010.2058861
Zhang K, Lu W, Marziliano P (2013) Automatic knee cartilage segmentation from multi-contrast MR images using support vector machine classification with spatial dependencies. Magn Reson Imaging 31:1731–1743. https://doi.org/10.1016/j.mri.2013.06.005
Zhang W, Ouyang H, Dass CR, Xu J (2016) Current research on pharmacologic and regenerative therapies for osteoarthritis. Bone Res 4:15040–15040. https://doi.org/10.1038/boneres.2015.40
Zhang B, Zhang Y, Cheng H-D, Xian M, Gai S, Cheng O, Huang K (2018) Computer-aided knee joint magnetic resonance image segmentation—a survey. CoRR abs/1802.04894:1–10
Zhou Z, Zhao G, Kijowski R, Liu F (2018) Deep convolutional neural network for segmentation of knee joint anatomy. Magn Reson Med 80:2759–2770. https://doi.org/10.1002/mrm.27229
Zhou T, Ruan S, Canu S (2019) A review: deep learning for medical image segmentation using multi-modality fusion. Array 3–4:100004. https://doi.org/10.1016/j.array.2019.100004
Acknowledgements
The authors acknowledge the valuable help provided by Prashant Shukla Kumar in collecting the literature materials.
Funding
The study was funded by Fundamental Research Grant Scheme (FRGS) (project title: Graph Transformed Deep ‘Interactive’ Learning Framework in Medical Image Segmentation, grant no: FRGS/1/2018/ICT02/UNIKL/02/4) provided by Ministry of Education, Malaysia (MoE).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gan, HS., Ramlee, M.H., Wahab, A.A. et al. From classical to deep learning: review on cartilage and bone segmentation techniques in knee osteoarthritis research. Artif Intell Rev 54, 2445–2494 (2021). https://doi.org/10.1007/s10462-020-09924-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-020-09924-4