A Multi-task Learning framework for Segmentation and Classification of Patellofemoral Osteoarthritis in Multi-parametric Magnetic Resonance Imaging
Pages 449 - 456
Abstract
Patellofemoral osteoarthritis (PFOA) is a common disease that seriously impacts patients' quality of life. The diagnosis of PFOA is related to the shape of bones and joints. In addition, multi-parametric magnetic resonance imaging (mMRI) can provide complementary information for diagnosing PFOA. Herein, we proposed a multi-parametric multi-task learning framework (MM-ResUNet) for the segmentation and classification of PFOA. This framework includes an encoder-decoder network for segmentation and a sub-network for classification. At the same time, we also proposed a learnable feature fusion module to further improve the classification performance for PFOA. Lastly, a hybrid multi-task loss function is proposed to supervise the training process of the framework. We retrospectively collected 200 patients' multi-parameter MRI images, and the result shows that our multi-parameter multi-task learning framework significantly improves the performance of both tasks.
References
[1]
Lipton R B, Schwedt T J, Friedman B W. GBD 2015 Disease and Injury Incidence and Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990-2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet. 2017 Jan 5; 388 (10053): 1545–602. 27733282. [ : 27733282][Cross Ref][J]. 2016.
[2]
Klareskog L, Catrina A I. Paget S[J]. Rheumatoid arthritis. Lancet, 2009, 373: 659-72.
[3]
Qian Wang, Cibo Huang. Advances in the diagnosis and treatment of osteoarthritis[J]. Journal of Clinical Pharmacotherapy, 2010, 8(2): 36-38, 4.
[4]
Yan Ren, YaYa Shi, Bo Tan. Meta-analysis of risk factors for patellar osteoarthritis in Chinese population [J]. Modern Preventive Medicine, 2015, 42(12): 2282-2284.
[5]
Cross M, Smith E, Hoy D, The global burden of hip and knee osteoarthritis: estimates from the global burden of disease 2010 study[J]. Annals of the rheumatic diseases, 2014, 73(7): 1323-1330.
[6]
Roemer F W, Kwoh C K, Hayashi D, The role of radiography and MRI for eligibility assessment in DMOAD trials of knee OA[J]. Nature Reviews Rheumatology, 2018, 14(6): 372-380.
[7]
Roemer F W, Collins J, Kwoh C K, MRI-based screening for structural definition of eligibility in clinical DMOAD trials: rapid OsteoArthritis MRI Eligibility Score (ROAMES)[J]. Osteoarthritis and cartilage, 2020, 28(1): 71-81.
[8]
LeCun Y, Bengio Y, Hinton G. Deep learning[J]. nature, 2015, 521(7553): 436-444.
[9]
Namiri N K, Lee J, Astuto B, Deep learning for large scale MRI-based morphological phenotyping of osteoarthritis[J]. Scientific reports, 2021, 11(1): 1-10.
[10]
Tolpadi A A, Lee J J, Pedoia V, Deep learning predicts total knee replacement from magnetic resonance images[J]. Scientific reports, 2020, 10(1): 1-12.
[11]
von Schacky C E, Sohn J H, Liu F, Development and validation of a multitask deep learning model for severity grading of hip osteoarthritis features on radiographs[J]. Radiology, 2020, 295(1): 136-145.
[12]
WangLong Tang. The value of total knee magnetic resonance imaging in the diagnosis of patellar arthritis [J]. Imaging Research and Medical Applications, 2021.
[13]
Pin Li, QinFeng Ma, XianXia Chen. Comparison of the application value of high frequency ultrasound and MRI in the diagnosis of patellar arthritis [J]. collect, 2019, 10.
[14]
Wang P, Patel V M, Hacihaliloglu I. Simultaneous segmentation and classification of bone surfaces from ultrasound using a multi-feature guided cnn[C]//International conference on medical image computing and computer-assisted intervention. Springer, Cham, 2018: 134-142.
[15]
Xie X, Shi F, Niu J, Breast ultrasound image classification and segmentation using convolutional neural networks[C]//Pacific rim conference on multimedia. Springer, Cham, 2018: 200-211.
[16]
Song L, Lin J, Wang Z J, An end-to-end multi-task deep learning framework for skin lesion analysis[J]. IEEE journal of biomedical and health informatics, 2020, 24(10): 2912-2921.
[17]
Chakravarty A, Sivswamy J. A deep learning based joint segmentation and classification framework for glaucoma assesment in retinal color fundus images[J]. arXiv preprint arXiv:1808.01355, 2018.
[18]
Chen C, Bai W, Rueckert D. Multi-task learning for left atrial segmentation on GE-MRI[C]//International workshop on statistical atlases and computational models of the heart. Springer, Cham, 2018: 292-301
[19]
Qu H, Riedlinger G, Wu P, Joint segmentation and fine-grained classification of nuclei in histopathology images[C]//2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, 2019: 900-904.
[20]
Wang G, Li W, Ourselin S, Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks[C]//International MICCAI brainlesion workshop. Springer, Cham, 2017: 178-190
[21]
Chen L, Wu Y, DSouza A M, MRI tumor segmentation with densely connected 3D CNN[C]//Medical Imaging 2018: Image Processing. International Society for Optics and Photonics, 2018, 10574: 105741F.
[22]
Zhou C, Ding C, Lu Z, One-pass multi-task convolutional neural networks for efficient brain tumor segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2018: 637-645.
[23]
Dolz J, Gopinath K, Yuan J, HyperDense-Net: a hyper-densely connected CNN for multi-modal image segmentation[J]. IEEE transactions on medical imaging, 2018, 38(5): 1116-1126.
[24]
Dolz J, Desrosiers C, Ayed I B. IVD-Net: Intervertebral disc localization and segmentation in MRI with a multi-modal UNet[C]//International workshop and challenge on computational methods and clinical applications for spine imaging. Springer, Cham, 2018: 130-143.
[25]
He K, Zhang X, Ren S, Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.
[26]
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556, 2014.
[27]
Zhou, Y., Chen, H., Li, Y., Liu, Q., Xu, X., Wang, S., ... & Shen, D. (2021). Multi-task learning for segmentation and classification of tumors in 3D automated breast ultrasound images. Medical Image Analysis, 70, 101918.
Index Terms
- A Multi-task Learning framework for Segmentation and Classification of Patellofemoral Osteoarthritis in Multi-parametric Magnetic Resonance Imaging
Recommendations
In-depth learning of automatic segmentation of shoulder joint magnetic resonance images based on convolutional neural networks
Highlights- CNN is used to segment multiple bone joints to assist medical diagnosis.
- Image ...
Abstract ObjectiveMagnetic resonance imaging (MRI) is gradually replacing computed tomography (CT) in the examination of bones and joints. The accurate and automatic segmentation of the bone structure in the MRI of the shoulder ...
Deep Learning for Multi-task Medical Image Segmentation in Multiple Modalities
Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016AbstractAutomatic segmentation of medical images is an important task for many clinical applications. In practice, a wide range of anatomical structures are visualised using different imaging modalities. In this paper, we investigate whether a single ...
Comments
Information & Contributors
Information
Published In
September 2022
1221 pages
ISBN:9781450396899
DOI:10.1145/3573942
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 16 May 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
AIPR 2022
AIPR 2022: 2022 5th International Conference on Artificial Intelligence and Pattern Recognition
September 23 - 25, 2022
Xiamen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 39Total Downloads
- Downloads (Last 12 months)12
- Downloads (Last 6 weeks)2
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format