Successful Real-World Application of an Osteoarthritis Classification Deep-learning Model Using 9210 Knees-An Orthopedic Surgeon’s View.

Researchers

Journal

Modalities

Models

Abstract

This study aimed to evaluate the performance of a deep-learning model to evaluate knee osteoarthritis using Kellgren-Lawrence grading in real-life knee radiographs. A deep convolutional neural network model was trained using 8,964 knee radiographs from the Osteoarthritis Initiative, including 962 testing set images. Another 246 knee radiographs from the Far Eastern Memorial Hospital were used for external validation. The Osteoarthritis Initiative testing set and external validation images were evaluated by experienced specialists, two orthopedic surgeons, and a musculoskeletal radiologist. The accuracy, inter-observer agreement, F1 score, precision, recall, specificity, and ability to identify surgical candidates were used to compare the performances of the model and specialists. Attention maps illustrated the interpretability of the model classification. The model had a 78% accuracy and consistent inter-observer agreement for the Osteoarthritis Initiative (model-surgeon 1 К=0.80, model-surgeon 2 К=0.84, model-radiologist К=0.86) and external validation (model-surgeon 1 К=0.81, model-surgeon 2 К=0.82, model-radiologist К=0.83) images. A lower inter-observer agreement was found in the images misclassified by the model (model-surgeon 1 К=0.57, model-surgeon 2 К=0.47, model-radiologist К=0.65). The model performed better than specialists in identifying surgical candidates (Kellgren-Lawrence stages 3 and 4) with an F1 score of 0.923. Our model not only had comparable results with specialists with respect to the ability to identify surgical candidates but also performed consistently with open database and real-life radiographs. We believe the controversy of the misclassified knee osteoarthritis images was based on a significantly lower inter-observer agreement. This article is protected by copyright. All rights reserved.This article is protected by copyright. All rights reserved.