THE EFFECT OF VARIABLE LABELS ON DEEP LEARNING MODELS TRAINED TO PREDICT BREAST DENSITY.
Researchers
Journal
Modalities
Models
Abstract
High breast density is associated with reduced efficacy of mammographic screening
and increased risk of developing breast cancer. Accurate and reliable automated density
estimates can be used for direct risk prediction and passing density related information to
further predictive models. Expert reader assessments of density show a strong relationship to
cancer risk but also inter-reader variation. The effect of label variability on model performance
is important when considering how to utilise automated methods for research and clinical
purposes.
Methods: We utilise subsets of images with density labels from the same 13 readers and 12
reader pairs, and train a deep transfer learning model which is used to assess how label
variability affects the mapping from representation to prediction. We create two
end-to-end models: one that is trained on averaged labels across the reader pairs and the
second that is trained using individual reader scores, with a novel alteration to the objective
function. The combination of these two end-to-end models allows us to investigate the effect of
label variability on the model representation formed.
Results: The trained mappings from representations to labels are altered
considerably by the variability of reader scores. Training on labels with distribution variation
removed causes the Spearman rank correlation coefficients to rise from 0.751 ± 0.002 to either
0.815 ± 0.006 when averaging across readers or 0.844 ± 0.002 when averaging across images.
However, when we train different models to investigate the representation effect we see little
difference, with Spearman rank correlation coefficients of 0.846 ± 0.006 and 0.850 ± 0.006
showing no statistically significant difference in the quality of the model representation with
regard to density prediction.
Conclusions: The mapping between representation and mammographic density
prediction is significantly affected by label variability. However, the effect of the label variability
on the model representation is limited.Creative Commons Attribution license.