Vision Transformer-Based Multi-Label Survival Prediction for Oropharynx Cancer Radiotherapy Using Planning CT.

Researchers

Journal

Modalities

Models

Abstract

A reliable and comprehensive cancer prognosis model for oropharyngeal cancers (OPCs) could better assist in personalizing treatment. In this work, we developed a vision transformer-based (ViT-based) multi-label model with multi-modal input to learn complimentary information from available pretreatment data and predict multiple associated endpoints for OPC patient radiotherapy.In our study, a publicly available dataset of 512 OPC patients was utilized for both model training and evaluation. Planning CT images, primary gross tumor volume (GTVp) masks and 16 clinical variables representing patient demographic, diagnosis, and treatment were used as input. To extract deep image features with global attention, we utilized a ViT module. Clinical variables were concatenated with the learnt image features and fed to fully-connection layers to incorporate cross-modality features. To learn the mapping between the features and correlated survival outcomes, such as overall survival (OS), local failure-free survival (LFFS), regional failure-free survival (RFFS), and distant failure-free survival (DFFS), we employed four Multi-Task Logistic Regression (MTLR) layers. The proposed model was optimized by combining the MTLR negative-log likelihood losses of different prediction targets.We employed the C-index and AUC metrics to assess the performance of our model for time-to-event prediction and time-specific binary prediction, respectively. Our proposed model outperformed corresponding single-modality and single-label models on all prediction labels, which achieved C-indices of 0.773, 0.765, 0.776, and 0.773 for OS, LFFS, RFFS, and DFFS, respectively. The AUC values ranged between 0.799 and 0.844 for different tasks at different time points. Furthermore, when using the medians of predicted risks as the thresholds to identify high-risk and low-risk patient groups, the log-rank test results showed significant larger separations in different event-free survivals.We developed the first model capable of predicting multiple labels for OPC simultaneously. Our model demonstrated better prognostic ability for all the prediction targets compared with corresponding single-modality models and single-label models.Copyright © 2023 Elsevier Inc. All rights reserved.