Multi-task Relational Learning Network for MRI Vertebral Localization, Identification and Segmentation.

Researchers

Journal

Modalities

Models

Abstract

Magnetic resonance imaging (MRI) vertebral localization, identification, and segmentation are important steps in the automatic analysis of spines. Due to the similar appearances of vertebrae, various pathological patterns and imaging artifacts, the accurate segmentation, localization and identification of vertebrae remain challenging. With the emergence of convolutional neural networks, deep learning based methods have been successfully developed to address these three tasks. However, previous methods solve the three tasks independently, ignoring the intrinsic correlation among them. In this paper, we propose a multi-task relational learning network (MRLN) that utilizes both the relationships between vertebrae and the relevance of the three tasks. Specifically, we combine a dilation convolution group and LSTM to learn the prior knowledge that the identification information is always in a fixed order for spine images. A co-attention module is proposed to learn the localization-guided segmentation attention and segmentation-guided localization attention, which improves segmentation and localization performance based on the identification information. In addition, a novel multi-task loss function named XOR loss is been create. This method was evaluated on a dataset which includes multiple MRI modalities (T1 and T2), various fields of view and four variety (intensity variety, pathological variety, uneven variety, and size variety, shown in Fig. 1). For vertebrae segmentation, the average Dice score was 95.38%. The anatomical identification accuracy is 93.55% and mean localization error is 2.6265 mm. Overall, the advantage of our framework is that it can fully make use of the relationships of different vertebrae and three tasks simultaneously, which makes it an attractive method for the automatic analysis of spine.