Artificial Intelligence in Musculoskeletal Radiology: Past, Present, and Future

Artificial neural networks applied to musculoskeletal radiology

Today, the importance of adapting AI and ML into day-to-day use in musculoskeletal radiology is being noted by more and more scholars.^[9] Recent studies have tried to use AI and ML to transform the field of musculoskeletal radiology by enhancing the quality of assessment that can be derived from radiological techniques. DNNs, radial basis function neural networks, and generative adversarial ML are among the most commonly used AI-related approaches in musculoskeletal radiology. DNNs, such as convolutional neural networks (CNNs), constitute a specific form of DL AI algorithms that allow the compilation of data from multiple characteristics of the same input dataset provided to them.^[9] A subset of the available data is used to train the AI algorithms, which can then generalize the information and use it to analyze and interpret the information contained in other similar data sets.^[15] CNNs are best suited for working with imaging data as they are designed to preserve spatial relationships. A grid-based input image is provided to a CNN, within which a filter is applied to all locations, giving rise to a convolutional layer of “artificial neurons” [Figures 2 and 3]. Features identified through the filter are pooled and further inputted into a new convolutional layer and so on. Each new layer focuses on a subset of the area in the previous layer. In the end, a fully connected layer is formed, which connects all neurons of the most recent layer to all neurons of the previous layer. Finally, the predicted output(s) are presented based on the prior training of the algorithm, along with the probability of each predicted output.^[15]

Close

Figure 2:

A simplistic representation of a convolutional neural network. Data from an input image are transferred through a convolutional layer followed by pooling of features. Data from the pool are further inputted into another convolutional layer and so on. In the end, a fully connected layer maps all features between the final convolutional layer and the previous layer. The final output provides probabilistic predictions, highlighting which output is more likely based on how the algorithm was trained.

Export to PPT

Close

Figure 3:

Diagrammatic representation of an artificial neuron and biologic neuron. Input of data is received through the dendrites, which are usually termed weights in the artificial neuron. Each input is multiplied by its corresponding weight, and all the multiplications are summed. A non-linear mathematical formula, rectified linear unit function is performed on the result. The output of the neuron serves as an input in the next layer of neurons.

Export to PPT

Meanwhile, another class of neural networks called the radial basis function neural networks has been found to be better in performance and faster in training compared to many other neural networks.^[16] Another commonly used AI approach is generative adversarial learning. Generative adversarial networks (GANs) represent a class of DL methods that generate new data instead of processing existing information. GANs combine a generative network and a discriminative network to enhance the performance of a DL model.^[17] Neural networks such as DNNs may be used as discriminative networks to train GANs.^[17] In the field of musculoskeletal imaging, the use of GANs is still very limited. However, some studies have utilized GANs to analyze musculoskeletal MRI scans and radiographs. Using the above-mentioned AI and ML techniques, numerous studies have improved on the pre-processing, acquisition and analysis of radiographs, CT scans, MRI scans, and ultrasonographs [Figure 4 and Table 1]. Some of these studies have been discussed below.

Close

Figure 4:

Steps involved and basic differences in the processing of data in artificial intelligence, machine learning, and deep learning.

Export to PPT

Image pre-processing

Many recent studies have used the musculoskeletal radiographs dataset (MURA) to train and test different AI models to improve on image pre-processing.^[18] MURA is an open-source X-ray image dataset that makes both normal and abnormal musculoskeletal images available to researchers and radiologists for studies.^[19] Mall et al. used ML algorithms to classify bone radiographs from MURA as fractured or intact.^[18] The four ML algorithms used in this study performed image pre-processing tasks such as color transformation and contrast enhancement before extracting as many as 12 relevant gray level cooccurrence matrix features, including autocorrelation, contrast, dissimilarity, and energy from each image. These textural features were subsequently used to classify the images into “fracture” and “non-fracture” groups. About 67% of the data was used to train the algorithm while the rest was used for testing. The radial basis function support vector machine (LBF SVM) algorithm achieved the highest accuracy (62%) among the four ML algorithms tested.^[18]

Galbusera et al. conducted a study on applying GANs to preprocess spine MR images.^[20] They evaluated the efficacy of GANs in detecting defects such as herniation, degenerative disc diseases, and end-plate changes from MRI data. Moreover, radiographs of the lumbar spine were used to identify abnormal numbering of the vertebra and vertebral body compression fractures. Ultimately, the GANs used in the study were able to make low-resolution images sharper and more informative, highlighting the importance of AI-based approaches in aiding musculoskeletal radiologists in their evaluation of spine imaging data.

Image acquisition

MRI is preferred when the application necessitates superior soft-tissue contrast.^[21] Superresolution musculoskeletal MRI can help generate thin, high-resolution image slices from thicker slices of complex body regions such as the knee.^[21] This can also help accelerate the process of MRI data acquisition as data, that is, inferior in amount or quality can later be converted to high-resolution images using AI methods.^[7] In one study, Chaudhari et al. used CNNs to develop a superresolution technique to improve the usability of thick slice MRI data.^[4] Chaudhari et al. developed a 3D CNN called DeepResolve to create high-resolution thin slice resolved from a low-resolution thick slice. DeepResolve includes 0.7 mm thin slices from knee images obtained using Double Echo in Steady State. The training dataset for this model came from 124 patients from the osteoarthritis initiative. The model was subsequently tested on data from 17 patients. Based on qualitative assessment of the images by two radiologists along with quantitative assessment, it was found that DeepResolve could generate superresolution knee MR images of the required diagnostic quality or better.^[4] Other studies that have focused on improving MRI image acquisition have used AI to help decide if intravenous contrast is needed in musculoskeletal MRI.^[8] Similarly, there are studies that have shown the use of AI to reduce the amount of radiation used while acquiring CT data from patients.^[7]

Ultrasonographic image acquisition has also been targeted by some AI-based studies, particularly because commonly used ultrasonographic techniques are limited by a narrow field of view.^[22] Jabbar et al. suggested a way to overcome this issue.^[22] The idea was to obtain panoramic ultrasound images by stitching together individual images, thereby helping to visualize wider tissues through a single panoramic image.^[22] Quantifying geometric parameters using panoramic ultrasound images require image processing, including image enhancement, edge detection, and segmentation. The previous studies have noted that the edges of muscle tissue are not preserved upon image segmentation. Jabbar et al. addressed this issue by designing a CNN to identify the pixels in ultrasound images that constitute the edge of the musculoskeletal tissue [Table 1].

Image analysis

Many AI-based studies have focused on improving image analysis to help radiologists with the interpretation of imaging data. To identify the instances of fracture in a set of X-ray images, Cao et al. developed an ML algorithm using the stacked random forest multilayer classifier.^[1] The algorithm collates and fuses different features identified from the training dataset of fractured and healthy bone X-ray images. After training, the algorithm can generate confidence score maps for areas in the X-ray images to indicate which regions have a greater likelihood of having a fracture.^[1] This study used 80% of the imaging dataset for training the algorithm and the remaining 20% for testing. In another study, Wang achieved a fracture identification accuracy of 73% using a Keras DL framework, which is a CNN based approach.^[23] In another study, Li were able to detect skeletal muscle damage using an adversarial learning assisted deep CNN (DCNN) on radiographs.^[17] They used MURA images to conduct their study on various arm musculoskeletal areas such as fingers, shoulder, elbow, and forearm. This model achieved an overall accuracy of 82.1% in the identification of damaged muscular regions. Yet, another study found that DL resulted in human radiologist-like fracture and body part identification in X-ray images.^[24]

Some other X-ray-based studies have explored the use of AI technologies such as artificial neural networks, SVMs, and regression-based methods in determining bone age.^[2,25,26] These include studies conducted to assess differences in skeletal maturity based on gender, race, and age.^[25,26] Other studies have used Bayesian models to calculate the probability of different diagnoses based on radiographical characteristics.^[3,27] For example, Do et al. conducted a study that tested radiological bone tumor data against 29 potential diagnoses, predicting the correct diagnosis in only 44% of the cases.^[27] This study used a minimum of five image samples per diagnosis to train their model^[27] (as a comparison, ML-based methods always require larger imaging datasets to train the AI models^[3]). In another study, Hiasa et al. used Bayesian DL and CNNs to accelerate the segmentation of muscles in CT imaging and improve its accuracy.^[5] By demonstrating the segmentation of 19 muscles in the hip and thigh region, this study showed that the use of CNNs can significantly improve the accuracy of muscular segmentation.^[5] In another MRI-based study, Irmakci et al. used DL algorithms on multiview MRI scans to identify meniscus and anterior cruciate ligament tears as well as other abnormalities in the knee.^[21] In addition to X-ray, CT, and MRI, ultrasonography is another common radiological technique that is used for diagnosing a wide variety of joint and soft-tissue disorders. It holds the advantage of having a short diagnosis time as well as easy applicability for a host of patients.^[28] Musculoskeletal soft-tissue damage is identified as disorganized and hyperechoic patterns in ultrasonography images.^[28] These differences in the visual appearance of damaged musculoskeletal tissue can also be quantified in the form of tissue thickness, cross-sectional area, and mean echogenicity.^[28] However, the identification of visual differences is highly dependent on the experience level of the radiologist, and even with the same experience level, it can still vary from one expert to another.^[6,28] Moreover, segmentation of musculoskeletal tissues such as tendons can be difficult in ultrasound images.^[28] Some AI studies have attempted to overcome these challenges by improving on image analysis.

To address the challenge of radiologist-to-radiologist difference in interpretation, Gupta et al. developed an AI-based automatic segmentation approach using ultrasonographs of the rotator cuff supraspinatus tendon in the shoulder.^[28] In another study on the supraspinatus tendon, Chang et al. developed a computer-aided diagnosis system to improve the diagnosis of rotator cuff lesions and their classification into relevant pathological categories such as calcific tendonitis, inflammation, or thickness tears [Table 1]. They recommended that their model could be used as a suggestive diagnostic tool by radiologists whose diagnostic accuracy may be lower than an orthopedic surgeon with relevant experience in shoulder surgeries.

Natural language processing (NLP) in musculoskeletal radiology

In addition to the literature discussed above, some studies provide a unique perspective on AI applications in musculoskeletal radiology using AI models indirectly on textual reports and protocols created by radiologists instead of directly on radiological images.^[7,29] For example, Wang et al. developed a NLP algorithm that successfully extracted skeletal fracture related information from radiology reports of osteoporosis patients.^[29] Annarumma et al. developed an NLP processing system to classify adult chest X-ray radiology reports as “critical,” “urgent,” “non-urgent,” and “normal.”^[30] This classification was followed by the use of a CNN algorithm to analyze the chest X-ray images and further prioritize them based on the abnormalities detected. This way, AI-based NLP processing in concert with image analysis could be used for triage (that is, classification of patients based on the urgency of treatment so as to optimize the functioning of a medical facility).

Some other studies in the area of NLP have used AI methods to determine which musculoskeletal imaging protocols should be used for MRI (regular or tumor oriented) and whether or not intravenous contrast is needed when performing MRI.^[7,31,32]

Based on the studies discussed in this section, it is clear that all commonly used modalities for musculoskeletal radiology can be improved in terms of accuracy, speed, and resolution with the help of AI-based techniques, which is the foremost advantage of using AI in this field [Table 1]. By improving image pre-processing, image acquisition, image analysis, as well as interpretation of textual radiological reports, AI-based methods can either assist radiologists in their interpretation of radiological data or perform automatic diagnoses to varying degrees of accuracy.