Developing an Artificial Intelligence Project in your Radiology Department

THE PROBLEM – IDENTIFY REAL-WORLD PROBLEMS SOLVABLE BY AI SOLUTIONS

AI as we know is really artificial narrow intelligence (AIN) and is only good at very specific tasks.^[4] AI is very good at answering one question, the more questions you ask, the more work is required to improve the accuracy of the algorithm.^[4] Some of the best-known AI companies only focus on very specific tasks; for example, is the chest radiograph normal or abnormal?, is there osteoporosis?, and is there a fatty liver?.^[5,6] Therefore, consider a high impact and specific question, which you would like AI software to answer. AI solutions may work to improve working practices in terms of time, efficiency, cost, quality, radiation dose, or any other measurable factor.^[7]

THE TEAM – INDIVIDUALS AND COLLABORATION

A variety of skills and an extensive knowledge base is required when it comes to AI development in radiology. These include:

• Image interpretation and clinical understanding

• Picture archiving and communication system (PACS)/ network skills

• Coding using a language optimized for AI development, for example, Python^[8]

These skills may well be present in a radiology department or can be sought from elsewhere. Most universities will have computer or data science students eager to get involved and university collaboration can be very beneficial. Depending on funding of a project, it may be feasible to hire skills such as an experienced programmer to assist with the training and testing of a neural network.

DATA – SECURITY, HARVESTING, AND LABELLING

A high-quality labeled database is necessary to train and test a neural network.^[4] However, achieving that database involves consideration of data security, harvesting, and labeling itself.

Security: In recent times, there has been a big emphasis on the security and handling of data, and this should be at the forefront when utilizing patient data for software development. DICOM images, for example, can be stripped of their patient identifiable data during the downloading process.^[9]

Labeling: The majority of deep learning in radiological imaging utilizes supervised learning. This means that the AI algorithm is trained against data to which the answer is shown to the algorithm. This way the algorithm will learn that these data equal this answer, for example, this radiograph equals fracture.^[10] Labeling data can be as simple as assigning an image to a category or providing outlines to demonstrate segmentation, complex measurements, 3D volume labeling, or time series labeling. Labeling and segmentation software can be developed in-house, purchased off-the-shelf, or available free as open source. Inaccuracies in labeling will lead to a poor performing algorithm, which can be very difficult to fix downstream.^[11]

Harvesting: Large volumes of data are generally required to train AI algorithms (neural networks), and there are many ways to identify relevant data on a radiology information system.^[11] For those with basic structured query language (SQL) programming knowledge, it is often possible to perform SQL searches of PACS systems to request all potentially usable data. Harvested data can be stored securely on local servers or on the cloud.

KIT – HARDWARE AND SOFTWARE

Deployment of trained AI models requires very little equipment, and most computers would be able to run a neural network with little difficulty. Training a neural network is, however, generally a more computing hungry task. The amount of power required varies dependent on the amount of data available for training and testing and the complexity of the desired model. A very basic model can be trained in a reasonable timeframe on almost any everyday computer.^[12]

More complex models can be trained for relatively low cost using dedicated graphics processing units (GPUs). These can be installed in desktop computers with little difficulty or come included in pre-built units, under the “gaming” or “workstation” banners. These GPUs increase the number of cores in comparison to central processing unit (CPU), often by multiple orders of magnitude.^[8] Software packages optimized for use with GPUs can leverage this greater number of cores and train neural networks in a more efficient manner.^[12]

The choice of which hardware to utilize is, therefore, highly dependent on the complexity of the problem and the amount of data necessary to train the model accurately. A good option for many developers is to use cloud-based computing services, such as Microsoft Azure or Amazon Web Services. Licenses and cloud computing power can be purchased for relatively low cost making high-powered computing available to most radiology departments. Specialist high-end servers optimized for AI can be incorporated into a PACS server system to handle training data on a large scale.

AI development is most commonly performed in the Python or R programming languages. Many software environments, which are used for the AI algorithm training, are readily available to deploy AI optimized packages such as TensorFlow and Keras which are open source.^[8]

NEURAL NETWORK – TRAINING AND TESTING

A neural network mimics the biology of learning that occurs in nature. In its simplest form, it is a vast collection of nodes often organized in layers that receive input from a prior layer, performs a local calculation – usually involving multiplication by a “weight,” and then provides an output to the next layer.^[13] The output layer is then compared to what is expected/desired (a representation of the label). The difference or loss is calculated. This is then propagated back through the network (back propagation) altering the weights of the network nodes in a way to best achieve the desired outcome. The process is repeated (iteration), with the next piece of labeled data, improving the accuracy of the network at predicting the label of a previously unseen dataset.^[13]

To make things easier for small-scale development of neural networks, there is a variety of pre-existing architectures that are freely available, validated, and easy to implement. Each of these neural networks are highly customizable to the needs of a developer, and as a result, there are many variables that can be adjusted, making an almost infinite combination of variables that can be used to train a network.^[14,15] Often, it is not completely understood why one set of variables improves the accuracy over another set, and experimentation is often the best way to achieve higher accuracy.^[13]

Good organization, record keeping, and fast servers can help, ensuring many experiments can be performed in a short space of time and the combination of variables that produce the best outcome can be decided on.

VALIDATION – CLINICAL TESTING AND REGULATORY BODY APPROVALS

Before implementation and a neural network “going live” in any clinical radiology department, it must be validated.^[16] This should ideally be done in a simulated environment mimicking as closely as possible to actual conditions – therefore mitigating any potential risk to patients. This process can be used to not only evaluate the model and its accuracy but how it fits into the workflow within a department and end-user acceptability. All medical devices and technologies are subject to regulatory body approvals, and neural networks are no different.^[17] These regulatory bodies vary dependent on geographical region. Presentation of good validation data will greatly increase the likelihood that a model is approved.

GOVERNANCE – PATIENT SAFETY AND COST- EFFECTIVENESS

Ongoing assessment and analysis of a model are crucial to ensure patient safety. Protocols and procedures need to be developed to ensure rigorous audit trails and accountability.^[16] AI solutions, which directly impact patient care, will be under greater scrutiny, for example, production of a clinically actionable automated report. Those that improve workflow but have clinical radiologist supervision throughout its usage are a good starting point as they are less likely to cause patient harm.

Continuing assessment is also important to promote further learning of a model. As a model evaluates previously unseen data, this can be used to provide further labeled data to increase accuracy.^[18] The greatest costs when it comes to AI in radiology is the development of a neural network, but once implemented the model can then be used to deliver cost savings to the department. Evaluating these cost savings and cost-effectiveness analyses can highlight the benefits of AI development hence making a case for the next development project.^[18]

Applications in musculoskeletal imaging

We will now discuss some examples of AI applications in musculoskeletal imaging which can be developed with in a department. Although there are other groups working on similar projects, a good algorithm that is trained and validated on a population which it will be implemented on should be superior to one developed on an unrelated demographic.^[11]

Bone age: This is a perfect case of use of AI in radiology. Calculating bone age using hand radiographs is tedious, subjective, and uses normal ranges developed using an American population more than half a century ago,^[19] whereas AI algorithms now do this accurately, instantly, with results relevant to the local population and produce detailed metrics.^[20] There are commercial products available, however, this can be done very accurately as a homegrown system.

Fractures on plain films: With a high volume of plain films, most centers can produce good data sets. As a starting project, we would recommend one region, for example, ankle, wrist, or pelvis to improve accuracy. The labeling of fractures can be done by drawing a bounding box around the fracture or segmentation of the fracture. As a first step, it would be wise to develop a system which simply alerts the system of an abnormal film with a possible diagnosis rather than an autonomous reporting system.^[19,21]

Accelerating MRI imaging: This is a promising area of AI development and will likely be integrated in future MRI scanners from the various vendors. www.FastMRI.orgis a challenge from a Facebook and NYU Langone Health collaboration, where there is an excellent MRI knee data set available to use. The challenge involves MR image reconstruction from 4× or 8× undersampled k-space data.^[22] This project would require an expert in MRI physics.

Bone mineral density from CT: It has been shown that bone density can be accurately calculated using data from CT imaging and thus identify patients with osteoporosis who may in future present with debilitating insufficiency fractures.^[23] This would need a data set of patients who have had a DEXA scan (dual-energy X-ray absorptiometry) and a CT abdomen and pelvis within few months of each other.^[23]