The InnerEye project at Microsoft Research has developed a search engine for medical images of the human body. It automatically searches for and identifies organs in CT scans and Microsoft is training it to recognise types of brain tumours. The software is designed to help doctors navigate and work with 3-D medical information more efficiently.
A CT scan is made up of many X-rays generated images, cross sections that are combined to build a 3-D image. Although a great diagnostic tool, Antonio Criminisi the project leader at Microsoft Research Cambridge says it is difficult to navigate through a CT scan to find the source of a problem. His team has developed software that uses both 2-D and 3-D pictures and indexes them together.
The experts marked all major organs on hundreds of CT scans and then trained machine-learning algorithms, computer programmes that can mimic intelligent human behaviour and subsequently improve on actions, to recognise important characteristics in them. When a scan is loaded into the software it uses patterns of light and dark on the scans to identify particular structures. This takes only a few seconds. The program then indexes the data and creates a list of identified organs as a table of hyperlinks on the side of the screen.
Doctors can click on the name of the organ of interest and bring up a clear image of it without having to sort through the pictures manually. 2-D and 3-D views of the organs in the area are shown, which can be navigated via a touch-screen. New scans or past ones from other patients can be matched up alongside the generated ones to help glean information regarding the progression of a disease.
The InnerEye project is a collaborative one with doctors at Addenbrookes Hospital in Cambridge and is now exploring the use of voice and gestures to control the system. This is technology used by the X-box Kinect controller. Kinect is based on a webcam-style add on for the console which allows gamers to control and interact with the X-box using only gestures and spoken commands.
It is hoped this technology will enable surgeons to refer to images in the operating theatre without having to touch a potentially dirty keyboard or mouse. Its disease-spotting ability is also under development with on-going research trying to train the system to detect differences between grades of glioma – a type of brain tumour.
Criminisi’s team’s long-term aim is to integrate the new indexed images into already existing databases so that the system can be implemented on a large scale across hospitals.
Image by Stephen Fruitsmaak
Other Elements articles in which you might be interested:
What people are saying