Researchers in China have tested a new type of eye clinic designed to make extensive use of artificial intelligence, while keeping human doctors involved in patient care. The clinic, known as the AI-Agent Augmented Tsinghua Eye Clinic, or AI-TEC, was developed by researchers from the Beijing Visual Science and Translational Eye Research Institute. AI was used across much of the patient journey, from pre-consultation and eye scans to follow-up care.
The research, published in Nature Medicine, found that the AI's performance improved significantly after ophthalmologists supplied it with 1,426 high-quality eye scan images with accurate diagnoses.
The system had initially been trained using nearly 27,000 images, but the researchers found that the smaller set of better-quality and more carefully labelled images produced better results.
After the updated training, the AI achieved an AUROC score of more than 0.93 when identifying eye diseases such as glaucoma and age-related macular degeneration. This was broadly comparable with other advanced eye-scanning systems.
However, the trial also showed that getting AI to work effectively in a real clinic is not simply a question of how accurate the technology is.
Use of the AI system dropped considerably during the trial. After five months, it was used in just 41 of 1,113 examinations, or 3.8 per cent. The figure rose to 259 of 1,126 examinations, or 23 per cent, after researchers made the system faster and reduced the amount of manual work and the number of clicks needed by staff.
The researchers said the experience showed that AI-based healthcare depends on several factors, including the quality of data, how easily the technology fits into clinical work, involvement from doctors, proper monitoring and whether it provides clear benefits to patients.
They also found that doctors need to provide feedback quickly so that AI systems can be improved. Delayed feedback, arriving weeks later, may make it harder to adapt the technology to the needs of a busy clinic.
The study also highlights an important difference between AI and doctors. AI can focus on whether a scan shows a particular disease, while doctors also consider a patient's symptoms and wider clinical situation.
The researchers therefore say that the success of AI in healthcare should not be judged only by test scores. What matters is whether it improves clinical workflows and, ultimately, patient health.
The researchers described the trial as an early step towards AI-focused healthcare. The findings suggest that high-quality data, practical systems and close cooperation between doctors and AI researchers will be important if similar clinics are to be developed in the future.
