AI outperforms physicians in telemedicine, Israeli researchers say

Furthermore, only 2.8% of AI suggestions were rated as potentially harmful, while 4.6% of the physicians’ decisions fell into the same category.

By Pesach Benson, TPS

Artificial intelligence-based diagnostic and treatment recommendations outperformed human physicians in an Israeli study released on Tuesday, suggesting that AI could serve as a valuable decision-support tool in medical practice, particularly in virtual urgent care settings.

The study, led by Prof. Dan Zeltzer, a digital health expert from Tel Aviv University’s Berglas School of Economics, was conducted at Cedars-Sinai Connect, a virtual urgent care clinic in Los Angeles.

The clinic, operated in collaboration with Israeli startup K Health, offers telemedicine consultations, integrating an AI system into its workflow to assist doctors in diagnosing and treating patients.

The findings were published in the peer-reviewed Annals of Internal Medicine and presented at the American College of Physicians (ACP) annual conference.

“Recently, Cedars-Sinai integrated an AI system that uses machine learning to conduct initial patient intake, incorporating medical record data and providing physicians with detailed diagnostic and treatment suggestions,” Zeltzer explained.

“The system offers recommendations only when its confidence level is high, with no suggestions in about one in five cases.”

The study evaluated 461 online clinic visits over the summer of 2024, focusing on adult patients with common symptoms such as respiratory, urinary, eye, vaginal, and dental issues.

In each case, the AI system provided initial recommendations, followed by a video consultation with a physician.

A panel of four experienced doctors then reviewed and rated the recommendations based on a four-point scale: optimal, reasonable, inadequate, or potentially harmful.

The results were striking: AI recommendations were rated as optimal in 77% of cases, compared to 67% of the physicians’ decisions.

Furthermore, only 2.8% of AI suggestions were rated as potentially harmful, while 4.6% of the physicians’ decisions fell into the same category.

In 68% of cases, the AI and physician received the same score, but in 21% of cases, the AI’s recommendation was rated higher, and in 11% of cases, the physician’s decision was deemed superior.

The study’s evaluators noted several reasons why the AI system outperformed human physicians.

One key factor was that the AI adhered more strictly to medical association guidelines, such as avoiding unnecessary prescriptions, like antibiotics for viral infections.

Additionally, the AI was able to process a wider range of data from patients’ medical histories, identifying recurring conditions and recognizing symptoms that could signal more serious health issues.

For example, the AI was particularly effective in identifying eye pain in contact lens users, which could indicate a potential infection.

Physicians, however, had an advantage in assessing a patient’s real-time condition and using clinical judgment to adapt to nuances in the symptoms presented.

For instance, while an AI system might refer a COVID-19 patient with shortness of breath to the emergency room, a physician could evaluate the symptom in context, determining whether the issue was mild respiratory congestion instead.

Zeltzer stressed that while the study demonstrated the accuracy of AI in diagnostic and treatment recommendations, it had limitations.

“We do not know which physician reviewed the AI’s recommendations or how much they relied on them. Therefore, this study only measures the algorithm’s accuracy, not its direct impact on physician decision-making,” he said.

The study also highlighted the real-world applicability of AI in medical settings, distinguishing it from previous research that focused on artificial cases or textbook examples. “The conditions we studied represent about two-thirds of the clinic’s case volume,” Zeltzer explained.

“This makes our findings particularly meaningful when considering AI as a decision-support tool in everyday medical practice.”

Looking ahead, Zeltzer envisions a future where AI plays an increasingly significant role in medical decision-making, working alongside doctors to highlight critical data and reduce human error.

“The study shows that AI can be an effective assistant in medical diagnostics and treatment, but many questions remain about the best way to integrate it into healthcare practices,” he concluded. “Ultimately, the goal is to strike the right balance between human expertise and AI to improve patient outcomes.”