Russian AI passes medical exam
A neural network model designed by Russia’s Sberbank has managed to pass a medical doctor exam which is usually given to students who have completed six years of medical school, Sber’s press service told Izvestiya newspaper on Tuesday.
According to the outlet, GigaChat managed to complete a 100 question test, scoring 82%, as well as an oral exam which requires the examinee to solve three situational tasks relating to surgery, therapy, obstetrics and gynecology and answer questions relating to them.
In one, the program was asked to establish a diagnosis, create a treatment plan and prescribe additional examinations.
The developers of the model have said that training the algorithm to pass the test took six months, during which it was given a 42 GB dataset which included educational and methodological material, articles and manuals used by Russian medical students and anonymized medical data.
Despite the impressive results, experts nevertheless insist that AI is still far from replacing actual doctors. Sergey Zhdanov, the director of Sberbank’s Health Industry Center, has suggested that in the future, the GigaChat model could become the basis for creating a doctor and patient assistant, which would be capable of providing new conditions for healthcare and help clinicians in their everyday work.
However, while AI is essentially a system capable of examining a specific dataset at a speed impossible for humans, it also means that such models cannot yet come to conclusions that are outside the scope of that dataset, Ruslan Permyakov, an expert at the Competence Center of the National Technology Initiative told Izvestia.
“Let’s say we have a system that has been taught to classify apples and pears… Such a system would be able to determine what is in an apple and what is a pear and at what stage of ripeness the fruit is at much faster than a human. However, if we ask it to analyze an eggplant, we can be 100% sure that it will give a wrong answer because the system hasn’t been taught to say ‘I don’t know,’” Permyakov explained, noting that this is the weakness of all current Artificial Intelligence systems.
Sber, the largest banking and financial service company in Russia, officially launched GigaChat in April 2023, positioning it as an alternative to ChatGPT. As of January 2024, the chatbot has more than 2.5 million users.
Since the advent of generative AI programs like ChatGPT, such systems have been able to pass a number of tests designed for humans in fields like law, business, medicine and others. In October, for example, the Russian YandexGPT 2 neural network managed to pass the country’s Unified State Exam which included a test on literature - considered to be particularly difficult for algorithmic networks to solve.
