vitjВрач и информационные технологииMedical Doctor and Information Technologies1811-01932413-5208Pirogov National Medical and Surgical Center10.25881/18110193_2022_2_4vitj-123Research ArticleМНЕНИЕ РЕДАКЦИИEDITORIALПрозрачный искусственный интеллект для медициныExplainable artificial intelligence for medicineКарповО. Э.KarpovO. E.

академик РАН, д.м.н., профессор

Москва

Academician of the RAS, Dr. Sci. (Medicine), Professor

Moscow

nmhc@mail.ruАндриковД. А.AndrikovD. A.

к.т.н.

Москва

PhD

Moscow

andrikovda@immersmed.ruМаксименкоВ. А.MaksimenkoV. A.

д.ф.м.н.

г. Казань

Dr. Sci.

Kazan

maximenkovl@gmail.comХрамовА. Е.HramovA. E.

д.ф.м.н., профессор

г. Калининград

Dr. Sci., Professor

Kaliningrad

hramovae@gmail.comФГБУ «Национальный медико-хирургический Центр имени Н.И. Пирогова»РоссияPirogov National Medical and Surgical CenterRussian FederationООО «Иммерсмед»РоссияImmersmed LLCRussian FederationУниверситет ИннополисРоссияInnopolis UniversityRussian FederationБалтийский федеральный университет им. И. КантаРоссияImmanuel Kant Baltic Federal UniversityRussian Federation20223003202502411Copyright © Карпов О.Э., Андриков Д.А., Максименко В.А., Храмов А.Е., 20252025Карпов О.Э., Андриков Д.А., Максименко В.А., Храмов А.Е.Karpov O.E., Andrikov D.A., Maksimenko V.A., Hramov A.E.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://www.vit-j.ru/jour/article/view/123

Успех и массовое применение современных технологий искусственного интеллекта (ИИ) и, в частности, методов глубокого обучения нейронных сетей привели нас к четкому пониманию двух основных проблем: проблемы ошибок (проблема надежности) и проблемы явного объяснения решений, принимаемых ИИ (проблема прозрачности). Эти проблемы тесно связаны между собой: необъяснимые ошибки ИИ могут повторяться снова и снова. Это совершенно неприемлемо с точки зрения применения ИИ в здравоохранении, потому что является критичным для жизни и здоровья пациентов. Если оставить проблемы ошибок и объяснимости нерешенными, то непрозрачность решений ИИ может привести к отказу или существенному ограничению от использования систем ИИ в задачах медицины. В данном комментарии мы обсуждаем проблемы прозрачного объяснимого интеллекта для медицины и рассматриваем различные подходы к их решению.

The success and wide-range applications of artificial intelligence (AI) technologies and, in particular, deep learning neural networks methods have led us to a clear understanding of two main problems: the problem of errors (the reliability problem) and the problem of explicitly explaining the decisions made by AI (the explanability problem). These problems are closely related: unexplained AI errors can happen again and again. This is completely unacceptable from the perspective of AI applications in health care because it is critical to the lives and health of patients. If left unresolved, the problems of error and explainability can lead to the rejection or significant restriction of AI systems in medical applications. In this paper, we discuss the problems of explainable artificial intelligence (XAI) for medicine and consider different approaches to solving them.

искусственный интеллектцифровые медицинские технологиибиомаркернейроинтерфейсartificial intelligencedigital healthcarebiomarkerneurointerfaceРабота поддержана Президентским грантом (проект НШ-589.2022.1.2).ReferencesCB Insights Research. Healthcare remains the hottest AI category for deals. April 12, 2017. [Last accessed on 2022 Feb 11]. Available at: https://www.cbinsights.com/research/artificial-intelligence-healthcare-startups-investors.CB Insights Research. Healthcare remains the hottest AI category for deals. April 12, 2017. [Last accessed on 2022 Feb 11]. Available at: https://www.cbinsights.com/research/artificial-intelligence-healthcare-startups-investors.На основе данных State of AI Q3’21. [Last accessed on 2022 March 22]. Available at: https:// www.cbinsights.com/research/report/ai-trends-q3-2021.На основе данных State of AI Q3’21. [Last accessed on 2022 March 22]. Available at: https://www.cbinsights.com/research/report/ai-trends-q3-2021.He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nature medicine. 2019; 25(1): 30-36.He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nature medicine. 2019; 25(1): 30-36.Johnson Kevin B, et al. Precision medicine, AI, and the future of personalized health care. Clinical and translational science. 2021;14(1): 86-93.Johnson Kevin B, et al. Precision medicine, AI, and the future of personalized health care. Clinical and translational science. 2021;14(1): 86-93.Карпов О.Э., Храмов А.Е. Прогностическая медицина // Врач и информационные технологии. — 2021. — №3. — С.20-37. doi: 10.25881/18110193_2021_3_20.Karpov OE, Hramov AE. Prognosticheskaya medicina. Vrach i informacionnye tekhnologii. 2021; 3: 20-37. (In Russ). doi: 10.25881/18110193_2021_3_20.Strabo K, Tavel JA. What are biomarkers? Current Opinion in HIV and AIDS. 2010; 5(6): 463.Strabo K, Tavel JA. What are biomarkers? Current Opinion in HIV and AIDS. 2010; 5(6): 463.Toprol E.J. Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again. Basic Books, New York, NY, 2019.Toprol E.J. Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again. Basic Books, New York, NY, 2019.Kundu S. AI in medicine must be explainable. Nature Medicine. 2021; 27(8): 1328.Kundu S. AI in medicine must be explainable. Nature Medicine. 2021; 27(8): 1328.Ashby WR. An Introduction to Cybernetics. Chapman & Hall Ltd., London, UK, 1957.Ashby WR. An Introduction to Cybernetics. Chapman & Hall Ltd., London, UK, 1957.Friedman CP. A «fundamental theorem» of biomedical informatics. J. Am. Med. Inform. Assoc. 2009; 16: 169-170.Friedman CP. A «fundamental theorem» of biomedical informatics. J. Am. Med. Inform. Assoc. 2009; 16: 169-170.Karpov OE, Grubov VV, Maksimenko VA, Utaschev N, Semerikov VE, Andrikov DA, Hramov AE. Noise amplification precedes extreme epileptic events on human EEG. Physical Review E. 2021; 103: 022310.Karpov OE, Grubov VV, Maksimenko VA, Utaschev N, Semerikov VE, Andrikov DA, Hramov AE. Noise amplification precedes extreme epileptic events on human EEG. Physical Review E. 2021; 103: 022310.Perera P, Patel VM. Learning deep features for one-class classification. IEEE Transactions on Image Processing. 2019; 28(11): 5450-5463.Perera P, Patel VM. Learning deep features for one-class classification. IEEE Transactions on Image Processing. 2019; 28(11): 5450-5463.Кучин А.С., Грубов В.В., Максименко В.А., Утяшев Н.П. Автоматизированное рабочее место врача эпилептолога с возможностью автоматического поиска приступов эпилепсии // Врач и информационные технологии. — 2021. — №3. — С.62-73. doi: 1025881/18110193_2021_3_62.Kuchin AS, Grubov VV, Maksimenko VA, Utyashev NP. Avtomatizirovannoe rabochee mesto vracha epileptologa s vozmozhnost’yu avtomaticheskogo poiska pristupov epilepsii. Vrach i informacionnye tekhnologii. 2021; 3: 62-73. (In Russ). doi: 1025881/18110193_2021_3_62.Chaudhary U, Birbaumer N, Ramos-Murguialday A. Brain–computer interfaces for communication and rehabilitation. Nature Reviews Neurology. 2016; 12(9): 513-525.Chaudhary U, Birbaumer N, Ramos-Murguialday A. Brain–computer interfaces for communication and rehabilitation. Nature Reviews Neurology. 2016; 12(9): 513-525.Shah RB, Patel M, Maahs DM, Shah V N. Insulin delivery methods: Past, present and future. International journal of pharmaceutical investigation. 2016; 6(1): 1.Shah RB, Patel M, Maahs DM, Shah V N. Insulin delivery methods: Past, present and future. International journal of pharmaceutical investigation. 2016; 6(1): 1.Ricci RP, Morichelli L, Santini M. Home monitoring remote control of pacemaker and implantable cardioverter defibrillator patients in clinical practice: impact on medical management and health-care resource utilization. Europace. 2008; 10(2): 164-170.Ricci RP, Morichelli L, Santini M. Home monitoring remote control of pacemaker and implantable cardioverter defibrillator patients in clinical practice: impact on medical management and health-care resource utilization. Europace. 2008; 10(2): 164-170.Hramov AE, Maksimenko VA, Pisarchik AN. Physical principles of brain–computer interfaces and their applications for rehabilitation, robotics and control of human brain states. Physics Reports. 2021; 918: 1-133.Hramov AE, Maksimenko VA, Pisarchik AN. Physical principles of brain–computer interfaces and their applications for rehabilitation, robotics and control of human brain states. Physics Reports. 2021; 918: 1-133.Sarkisova K, van Luijtelaar G. The WAG/Rij strain: a genetic animal model of absence epilepsy with comorbidity of depressiony. Progress in Neuro-Psychopharmacology and Biological Psychiatry. 2011; 35(4): 854-876.Sarkisova K, van Luijtelaar G. The WAG/Rij strain: a genetic animal model of absence epilepsy with comorbidity of depressiony. Progress in Neuro-Psychopharmacology and Biological Psychiatry. 2011; 35(4): 854-876.van Luijtelaar G, Lüttjohann A, Makarov VV, Maksimenko VA, Koronovskii AA, Hramov AE. Methods of automated absence seizure detection, interference by stimulation, and possibilities for prediction in genetic absence models. Journal of Neuroscience Methods. 2016; 260: 144-158.van Luijtelaar G, Lüttjohann A, Makarov VV, Maksimenko VA, Koronovskii AA, Hramov AE. Methods of automated absence seizure detection, interference by stimulation, and possibilities for prediction in genetic absence models. Journal of Neuroscience Methods. 2016; 260: 144-158.Мaksimenko VA, Heukelum S, Makarov VV, Kelderhuis J, Lüttjohann A, Koronovskii AA, Hramov AE, Luijtelaar G. Absence Seizure Control by a Brain Computer Interface. Scientific Reports. 2017; 7: 2487.Мaksimenko VA, Heukelum S, Makarov VV, Kelderhuis J, Lüttjohann A, Koronovskii AA, Hramov AE, Luijtelaar G. Absence Seizure Control by a Brain Computer Interface. Scientific Reports. 2017; 7: 2487.Budde B, Maksimenko V, Sarink K, Seidenbecher T, van Luijtelaar G, Hahn T, Pape HC, Lüttjohann Seizure prediction in genetic rat models of absence epilepsy: improved performance through multiple-site cortico-thalamic recordings combined with machine learning. Eneuro. 2022; 9(1).Budde B, Maksimenko V, Sarink K, Seidenbecher T, van Luijtelaar G, Hahn T, Pape HC, Lüttjohann Seizure prediction in genetic rat models of absence epilepsy: improved performance through multiple-site cortico-thalamic recordings combined with machine learning. Eneuro. 2022; 9(1).

The authors declare that there are no conflicts of interest present.