vitjВрач и информационные технологииMedical Doctor and Information Technologies1811-01932413-5208Pirogov National Medical and Surgical Center10.25881/18110193_2025_4_86vitj-276Research ArticleОРИГИНАЛЬНЫЕ ИССЛЕДОВАНИЯORIGINAL RESEARCHСравнение точности прогнозирования фатальных осложнений сердечно-сосудистых заболеваний с использованием шкалы SCORE и модели машинного обученияComparison of prognostic accuracy of score scale and a machine learning model in predicting fatal cardiovascular complicationsЕрмакА. Д.ErmakA. D.

г. Петрозаводск

Petrozavodsk

andrewermakwork@gmail.comГавриловД. В.GavrilovD. V.

г. Петрозаводск

Petrozavodsk

dgavrilov@webiomed.ruКузнецоваТ. Ю.KuznetsovaT. Yu.

д.м.н., доцент

г. Петрозаводск

DSc, Associate Professor

Petrozavodsk

eme@sampo.ruАндрейченкоА. Е.AndreichenkoA. E.

к.т.н.

г. Санкт-Петербург

PhD

Saint Petersburg

anna.ev.andreychenko@yandex.ruМакароваЕ. А.MakarovaE. A.

к.т.н.г. Петрозаводск

PhD

Petrozavodsk

emakarova@webiomed.ruНовицкийР. Э.NovitskiyR. E.

г. Петрозаводск

Petrozavodsk

roman@webiomed.ruГусевА. В.GusevA. V.

к.т.н.

г. Москва

PhD

Moscow

agusev@webiomed.ruООО «К-Скай»РоссияK-SkAI LLCRussian FederationФГБОУ ВО Петрозаводский государственный университетРоссияPetrozavodsk State UniversityRussian FederationУниверситет ИТМОРоссияITMO UniversityRussian FederationФГБУ «Центральный научно-исследовательский институт организации и информатизации здравоохранения» Минздрава РоссииРоссияFederal Research Institute for Health Organization and InformaticsRussian Federation202523122025048698Copyright © Ермак А.Д., Гаврилов Д.В., Кузнецова Т.Ю., Андрейченко А.Е., Макарова Е.А., Новицкий Р.Э., Гусев А.В., 20252025Ермак А.Д., Гаврилов Д.В., Кузнецова Т.Ю., Андрейченко А.Е., Макарова Е.А., Новицкий Р.Э., Гусев А.В.Ermak A.D., Gavrilov D.V., Kuznetsova T.Y., Andreichenko A.E., Makarova E.A., Novitskiy R.E., Gusev A.V.This work is licensed under a Creative Commons Attribution 4.0 License.https://www.vit-j.ru/jour/article/view/276

Выявление пациентов высокого риска фатальных осложнений сердечно-сосудистых заболеваний (ССЗ) является одной из важнейших задач для сокращения предотвратимой заболеваемости и смертности от ССЗ. Для этого широко применяются различные алгоритмы оценки риска и шкалы, недостатком которых является использование ограниченного набора предикторов и невысокая точность. Методы машинного обучения (МО) открывают возможности для устранения данных недостатков и персонализации оценки сердечно-сосудистого риска.

Цель исследования

Цель исследования: сравнить точность шкалы SCORE и моделей МО в задаче прогнозирования фатальных осложнений ССЗ.

Материалы и методы

Материалы и методы. Проведено многоцентровое ретроспективное исследование (1999–2018 гг.). Включен 3891 случай лечения 1064 пациентов Российской Федерации 40–69 лет. Для прогнозирования применялись логистическая регрессия, ансамблевые методы МО и Multi-layer Perceptron. Сравнение с SCORE проводилось на независимом валидационном наборе, включавшем 440 записей.

Результаты

Результаты. Лучшие показатели точности продемонстрировала модель МО CatBoost (AUROC 0,879; чувствительность 0,938; специфичность 0,777). На валидации CatBoost показал сопоставимую со SCORE дискриминацию, но превосходил шкалу по специфичности (0,653 против 0,408) и точности (0,673 против 0,45) при разделении пациентов на группы низкого и умеренного риска. Ключевыми предикторами для модели были пол, возраст, курение, систолическое давление, индекс массы тела, ЧСС и липидный профиль.

Заключение

Заключение. Модель МО превзошла шкалу SCORE по точности прогнозирования фатальных сердечно-сосудистых событий. Использование МО в прогнозировании сердечно-сосудистого риска позволяет повысить эффективность профилактики ССЗ и способствует персонализированному ведению пациентов.

Identifying patients at high risk for fatal cardiovascular disease (CVD) complications is a critical task in reducing preventable CVD morbidity and mortality. Various risk assessment algorithms and scores are widely used for this purpose, but their limitations include a limited set of predictors and low accuracy. Machine learning methods offer the potential to address these shortcomings and personalize cardiovascular risk assessment.Objective: To compare the accuracy of the SCORE scale and machine learning models in predicting fatal cardiovascular complications.Materials and Methods: A multicenter retrospective study was conducted (1999–2018), including 3,891 treatment cases of 1,064 patients aged 40–69 years in the Russian Federation. Logistic regression, ensemble machine learning (ML) methods, and Multi-Layer Perceptron were used for forecasting. Comparison with SCORE was performed on an independent validation set consisting of 440 records.Results: The CatBoost ML model demonstrated the best accuracy (AUROC 0.879; sensitivity 0.938; specificity 0.777). During validation, CatBoost demonstrated comparable discrimination to SCORE but outperformed the scale in specificity (0.653 vs. 0.408) and accuracy (0.673 vs. 0.45) when referencing patients to lowand intermediate-risk groups. Key predictors for the model were gender, age, smoking, systolic blood pressure, body mass index, heart rate, and lipid profile.Conclusion: The machine learning model outperformed the SCORE scale in predicting fatal cardiovascular events. The use of machine learning in predicting cardiovascular risk can improve the effectiveness of CVD prevention and facilitate personalized patient care.

сердечно-сосудистый рискSCOREмашинное обучениеCatBoostпрофилактикаcardiovascular riskSCOREmachine learningCatBoostpreventionReferencesRoth GA, Mensah GA, Johnson CO, Addolorato G, et al. Global Burden of Cardiovascular Diseases and Risk Factors, 1990-2019: Update From the GBD 2019 Study. Journal of the American College of Cardiology. Elsevier Inc. 2020; 76: 2982-3021.Roth GA, Mensah GA, Johnson CO, Addolorato G, et al. Global Burden of Cardiovascular Diseases and Risk Factors, 1990-2019: Update From the GBD 2019 Study. Journal of the American College of Cardiology. Elsevier Inc. 2020; 76: 2982-3021.Damen JAAG, Hooft L, Schuit E, Debray TPA, et al. Prediction models for cardiovascular disease risk in the general population: Systematic review. BMJ (Online). BMJ Publishing Group. 2016; 353.Damen JAAG, Hooft L, Schuit E, Debray TPA, et al. Prediction models for cardiovascular disease risk in the general population: Systematic review. BMJ (Online). BMJ Publishing Group. 2016; 353.Gavrilov DV, Gusev AV, Nikulina AV, Kuznetsova TY, Drapkina OM. Correctness of cardiovascular risk assessment in daily clinical practice. Profilakticheskaya Meditsina. 2021; 24(4): 69-75.Gavrilov DV, Gusev AV, Nikulina AV, Kuznetsova TY, Drapkina OM. Correctness of cardiovascular risk assessment in daily clinical practice. Profilakticheskaya Meditsina. 2021; 24(4): 69-75.Boytsov SA, Pogosova NV. Cardiovascular prevention 2022. Russian national guidelines: Russian Society of Cardiology, National Society of Preventive Cardiology. Russian Journal of Cardiology. 2023; 28(5).Boytsov SA, Pogosova NV. Cardiovascular prevention 2022. Russian national guidelines: Russian Society of Cardiology, National Society of Preventive Cardiology. Russian Journal of Cardiology. 2023; 28(5).Global Effect of Modifiable Risk Factors on Cardiovascular Disease and Mortality. New England Journal of Medicine [Internet]. 2023; 389(14): 1273-85. Available from: http://www.nejm.org/doi/10.1056/NEJMoa2206916.Global Effect of Modifiable Risk Factors on Cardiovascular Disease and Mortality. New England Journal of Medicine [Internet]. 2023; 389(14): 1273-85. Available from: http://www.nejm.org/doi/10.1056/NEJMoa2206916.Conroy RM, Pyörälä K, Fitzgerald AP, Sans S, et al. Estimation of ten-year risk of fatal cardiovascular disease in Europe: The SCORE project. Eur Heart J. 2003; 24(11): 987-1003.Conroy RM, Pyörälä K, Fitzgerald AP, Sans S, et al. Estimation of ten-year risk of fatal cardiovascular disease in Europe: The SCORE project. Eur Heart J. 2003; 24(11): 987-1003.Tokgozoglu L, Torp-Pedersen C. Redefining cardiovascular risk prediction: Is the crystal ball clearer now? European Heart Journal. Oxford University Press. 2021; 42: 2468-71.Tokgozoglu L, Torp-Pedersen C. Redefining cardiovascular risk prediction: Is the crystal ball clearer now? European Heart Journal. Oxford University Press. 2021; 42: 2468-71.Kapoor S, Narayanan A. Leakage and the reproducibility crisis in machine-learning-based science. Patterns. 2023; 4(9).Kapoor S, Narayanan A. Leakage and the reproducibility crisis in machine-learning-based science. Patterns. 2023; 4(9).Com YD, Simonoff JS. An Investigation of Missing Data Methods for Classification Trees Applied to Binary Response Data Yufeng Ding. Journal of Machine Learning Research. 2010; 11.Com YD, Simonoff JS. An Investigation of Missing Data Methods for Classification Trees Applied to Binary Response Data Yufeng Ding. Journal of Machine Learning Research. 2010; 11.Cao XH, Stojkovic I, Obradovic Z. A robust data scaling algorithm to improve classification accuracies in biomedical data. BMC Bioinformatics. 2016; 17(1).Cao XH, Stojkovic I, Obradovic Z. A robust data scaling algorithm to improve classification accuracies in biomedical data. BMC Bioinformatics. 2016; 17(1).De Amorim LB V, Cavalcanti GDC, Cruz RMO. The choice of scaling technique matters for classification performance [Internet]. Available from: https://github.com/amorimlb/scaling.De Amorim LB V, Cavalcanti GDC, Cruz RMO. The choice of scaling technique matters for classification performance [Internet]. Available from: https://github.com/amorimlb/scaling.Weiss GM. Foundations of imbalanced learning. 2012.Weiss GM. Foundations of imbalanced learning. 2012.Akiba T, Sano S, Yanase T, Ohta T, Koyama M. Optuna: A Next-generation Hyperparameter Optimization Framework. 2019 Jul 25. Available from: http://arxiv.org/abs/1907.10902.Akiba T, Sano S, Yanase T, Ohta T, Koyama M. Optuna: A Next-generation Hyperparameter Optimization Framework. 2019 Jul 25. Available from: http://arxiv.org/abs/1907.10902.Sokolova M, Lapalme G. A systematic analysis of performance measures for classification tasks. Inf Process Manag. 2009; 45(4): 427-37.Sokolova M, Lapalme G. A systematic analysis of performance measures for classification tasks. Inf Process Manag. 2009; 45(4): 427-37.Brodersen KH, Ong CS, Stephan KE, Buhmann JM. The balanced accuracy and its posterior distribution. In: Proceedings – International Conference on Pattern Recognition. 2010. Р.3121-4.Brodersen KH, Ong CS, Stephan KE, Buhmann JM. The balanced accuracy and its posterior distribution. In: Proceedings – International Conference on Pattern Recognition. 2010. Р.3121-4.Barandela R, Sã Anchez B; JS, Garcã V, Rangel E. Rapid and Brief Communication Strategies for learning in class imbalance problems [Internet]. Pattern Recognition. 2003. Available from: www.elsevier.com/locate/patcogBarandela R, Sã Anchez B; JS, Garcã V, Rangel E. Rapid and Brief Communication Strategies for learning in class imbalance problems [Internet]. Pattern Recognition. 2003. Available from: www.elsevier.com/locate/patcogChicco D, Jurman G. The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification. BioData Min. 2023; 16(1).Chicco D, Jurman G. The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification. BioData Min. 2023; 16(1).Zoubir AM, Iskandler DR. Bootstrap methods and applications. IEEE Signal Process Mag. 2007; 24(4): 10-9.Zoubir AM, Iskandler DR. Bootstrap methods and applications. IEEE Signal Process Mag. 2007; 24(4): 10-9.Lundberg SM, Erion G, Chen H, Degrave A, et al. Explainable AI for Trees: From Local Explanations to Global Understanding [Internet]. Available from: https://github.com/suinleelab/treeexplainer-study.Lundberg SM, Erion G, Chen H, Degrave A, et al. Explainable AI for Trees: From Local Explanations to Global Understanding [Internet]. Available from: https://github.com/suinleelab/treeexplainer-study.Wester DB. Comparing treatment means: overlapping standard errors, overlapping confidence intervals, and tests of hypothesis. Biom Biostat Int J. 2018; 7(1): 73-85.Wester DB. Comparing treatment means: overlapping standard errors, overlapping confidence intervals, and tests of hypothesis. Biom Biostat Int J. 2018; 7(1): 73-85.Fischer BG, Evans AT. SpPin and SnNout Are Not Enough. It’s Time to Fully Embrace Likelihood Ratios and Probabilistic Reasoning to Achieve Diagnostic Excellence. J Gen Intern Med. 2023; 38(9): 2202-4.Fischer BG, Evans AT. SpPin and SnNout Are Not Enough. It’s Time to Fully Embrace Likelihood Ratios and Probabilistic Reasoning to Achieve Diagnostic Excellence. J Gen Intern Med. 2023; 38(9): 2202-4.Baduashvili A, Guyatt G, Evans AT. ROC Anatomy — Getting the Most Out of Your Diagnostic Test. J Gen Intern Med. 2019; 34(9): 1892-8.Baduashvili A, Guyatt G, Evans AT. ROC Anatomy — Getting the Most Out of Your Diagnostic Test. J Gen Intern Med. 2019; 34(9): 1892-8.Thomas G, Kenny LC, Baker PN, Tuytten R. A novel method for interrogating receiver operating characteristic curves for assessing prognostic tests. Diagn Progn Res. 2017; 1(1).Thomas G, Kenny LC, Baker PN, Tuytten R. A novel method for interrogating receiver operating characteristic curves for assessing prognostic tests. Diagn Progn Res. 2017; 1(1).Gill SK, Karwath A, Uh HW, Cardoso VR, et al. Artificial intelligence to enhance clinical value across the spectrum of cardiovascular healthcare. European Heart Journal. Oxford University Press. 2023; 44: 713-25.Gill SK, Karwath A, Uh HW, Cardoso VR, et al. Artificial intelligence to enhance clinical value across the spectrum of cardiovascular healthcare. European Heart Journal. Oxford University Press. 2023; 44: 713-25.Friedrich S, Groß S, König IR, Engelhardt S, et al. Applications of artificial intelligence/machine learning approaches in cardiovascular medicine: A systematic review with recommendations. European Heart Journal — Digital Health. Oxford University Press. 2021; 2: 424-36.Friedrich S, Groß S, König IR, Engelhardt S, et al. Applications of artificial intelligence/machine learning approaches in cardiovascular medicine: A systematic review with recommendations. European Heart Journal — Digital Health. Oxford University Press. 2021; 2: 424-36.Gusev AV, Gavrilov DV, Novitsky RE, Kuznetsova TY, Boytsov SA. Improvement of cardiovascular risk assessment using machine learning methods. Russian Journal of Cardiology. 2021; 26(12): 171-80.Gusev AV, Gavrilov DV, Novitsky RE, Kuznetsova TY, Boytsov SA. Improvement of cardiovascular risk assessment using machine learning methods. Russian Journal of Cardiology. 2021; 26(12): 171-80.Song X MACJRK. Comparison of machine learning techniques with classical statistical models in predicting health outcomes. Stud Health Technol Inform. 2004; 107: 736-40.Song X MACJRK. Comparison of machine learning techniques with classical statistical models in predicting health outcomes. Stud Health Technol Inform. 2004; 107: 736-40.Wu J, Roy J, Stewart WF. Prediction Modeling Using EHR Data Challenges, Strategies, and a Comparison of Machine Learning Approaches [Internet]. 2010. Available from: www.lww-medicalcare.com.Wu J, Roy J, Stewart WF. Prediction Modeling Using EHR Data Challenges, Strategies, and a Comparison of Machine Learning Approaches [Internet]. 2010. Available from: www.lww-medicalcare.com.

The authors declare that there are no conflicts of interest present.