vitjВрач и информационные технологииMedical Doctor and Information Technologies1811-01932413-5208Pirogov National Medical and Surgical Center10.25881/18110193_2024_3_20vitj-59Research ArticleОБЗОРЫREVIEWSОбзор способов измерения когнитивной нагрузки мозга и методов машинного обучения для их идентификации на основе данных ЭЭГA review of ways to measure brain cognitive load and machine learning methods for their identification from EEG dataДедковА. Е.DedkovA. E.

г. Москва

Moscow

dedkov.ae@yandex.ruАндриковД. А.AndrikovD. A.

к.т.н., доцент

г. Москва

PhD, Associate Professor

Moscow

andrikov-da@rudn.ruХрамовА. Е.HramovA. E.

д.ф.-м.н., профессор

г. Калининград

DSc., Professor

Kaliningrad

hramovae@gmail.comРоссийский университет дружбы народовРоссияRUDNRussian FederationБФУ им. И. КантаРоссияImmanuel Kant Baltic Federal UniversityRussian Federation202414102024032031Copyright © Дедков А.Е., Андриков Д.А., Храмов А.Е., 20242024Дедков А.Е., Андриков Д.А., Храмов А.Е.Dedkov A.E., Andrikov D.A., Hramov A.E.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://www.vit-j.ru/jour/article/view/59

Цель данного обзора заключается в рассмотрении и анализе методов измерения когнитивной нагрузки, а также подходов к использованию методов машинного обучения для идентификации данных ЭЭГ.

Материалы и методы

Материалы и методы. В обзоре систематизированы и обобщены сведения по рассматриваемой теме. Поиск научных статей проведен в библиографических базах данных: eLIBRARY, ScienceDirect, Scopus.

Результаты

Результаты. В данном обзоре были рассмотрены способы измерения когнитивной нагрузки мозга, современные устройства для записи ЭЭГ, методы преобразования, извлечения и классификации признаков из полученных сигналов ЭЭГ.

Выводы

Выводы. С появлением новых носимых устройств для получения и обработки сигналов ЭЭГ появляется потребность в разработке новых подходов к использованию машинного обучения для идентификации когнитивных процессов мозга.

Aim

Aim. The purpose of this article is review and analyze methods for measuring cognitive load, as well as approaches to using machine learning techniques to identify EEG data.

Materials and methods

Materials and methods. The review systematizes and summarizes the information on the topic under consideration. Scientific articles were searched in bibliographic databases: eLIBRARY, ScienceDirect, Scopus.

Results

Results. This review focused on ways to measure the cognitive load of the brain, modern EEG recording devices, and methods for transforming, extracting, and classifying features from acquired EEG signals.

Conclusion

Conclusion. With new wearable devices available for acquiring and processing EEG signals, there is a need to develop new approaches for using machine learning to identify cognitive brain processes.

методы извлечения признаков ЭЭГметоды измерения когнитивной нагрузкиметоды преобразование сигналов ЭЭГметоды классификации признаков ЭЭГmethods for extracting EEG signsmethods for measuring cognitive loadmethods for converting EEG signalsmethods for classifying EEG signsReferencesШарова Д.Е. и др. Анализ мирового опыта в регулировании использования медицинских данных для целей создания систем искусственного интеллекта на основе машинного обучения // Врач и информационные технологии. – 2022. – №4. – C.28-39.SHarova DE, et al. Analiz mirovogo opyta v regulirovanii ispol'zovaniya medicinskih dannyh dlya celej sozdaniya sistem iskusstvennogo intellekta na osnove mashinnogo obucheniya. Vrach i informacionnye tekhnologii. 2022; 4: 28-39. (In Russ.)Карпов О.Э., Храмов А.Е. Информационные технологии, вычислительные системы и искусственный интеллект в медицине. – М.: ДПК Пресс, 2022. – С.321.Karpov OE, Hramov AE. Informacionnye tekhnologii, vychislitel'nye sistemy i iskusstvennyj intellekt v medicine. M.: DPK Press. 2022. Р.321. (In Russ.)Зенков Л.Р. Клиническая электроэнцефалография (с элементами эпилептологии). М.: МЕДпресс-информ, 2012.Zenkov LR. Klinicheskaya elektroencefalografiya (s elementami epileptologii). М.: MEDpress-inform. 2012. (In Russ.)Klimesch W. EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Research Reviews. 1999; 29(2): 169-195. doi:10.1016/S0165-0173(98)00056-3.Klimesch W. EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Research Reviews. 1999; 29(2): 169-195. doi:10.1016/S0165-0173(98)00056-3.Khramova MVK, Hramov AE, Fedorov AA. Current Trends in the Development of Neuroscientific Research in Education. Voprosy obrazovaniya. Educational Studies Moscow. 2023; 4. doi: 10.17323/vo-2023-16701.Khramova MVK, Hramov AE, Fedorov AA. Current Trends in the Development of Neuroscientific Research in Education. Voprosy obrazovaniya. Educational Studies Moscow. 2023; 4. doi: 10.17323/vo-2023-16701.Sheoran M, Kumar S, Chawla S. Methods of denoising of electroencephalogram signal: a review. International Journal of Biomedical Engineering and Technology. 2015; 18(4): 385-395. doi: 10.1504/IJBET.2015.071012.Sheoran M, Kumar S, Chawla S. Methods of denoising of electroencephalogram signal: a review. International Journal of Biomedical Engineering and Technology. 2015; 18(4): 385-395. doi: 10.1504/IJBET.2015.071012.Nuwer M. Assessment of digital EEG, quantitative EEG, and EEG brain mapping: Report of the American Academy of Neurology and the American Clinical Neurophysiology Society. Neurology. 2024; 1: 277-292. doi: 10.1212/WNL.49.1.277.Nuwer M. Assessment of digital EEG, quantitative EEG, and EEG brain mapping: Report of the American Academy of Neurology and the American Clinical Neurophysiology Society. Neurology. 2024; 1: 277-292. doi: 10.1212/WNL.49.1.277.Поликанова И.С., Сергеев А.В. Влияние длительной когнитивной нагрузки на параметры ЭЭГ // Национальный психологический журнал. –2014. – 1(13). – C.84-92.Polikanova IS, Sergeev AV. Vliyanie dlitel'noj kognitivnoj nagruzki na parametry EEG. Nacional'nyj psihologicheskij zhurnal. 2014; 1(13): 84-92. (In Russ.)Смирнов Н.М. и др. Новый электроэнцефалографический маркер эффективности решения когнитивной задачи // Известия Российской академии наук. Серия физическая. – 2023. – 1(87). – C.129-133.Smirnov NM, et al. Novyj elektroencefalograficheskij marker effektivnosti resheniya kognitivnoj zadachi. Izvestiya Rossijskoj akademii nauk. Seriya fizicheskaya. 2023; 1(87): 129-133. (In Russ.)Зеер Э.Ф. Нейродидактика-инновационный тренд персонализированного образования // Профессиональное образование и рынок труда. – 2021. – №4(47). – C.30-38.Zeer EF. Nejrodidaktikainnovacionnyj trend personalizirovannogo obrazovaniya. Professional'noe obrazovanie i rynok truda. 2021; 4(47): 30-38. (In Russ.)Grubov VV, Khramova MV, Goman S, et al. Open-loop neuroadaptive system for enhancing student’s cognitive abilities in learning. IEEE Access. Published online 2024. Accessed June 30, 2024. Available at: https://ieeexplore.ieee.org/abstract/document/10487923.Grubov VV, Khramova MV, Goman S, et al. Open-loop neuroadaptive system for enhancing student’s cognitive abilities in learning. IEEE Access. Published online 2024. Accessed June 30, 2024. Available at: https://ieeexplore.ieee.org/abstract/document/10487923.Максимова М.В., Этуев Х.Х. Опыт применения ЭЭГ в образовании: анализ зарубежных исследований // Отечественная и зарубежная педагогика. – 2023. – 2(1).Maksimova MV, Etuev HH. Opyt primeneniya eeg v obrazovanii: analiz zarubezhnyh issledovanij. Otechestvennaya i zarubezhnaya pedagogika. 2023; 2(1). (In Russ.)Meghdadi AH, Karić MS, Berka C. EEG analytics: benefits and challenges of data driven EEG biomarkers for neurodegenerative diseases. In: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). IEEE; 2019:1280-1285. Accessed June 30, 2024. Available at: https://ieeexplore.ieee.org/abstract/document/8914065.Meghdadi AH, Karić MS, Berka C. EEG analytics: benefits and challenges of data driven EEG biomarkers for neurodegenerative diseases. In: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). IEEE; 2019:1280-1285. Accessed June 30, 2024. Available at: https://ieeexplore.ieee.org/abstract/document/8914065.Örün Ö, Akbulut Y. Effect of multitasking, physical environment and electroencephalography use on cognitive load and retention. Computers in Human Behavior. 2019; 92: 216-229.Örün Ö, Akbulut Y. Effect of multitasking, physical environment and electroencephalography use on cognitive load and retention. Computers in Human Behavior. 2019; 92: 216-229.Park S, Han CH, Im CH. Design of wearable EEG devices specialized for passive brain – computer interface applications. Sensors. 2020; 20(16): 4572.Park S, Han CH, Im CH. Design of wearable EEG devices specialized for passive brain – computer interface applications. Sensors. 2020; 20(16): 4572.Самойлов В.А. и др. Использование мобильного нейроинтерфейса «BrainBit» для анализа профессиональных качеств сотрудника службы безопасности. 2021. – C.249-254.Samojlov VA, et al. Ispol'zovanie mobil'nogo nejrointerfejsa «BrainBit» dlya analiza professional'nyh kachestv sotrudnika sluzhby bezopasnosti. 2021: 249-254. (In Russ.)Krigolson OE, Williams CC, Norton A, Hassall CD, Colino FL. Choosing MUSE: Validation of a low-cost, portable EEG system for ERP research. Frontiers in neuroscience. 2017; 11: 109.Krigolson OE, Williams CC, Norton A, Hassall CD, Colino FL. Choosing MUSE: Validation of a low-cost, portable EEG system for ERP research. Frontiers in neuroscience. 2017; 11: 109.Permana K, Wijaya SK, Prajitno P. Controlled wheelchair based on brain computer interface using Neurosky Mindwave Mobile 2. In: AIP Conference Proceedings. Vol 2168. AIP Publishing; 2019. Accessed June 30, 2024. Available at: https://pubs.aip.org/aip/acp/article-abstract/2168/1/020022/611867.Permana K, Wijaya SK, Prajitno P. Controlled wheelchair based on brain computer interface using Neurosky Mindwave Mobile 2. In: AIP Conference Proceedings. Vol 2168. AIP Publishing; 2019. Accessed June 30, 2024. Available at: https://pubs.aip.org/aip/acp/article-abstract/2168/1/020022/611867.Корякин Ф.И., Сорочинский М.А. Возможности и педагогический потенциал нейроинтерфейсов (на примере NeuroPlay 6C). 2022.Koryakin FI, Sorochinskij MA. Vozmozhnosti i pedagogicheskij potencial nejrointerfejsov (na primere NeuroPlay 6C). 2022. (In Russ.)Ходашинский И.А. и др. Биометрические данные и методы машинного обучения в диагностике и мониторинге нейродегенеративных заболеваний: обзор // Компьютерная оптика. – 2022. – №6(46). – C.988-1020.Hodashinskij IA, et al. Biometricheskie dannye i metody mashinnogo obucheniya v diagnostike i monitoringe nejrodegenerativnyh zabolevanij: obzor. Komp'yuternaya optika. 2022; 6(46): 988-1020. (In Russ.)Winkler I, Haufe S, Tangermann M. Automatic Classification of Artifactual ICA-Components for Artifact Removal in EEG Signals. Behav Brain Funct. 2011; 7(1): 30. doi: 10.1186/1744-9081-7-30.Winkler I, Haufe S, Tangermann M. Automatic Classification of Artifactual ICA-Components for Artifact Removal in EEG Signals. Behav Brain Funct. 2011; 7(1): 30. doi: 10.1186/1744-9081-7-30.Kottaimalai R, Rajasekaran MP, Selvam V, Kannapiran B. EEG signal classification using principal component analysis with neural network in brain computer interface applications. 2013 IEEE International Conference on Emerging Trends in Computing, Communication and Nanotechnology (ICECCN). IEEE; 2013: 227-231. Accessed June 30, 2024. Available at: https://ieeexplore.ieee.org/abstract/document/6528498.Kottaimalai R, Rajasekaran MP, Selvam V, Kannapiran B. EEG signal classification using principal component analysis with neural network in brain computer interface applications. 2013 IEEE International Conference on Emerging Trends in Computing, Communication and Nanotechnology (ICECCN). IEEE; 2013: 227-231. Accessed June 30, 2024. Available at: https://ieeexplore.ieee.org/abstract/document/6528498.Vincent P, Larochelle H, Bengio Y, Manzagol PA. Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning – ICML’08. ACM Press; 2008: 1096-1103. doi: 10.1145/1390156.1390294.Vincent P, Larochelle H, Bengio Y, Manzagol PA. Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning – ICML’08. ACM Press; 2008: 1096-1103. doi: 10.1145/1390156.1390294.Li W, Zhang D, Xu Z. Palmprint identification by fourier transform. Int J Patt Recogn Artif Intell. 2002; 16(04): 417-432. doi: 10.1142/S0218001402001757.Li W, Zhang D, Xu Z. Palmprint identification by fourier transform. Int J Patt Recogn Artif Intell. 2002; 16(04): 417-432. doi: 10.1142/S0218001402001757.Schiff SJ, Aldroubi A, Unser M, Sato S. Fast wavelet transformation of EEG. Electroencephalography and clinical neurophysiology. 1994; 91(6): 442-455.Schiff SJ, Aldroubi A, Unser M, Sato S. Fast wavelet transformation of EEG. Electroencephalography and clinical neurophysiology. 1994; 91(6): 442-455.De Ridder D, Kouropteva O, Okun O, Pietikäinen M, Duin RPW. Supervised Locally Linear Embedding. Kaynak O, Alpaydin E, Oja E, Xu L, eds. Artificial Neural Networks and Neural Information Processing – ICANN/ICONIP. 2003; 2714. Lecture Notes in Computer Science. Springer Berlin Heidelberg; 2003: 333-341. doi: 10.1007/3-540-44989-2_40.De Ridder D, Kouropteva O, Okun O, Pietikäinen M, Duin RPW. Supervised Locally Linear Embedding. Kaynak O, Alpaydin E, Oja E, Xu L, eds. Artificial Neural Networks and Neural Information Processing – ICANN/ICONIP. 2003; 2714. Lecture Notes in Computer Science. Springer Berlin Heidelberg; 2003: 333-341. doi: 10.1007/3-540-44989-2_40.Zavarzin E, Milakhina N, Rudych P, Savostyanov A. EEG Data Preprocessing for Assessing the Degree of Connectivity of Areas of the Human Brain Under Resting Conditions and their Application to the Diagnosis of Socially Significant Diseases. 2021 IEEE 22nd International Conference of Young Professionals in Electron Devices and Materials (EDM). IEEE; 2021: 582-585. Accessed June 30, 2024. Available at: https://ieeexplore.ieee.org/abstract/document/9507667.Zavarzin E, Milakhina N, Rudych P, Savostyanov A. EEG Data Preprocessing for Assessing the Degree of Connectivity of Areas of the Human Brain Under Resting Conditions and their Application to the Diagnosis of Socially Significant Diseases. 2021 IEEE 22nd International Conference of Young Professionals in Electron Devices and Materials (EDM). IEEE; 2021: 582-585. Accessed June 30, 2024. Available at: https://ieeexplore.ieee.org/abstract/document/9507667.Буданцев А.В., Скляр А.Я. Методы машинного распознавания команд в ЭЭГ-сигнале // ИТ-Стандарт. – 2021. – №3. – C.21-27.Budancev AV, Sklyar AYA. Metody mashinnogo raspoznavaniya komand v EEG-signale. IT-Standart. 2021; 3: 21-27. (In Russ.)Hramov AE, Koronovskii AA, Makarov VA, Maksimenko VA, Pavlov AN, Sitnikova E. Wavelets in Neuroscience. Springer International Publishing; 2021. doi: 10.1007/978-3-030-75992-6.Hramov AE, Koronovskii AA, Makarov VA, Maksimenko VA, Pavlov AN, Sitnikova E. Wavelets in Neuroscience. Springer International Publishing; 2021. doi: 10.1007/978-3-030-75992-6.Mirzaei S, Ghasemi P. EEG motor imagery classification using dynamic connectivity patterns and convolutional autoencoder. Biomedical Signal Processing and Control. 2021; 68: 102584.Mirzaei S, Ghasemi P. EEG motor imagery classification using dynamic connectivity patterns and convolutional autoencoder. Biomedical Signal Processing and Control. 2021; 68: 102584.Stas S. Hybrid convolutional-multilayer perceptron artificial neural network for person recognition by high gamma EEG features. 2022; 17(2): 192-196.Stas S. Hybrid convolutional-multilayer perceptron artificial neural network for person recognition by high gamma EEG features. 2022; 17(2): 192-196.Hsu WY. EEG-based motor imagery classification using neuro-fuzzy prediction and wavelet fractal features. Journal of Neuroscience Methods. 2010; 189(2): 295-302.Hsu WY. EEG-based motor imagery classification using neuro-fuzzy prediction and wavelet fractal features. Journal of Neuroscience Methods. 2010; 189(2): 295-302.Володин М.В., Седых И.А. Оценка применения ConvLSTM-нейронной сети для прогнозирования эпилептических припадков. 2023.Volodin MV, Sedyh IA. Ocenka primeneniya ConvLSTM-nejronnoj seti dlya prognozirovaniya epilepticheskih pripadkov. 2023. (In Russ.)Wu SL, Wu CW, Pal NR, Chen CY, Chen SA, Lin CT. Common spatial pattern and linear discriminant analysis for motor imagery classification. In: 2013 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (Ccmb). IEEE; 2013: 146-151. Accessed June 30, 2024. Available at: https://ieeexplore.ieee.org/abstract/document/6609178.Wu SL, Wu CW, Pal NR, Chen CY, Chen SA, Lin CT. Common spatial pattern and linear discriminant analysis for motor imagery classification. In: 2013 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (Ccmb). IEEE; 2013: 146-151. Accessed June 30, 2024. Available at: https://ieeexplore.ieee.org/abstract/document/6609178.Капралов Н.В., Нагорнова Ж.В., Шемякина Н.В. Методы классификации ЭЭГ-паттернов воображаемых движений // Информатика и автоматизация. – 2021. – №1(20). – C.94-132.Kapralov NV, Nagornova ZHV, SHemyakina NV. Metody klassifikacii EEG-patternov voobrazhaemyh dvizhenij. Informatika i avtomatizaciya. 2021; 1(20): 94-132. (In Russ.)Zhang Y, Wang Y, Jin J, Wang X. Sparse Bayesian Learning for Obtaining Sparsity of EEG Frequency Bands Based Feature Vectors in Motor Imagery Classification. Int J Neur Syst. 2017; 27(02): 1650032. doi: 10.1142/S0129065716500325.Zhang Y, Wang Y, Jin J, Wang X. Sparse Bayesian Learning for Obtaining Sparsity of EEG Frequency Bands Based Feature Vectors in Motor Imagery Classification. Int J Neur Syst. 2017; 27(02): 1650032. doi: 10.1142/S0129065716500325.Шанин И.А., Ступников С.А. Методы анализа данных электроэнцефалографии с применением сверточных и рекуррентных нейронных сетей // Системы и средства информатики. – 2021. – №2(31). – C.36-46.SHanin IA, Stupnikov SA. Metody analiza dannyh elektroencefalografii s primeneniem svertochnyh i rekurrentnyh nejronnyh setej. Sistemy i sredstva informatiki. 2021; 2(31): 36-46. (In Russ.)Tabar YR, Halici U. A novel deep learning approach for classification of EEG motor imagery signals. Journal of neural engineering. 2016; 14(1): 016003.Tabar YR, Halici U. A novel deep learning approach for classification of EEG motor imagery signals. Journal of neural engineering. 2016; 14(1): 016003.Ямаев М.И., Шипицин С.П., Филатова Е.С. Нейронные сети для классификации ЭЭГ: от перцептрона до сверточной LSTM-сети, 2018. – C.765-768.YAmaev MI, SHipicin SP, Filatova ES. Nejronnye seti dlya klassifikacii EEG: ot perceptrona do svertochnoj LSTM-seti, 2018. Р.765-768. (In Russ.)Bentlemsan M, Zemouri ET, Bouchaffra D, Yahya-Zoubir B, Ferroudji K. Random forest and filter bank common spatial patterns for EEG-based motor imagery classification. In: 2014 5th International Conference on Intelligent Systems, Modelling and Simulation. IEEE. 2014: 235-238. Accessed June 30, 2024. Available at: https://ieeexplore.ieee.org/abstract/document/7280913.Bentlemsan M, Zemouri ET, Bouchaffra D, Yahya-Zoubir B, Ferroudji K. Random forest and filter bank common spatial patterns for EEG-based motor imagery classification. In: 2014 5th International Conference on Intelligent Systems, Modelling and Simulation. IEEE. 2014: 235-238. Accessed June 30, 2024. Available at: https://ieeexplore.ieee.org/abstract/document/7280913.

The authors declare that there are no conflicts of interest present.