Evaluating Large Language Model Localization for Clinical Psychology: Predicting Anxiety and Depression Scores in a Russian Student Sample

Aim. To evaluate the suitability of large language models (LLMs), deployed in a closed-loop environment, for predicting anxiety and depression scores on the Hospital Anxiety and Depression Scale (HADS) in a sample of Russian students.

Materials and methods. The empirical dataset consisted of de-identified responses from 946 first-year university students (669 female, 277 male) in Russia to 14 HADS items. For 11 models (DeepSeek-R1-32B, Qwen3-32B, T-pro-2.0/2.1, YandexGPT- 8B, GigaChat-20B, and others) deployed locally on GPUs, response probability distributions were extracted from token log probabilities. The experimental design included 5 prompt groups (textual instructions for the model), 2 presentation formats (structured dialogue and unstructured text), and 4 wording variations. Model-generated and empirical distributions were compared using Wasserstein distance (W); expected HADS subscale scores were additionally calculated.

Results. T DeepSeek-R1-32B demonstrated the best accuracy in reproducing distributions (W = 0.293); however, it consistently underestimated expected anxiety scores (5.6 vs. 7.7). The most accurate expected anxiety scores were produced by YandexGPTpretrain (7.2) and T-pro-2.0 (7.1). Model selection explained 47% of metric variance; persona detail and respondent sex had no significant effect (p > 0.05). The depression subscale was reproduced less accurately than the anxiety subscale.

Conclusions. Scores are robust to prompt variations: the level of detail in the persona description and the specific wording do not affect the outcome—a minimal description of the respondent suffices. Models trained in English favor English-language prompts, whereas for Russian-language models, language preferences are determined by the underlying architecture. The hypothesis regarding a systematic advantage for Russian-language models was not confirmed: quality is determined by domain adaptation rather than language. Practical application requires validation through experiments involving human subjects. The models do not replace clinical assessment by a specialist.

1. Snaith RP, Zigmond AS. The Hospital Anxiety and Depression Scale. Br Med J (Clin Res Ed). 1986; 292(6516): 344.

2. Herrero MJ, Blanch J, Peri JM, et al. A validation study of the Hospital Anxiety and Depression Scale (HADS) in a Spanish population. Gen Hosp Psychiatry. 2003; 25(4): 277-283. doi: 10.1016/s0163-8343(03)00043-4.

3. Tiksnadi BB, Triani N, Fihaya, FY, et al. Validation of Hospital Anxiety and Depression Scale in an Indonesian population: a scale adaptation study. Family Medicine and Community Health. 2023; 11; e001775. doi: 10.1136/fmch-2022-001775.

4. Michopoulos I, Douzenis A, Kalkavoura C, et al. Hospital Anxiety and Depression Scale (HADS): validation in a Greek general hospital sample. Ann Gen Psychiatry. 2008; 7: 4. doi: 10.1186/1744-859X-7-4.

5. Chan YF, Leung DY, Fong DY, et al. Psychometric evaluation of the Hospital Anxiety and Depression Scale in a large community sample of adolescents in Hong Kong. Qual Life Res. 2010; 19(6): 865-873. doi: 10.1007/s11136-010-9645-1.

6. Mumford DB, Tareen IA, Bajwa MA, Bhatti MR, Karim R. The translation and evaluation of an Urdu version of the Hospital Anxiety and Depression Scale. Acta Psychiatr Scand. 1991; 83(2): 81-85. doi: 10.1111/j.1600-0447.1991.tb07370.x.

7. Marakshina JA, Pavlova AA, Ismatullina VI, et al. Psychometric properties analysis of the Hospital Anxiety and Depression Scale (HADS) on a sample of Russian-speaking students. Acta Biomed Sci. 2025; 10(5): 182-193. doi: 10.29413/ABS.2025-10.5.20.

8. Triantafyllidis A, Segkouli S, Kokkas S, et al. Large language models for cardiovascular disease, cancer, and mental disorders: a review of systematic reviews. Healthcare (Basel). 2026; 14(1): 45. doi: 10.3390/healthcare14010045.

9. Ghorbian M, Ghobaei-Arani M. Large language models for mental health diagnosis and treatment: a survey. Artif Intell Rev. 2026; 59: 9. doi: 10.1007/s10462-025-11418-0.

10. Kuzmin G, Strepetov P, Stankevich M, et al. Exploring large language models for detecting mental disorders. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025: 34535-34559. doi: 10.18653/v1/2025.emnlp-main.1752.

11. Joshi P, Santy S, Budhiraja A, et al. The state and fate of linguistic diversity and inclusion in the NLP world. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 6282-6293. doi: 10.18653/v1/2020.acl-main.560.

12. Hedderich MA, Lange L, Adel H, Strotgen J, Klakow D. A survey on recent approaches for natural language processing in low-resource scenarios. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021: 2545-2568. doi: 10.18653/v1/2021.naacl-main.201.

13. Localizing AI in the Global South. Nat Mach Intell. 2025; 7: 675. doi: 0.1038/s42256-025-01057-z.

14. Gromenko E, Kalacheva D, Klokova K, et al. Cultural evaluation of LLMs in Russian: catchphrases and cultural types. In: Proceedings of the International Conference "Dialogue". 2025.

15. Grigoreva V, Ivanova A, Alimova I, et al. RuBia: a Russian language bias detection dataset. In: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). 2024: 14227-14239.

16. Nazarov DM, Badaev FI. Primenenie bol'shikh yazykovykh modelei v sfere zdravookhraneniya. Menedzher zdravookhraneniya. 2025; 5: 142-154. (In Russ.) doi: 10.21045/1811-0185-2025-5-142-154.

17. Zigmond AS, Snaith RP. The Hospital Anxiety and Depression Scale. Acta Psychiatr Scand. 1983; 67(6): 361-370. doi: 10.1111/j.1600-0447.1983.tb09716.x.

18. Agapov MA, Skalin YuE, Bocharova NN. Skriningovaya diagnostika psikhoemotsional'nogo sostoyaniya obuchayushchikhsya v sisteme vysshego obrazovaniya. Vestnik Baltiyskogo federal'nogo universiteta im. I. Kanta. Seriya: Filologiya, pedagogika, psikhologiya. 2023; 1: 90-101. (In Russ.) doi: 10.5922/pikbfu-2023-1-9.

19. Kwon W, Li Z, Zhuang S, et al. Efficient memory management for large language model serving with PagedAttention. In: Proceedings of the 29th Symposium on Operating Systems Principles. 2023. doi: 10.1145/3600006.3613165.

20. Structured outputs. vLLM documentation v0.8.2. 2025. Available from: https://docs.vllm.ai/en/v0.8.2/features/structured_outputs.html. Сited February 25, 2026. 21. Givens CR, Shortt RM. A class of Wasserstein

21. metrics for probability distributions. Michigan Math J. 1984; 31(2): 231-240. doi: 10.1307/mmj/1029003026. 22. Kim S, Jeong J, Han JS, et al. LLM-Mirror: a generated-persona approach for survey pre-testing. Preprint. 2024. doi: 10.48550/arXiv.2412.03162.

22. Moon S, Abdulhai M, Kang M, et al. Virtual personas for language models via an anthology of backstories. In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024: 19864-19897. doi: 10.18653/v1/2024.emnlp-main.1110.

23. Santurkar S, Durmus E, Ladhak F, et al. Whose opinions do language models reflect? In: Proceedings of the 40th International Conference on Machine Learning (ICML). 2023. doi: 10.48550/arXiv.2303.17548.

24. Tomczak M, Tomczak E. The need to report effect size estimates revisited: an overview of some recommended measures of effect size. Trends Sport Sci. 2014; 1(21): 19-25.

25. Zhao J, Yuan C, Luo W, et al. Large language models as virtual survey respondents: evaluating sociodemographic response generation. Preprint. 2025. doi: 10.48550/arXiv.2509.06337. 27. Sukiennik N, Gao C, Xu F, Li Y. An evaluation of cultural value alignment in LLM. Preprint. 2025. doi: 10.48550/arXiv.2504.08863.

26. Lutz M, Sen I, Ahnert G, Rogers E, Strohmaier M. The prompt makes the person(a): a systematic evaluation of sociodemographic persona prompting for large language models. In: Findings of the Association for Computational Linguistics: EMNLP 2025. 2025: 23212-23237. doi: 10.18653/v1/2025. findings-emnlp.1261.

27. Kashirina IL, Starichkova YuV, Le ChK. Tonkaya nastroika yazykovoi modeli RuBERT dlya povysheniya tochnosti analiza meditsinskikh zaprosov. Vrach i informatsionnye tekhnologii. 2026; 1: 64-73. (In Russ.) doi: 10.25881/18110193_2026_1_64.

28. Qi W, Huang F, An J, Kwak H. A cross-cultural comparison of LLM-based public opinion simulation. Preprint. 2025. doi: 10.48550/arXiv.2506.21587.

29. Kabir M, Abrar A, Ananiadou S. Break the checkbox: challenging closed-style evaluations of cultural alignment in LLMs. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025: 24-51. doi: 10.18653/v1/2025.emnlp-main.2.

30. Rupprecht J, Froehling L, Wagner C, Strohmaier M. German General Social Survey personas: a survey-derived persona prompt collection for population-aligned LLM studies. In: Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026). 2026: 1761-1780. doi: 10.63317/2sod6uekicbg.

31. Bjelland I, Dahl AA, Haug TT, et al. The validity of the Hospital Anxiety and Depression Scale: an updated literature review. J Psychosom Res. 2002; 52(2): 69-77. doi: 10.1016/S0022-3999(01)00296-3.

32. Petrov NB, Serapio-Garcia G, Rentfrow J. Limited ability of LLMs to simulate human psychological behaviours: a psychometric analysis. Preprint. 2024. doi: 10.48550/arXiv.2405.07248.

33. Teutloff JK. Synthetic founders: AI-generated social simulations for startup validation research in computational social science. Preprint. 2025. doi: 10.48550/arXiv.2509.02605.

34. Hu T, Collier N. Quantifying the persona effect in LLM simulations. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024: 10289- 10307. doi: 10.18653/v1/2024.acl-long.554.

35. Giorgi S, Liu T, Aich A, et al. Modeling human subjectivity in LLMs using explicit and implicit human factors in personas. In: Findings of the Association for Computational Linguistics: EMNLP 2024. 2024: 7174-7188. doi: 10.18653/v1/2024.findings-emnlp.420.

36. Sclar M, Choi Y, Tsvetkov Y, Suhr A. Quantifying language models' sensitivity to spurious features in prompt design or: how I learned to start worrying about prompt formatting. In: Proceedings of the Twelfth International Conference on Learning Representations (ICLR). 2024. doi: 10.48550/arXiv.2310.11324.