Methods of computational linguistics in the evaluation of artifi cial intelligence systems

The article discusses current research in the fi eld of applied linguistics dedicated to the evaluation of artifi cial intelligence (AI) systems. Linguistic tests are used as the principal tool for evaluating the level of intelligence of such systems, being the most aff ordable way of training AI systems and, at the same time, having high variability necessary for the formulation of intellectual tasks. This paper provides an overview of current methodology for training and testing AI systems and describes the gold standards of textual tasks (benchmarks) in the General Language Understanding Evaluation (GLUE) methodology. We also present an overview of how the theoretical apparatus and practices of linguistics are used to create a Russian-language test for examining the abilities of AI systems, the Russian SuperGLUE. Further convergence of machine learning and linguistic methods can fi ll gaps both in the practice of evaluating AI systems and in their eff ective training.