Automatic Evaluation of Recommendation Models


The paper presents an overview of state-of-the-art algorithms used in recommender systems. We discuss the goal of collaborative filtering (CF) as well as different approaches to the method. Specifically, we talk about Singular Value Decomposition (including optimizations, bias, time sensitive Singular Value Decomposition (SVD) and enhanced SVD methods as SVD++), clustering approaches (using K means clustering). We also discuss deep learning methods applied to recommender systems, such as Autoencoders and Restricted Boltzmann Machines. We also go through qualitative evaluation metrics of the algorithms, with a special emphasis on the classification quality metrics, as recommender systems are usually expected to have an order in which the recommendations are delivered. At the same time, we propose a tool that automates the processes of CF algorithms launch and evaluation, that contains data pre-processing, metrics selection, training launch, quality indicators checks and analyses of the resulted data. Our tool demonstrates the impact that parameter selection has on the quality of the algorithm execution. We observed that classical matrix factorization algorithms can compete with new deep learning methods, giving the correct tuning. Also, we demonstrate a significant gain in time between the manual (involving a person that launches all the algorithms individually) and the automatic (when the tool launches all the algorithms) algorithm launch

Сведения об авторах

Olga Alieva, Московский государственный университет имени М.В. Ломоносова

магистрант факультета вычислительной математики и кибернетики

Elena Gangan, Университет Бабеш-Боляи


Eugene Ilyushin, Московский государственный университет имени М.В. Ломоносова

ведущий программист лаборатории открытых информационных технологий, факультет вычислительной математики и кибернетики

Alexey Kachalin, ПАО "Сбербанк России"

заместитель руководителя центра киберзащиты

Как цитировать
ALIEVA, Olga et al. Automatic Evaluation of Recommendation Models. Международный научный журнал «Современные информационные технологии и ИТ-образование», [S.l.], v. 16, n. 2, sep. 2020. ISSN 2411-1473. Доступно на: <>. Дата доступа: 02 dec. 2020
Исследования и разработки в области новых ИТ и их приложений