Use Cases

Monitoring the Recommender System of the EOSC Marketplace/Search Service

The Recommender Metrics Framework currently monitors the Recommender System of the EOSC Marketplace/Search Service by reporting diagnostic metrics and visualizations. The Recommender System (RS) is a component of the EOSC Search Service designed to enhance the user experience by providing recommendations based on a multi-focal perspective of the users. The software calculates statistics, metrics, KPIs, and visualizations to measure the success of the RS that offer deeper insights into the performance of the system. The evaluation process involves quantitatively processing information such as resources, user actions, ratings, and recommendations to measure the impact of the AI-enhanced services and user satisfaction. The feedback obtained from the evaluation is incorporated to improve the services provided through a user-friendly API and dashboard UI.

Evaluate a third-party RS

The Recommender Metrics Framework due to its design can be used as an analysis tool of the recommendation engine of a Recommender System. Specifically, if the RMF is fed with the necessary input information such as resources, user actions, ratings, and recommendations can carry out the preprocessing stage which is retrieving data from multiple sources through a connector module that claims and transforms the data, establishing connections between service-related knowledge, eliminating dummy or irrelevant data, tagging associations in the data such as registered or anonymous users and services, and generating statistical information. Subsequently, RMF will perform the next phase's steps which are: processing the data, computing the designated evaluation metrics, and uniformly presenting the resulting information. The results such as statistics, metrics, and KPIs will be displayed through a visually appealing UI/dashboard and a REST API, with comprehensive documentation on how metrics are calculated and the expected output range.