Room: REC A5.10
Sanne Vrijenhoek is a PhD Candidate at the Institute of Information Law with a background in Artificial Intelligence. As part of the AI, Media and Democracy Lab she works in an interdisciplinary project on assessing diversity in news recommendations. The goal is to translate normative notions of diversity into concrete concepts that can be used to inform recommender system design and to bridge the gap between computer science and the social sciences.
| Bénédict, G., De Rijke, M., Gutierrez Granada, M., Odijk, D., Vrijenhoek, S.|
In: RecSys '22: Proceedings of the 16th ACM Conference on Recommender Systems, pp. 208-219, 2022.
(| | )
In traditional recommender system literature, diversity is often seen as the opposite of similarity, and typically defined as the distance between identified topics, categories or word models. However, this is not expressive of the social science’s interpretation of diversity, which accounts for a news organization’s norms and values and which we here refer to as normative diversity. We introduce RADio, a versatile metrics framework to evaluate recommendations according to these normative goals. RADio introduces a rank-aware Jensen Shannon (JS) divergence. This combination accounts for (i) a user’s decreasing propensity to observe items further down a list and (ii) full distributional shifts as opposed to point estimates. We evaluate RADio’s ability to reflect five normative concepts in news recommendations on the Microsoft News Dataset and six (neural) recommendation algorithms, with the help of our metadata enrichment pipeline. We find that RADio provides insightful estimates that can potentially be used to inform news recommender system design.
| Fokkens, A., Helberger, N., Mattis, N., Müller, J., Reuver, M., Sax, M., Tintarev, N., Van Atteveldt, W., Verberne, S., Vrijenhoek, S.|
In: The 1st Workshop on NLP for Positive Impact: NLP4PosImpact 2021 : proceedings of the workshop, pp. 47-59, 2021.
| Helberger, N., Kaya, M., Metoui, N., Möller, J., Odijk, D., Vrijenhoek, S.|
In: CHIIR '21: Proceedings of the 2021 Conference on Human Information Interaction and Retrieval, pp. 173-183, 2021.
(| | )
News recommenders help users to find relevant online content and have the potential to fulfill a crucial role in a democratic society, directing the scarce attention of citizens towards the information that is most important to them. Simultaneously, recent concerns about so-called filter bubbles, misinformation and selective exposure are symptomatic of the disruptive potential of these digital news recommenders. Recommender systems can make or break filter bubbles, and as such can be instrumental in creating either a more closed or a more open internet. Current approaches to evaluating recommender systems are often focused on measuring an increase in user clicks and short-term engagement, rather than measuring the user's longer term interest in diverse and important information.
This paper aims to bridge the gap between normative notions of diversity, rooted in democratic theory, and quantitative metrics necessary for evaluating the recommender system. We propose a set of metrics grounded in social science interpretations of diversity and suggest ways for practical implementations.
| Drunen, M. van, Helberger, N., Möller, J., Vrijenhoek, S.|
In: Internet Policy Review, 2021, (Opinion).