| Bénédict, G., De Rijke, M., Gutierrez Granada, M., Odijk, D., Vrijenhoek, S. RADio – Rank-Aware Divergence Metrics to Measure Normative Diversity in News Recommendations In: RecSys '22: Proceedings of the 16th ACM Conference on Recommender Systems, pp. 208-219, 2022. @article{nokey,
title = {RADio \textendash Rank-Aware Divergence Metrics to Measure Normative Diversity in News Recommendations},
author = {Vrijenhoek, S. and B\'{e}n\'{e}dict, G. and Gutierrez Granada, M. and Odijk, D. and De Rijke, M.},
url = {https://dl.acm.org/doi/abs/10.1145/3523227.3546780},
doi = {10.1145/3523227.3546780},
year = {2022},
date = {2022-09-13},
journal = {RecSys '22: Proceedings of the 16th ACM Conference on Recommender Systems},
pages = {208-219},
abstract = {In traditional recommender system literature, diversity is often seen as the opposite of similarity, and typically defined as the distance between identified topics, categories or word models. However, this is not expressive of the social science’s interpretation of diversity, which accounts for a news organization’s norms and values and which we here refer to as normative diversity. We introduce RADio, a versatile metrics framework to evaluate recommendations according to these normative goals. RADio introduces a rank-aware Jensen Shannon (JS) divergence. This combination accounts for (i) a user’s decreasing propensity to observe items further down a list and (ii) full distributional shifts as opposed to point estimates. We evaluate RADio’s ability to reflect five normative concepts in news recommendations on the Microsoft News Dataset and six (neural) recommendation algorithms, with the help of our metadata enrichment pipeline. We find that RADio provides insightful estimates that can potentially be used to inform news recommender system design.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In traditional recommender system literature, diversity is often seen as the opposite of similarity, and typically defined as the distance between identified topics, categories or word models. However, this is not expressive of the social science’s interpretation of diversity, which accounts for a news organization’s norms and values and which we here refer to as normative diversity. We introduce RADio, a versatile metrics framework to evaluate recommendations according to these normative goals. RADio introduces a rank-aware Jensen Shannon (JS) divergence. This combination accounts for (i) a user’s decreasing propensity to observe items further down a list and (ii) full distributional shifts as opposed to point estimates. We evaluate RADio’s ability to reflect five normative concepts in news recommendations on the Microsoft News Dataset and six (neural) recommendation algorithms, with the help of our metadata enrichment pipeline. We find that RADio provides insightful estimates that can potentially be used to inform news recommender system design. |
| Fokkens, A., Helberger, N., Mattis, N., Müller, J., Reuver, M., Sax, M., Tintarev, N., Van Atteveldt, W., Verberne, S., Vrijenhoek, S. Are we human, or are we users? The role of natural language processing in human-centric news recommenders that nudge users to diverse content In: The 1st Workshop on NLP for Positive Impact: NLP4PosImpact 2021 : proceedings of the workshop, pp. 47-59, 2021. @article{Reuver2021,
title = {Are we human, or are we users? The role of natural language processing in human-centric news recommenders that nudge users to diverse content},
author = {Reuver, M. and Mattis, N. and Sax, M. and Verberne, S. and Tintarev, N. and Helberger, N. and M\"{u}ller, J. and Vrijenhoek, S. and Fokkens, A. and Van Atteveldt, W.},
url = {https://aclanthology.org/2021.nlp4posimpact-1.6/},
doi = {https://doi.org/10.18653/v1/2021.nlp4posimpact-1.6},
year = {2021},
date = {2021-08-01},
journal = {The 1st Workshop on NLP for Positive Impact: NLP4PosImpact 2021 : proceedings of the workshop},
pages = {47-59},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
|
| Helberger, N., Kaya, M., Metoui, N., Möller, J., Odijk, D., Vrijenhoek, S. Recommenders with a Mission: Assessing Diversity in News Recommendations In: CHIIR '21: Proceedings of the 2021 Conference on Human Information Interaction and Retrieval, pp. 173-183, 2021. @article{nokey,
title = {Recommenders with a Mission: Assessing Diversity in News Recommendations},
author = {Vrijenhoek, S. and Kaya, M. and Metoui, N. and M\"{o}ller, J. and Odijk, D. and Helberger, N.},
url = {https://dl.acm.org/doi/10.1145/3406522.3446019},
doi = {10.1145/3406522.3446019},
year = {2021},
date = {2021-03-14},
journal = {CHIIR '21: Proceedings of the 2021 Conference on Human Information Interaction and Retrieval},
pages = {173-183},
abstract = {News recommenders help users to find relevant online content and have the potential to fulfill a crucial role in a democratic society, directing the scarce attention of citizens towards the information that is most important to them. Simultaneously, recent concerns about so-called filter bubbles, misinformation and selective exposure are symptomatic of the disruptive potential of these digital news recommenders. Recommender systems can make or break filter bubbles, and as such can be instrumental in creating either a more closed or a more open internet. Current approaches to evaluating recommender systems are often focused on measuring an increase in user clicks and short-term engagement, rather than measuring the user's longer term interest in diverse and important information.
This paper aims to bridge the gap between normative notions of diversity, rooted in democratic theory, and quantitative metrics necessary for evaluating the recommender system. We propose a set of metrics grounded in social science interpretations of diversity and suggest ways for practical implementations.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
News recommenders help users to find relevant online content and have the potential to fulfill a crucial role in a democratic society, directing the scarce attention of citizens towards the information that is most important to them. Simultaneously, recent concerns about so-called filter bubbles, misinformation and selective exposure are symptomatic of the disruptive potential of these digital news recommenders. Recommender systems can make or break filter bubbles, and as such can be instrumental in creating either a more closed or a more open internet. Current approaches to evaluating recommender systems are often focused on measuring an increase in user clicks and short-term engagement, rather than measuring the user's longer term interest in diverse and important information.
This paper aims to bridge the gap between normative notions of diversity, rooted in democratic theory, and quantitative metrics necessary for evaluating the recommender system. We propose a set of metrics grounded in social science interpretations of diversity and suggest ways for practical implementations. |
| Drunen, M. van, Helberger, N., Möller, J., Vrijenhoek, S. Regulation of news recommenders in the Digital Services Act: empowering David against the Very Large Online Goliath In: Internet Policy Review, 2021, (Opinion). @article{Helberger2021b,
title = {Regulation of news recommenders in the Digital Services Act: empowering David against the Very Large Online Goliath},
author = {Helberger, N. and Drunen, M. van and Vrijenhoek, S. and M\"{o}ller, J.},
url = {https://policyreview.info/articles/news/regulation-news-recommenders-digital-services-act-empowering-david-against-very-large},
year = {2021},
date = {2021-02-26},
journal = {Internet Policy Review},
note = {Opinion},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
|
Sanne Vrijenhoek is a PhD Candidate at the Institute of Information Law with a background in Artificial Intelligence. As part of the AI, Media and Democracy Lab she works in an interdisciplinary project on assessing diversity in news recommendations. The goal is to translate normative notions of diversity into concrete concepts that can be used to inform recommender system design and to bridge the gap between computer science and the social sciences.