Multi-Layer Ranking with Large Language Models for News Source Recommendation
Abstract
To seek reliable information sources for news events, we introduce a novel task of expert recommendation, which aims to identify trustworthy sources based on their previously quoted statements. To achieve this, we built a novel dataset, called NewsQuote, consisting of 23,571 quote-speaker pairs sourced from a collection of news articles. We formulate the recommendation task as the retrieval of experts based on their likelihood of being associated with a given query. We also propose a multi-layer ranking framework employing Large Language Models to improve the recommendation performance. Our results show that employing an in-context learning based LLM ranker and a multi-layer ranking-based filter significantly improve both the predictive quality and behavioural quality of the recommender system.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2024
- DOI:
- 10.48550/arXiv.2406.11745
- arXiv:
- arXiv:2406.11745
- Bibcode:
- 2024arXiv240611745Z
- Keywords:
-
- Computer Science - Information Retrieval;
- Computer Science - Computation and Language
- E-Print:
- Accepted by the SIGIR 2024. arXiv admin note: text overlap with arXiv:2305.04825