CEUR-WS.org/Vol-3752 - Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)

PDF
LLMJudge: LLMs for Relevance Judgments 1-3 PDF
Nick Craswell, Charles L. A. Clarke, Paul Thomas 0001, Emine Yilmaz, Hossein A. Rahmani, Guglielmo Faggioli, Mohammad Aliannejadi, Bhaskar Mitra 0001, Clemencia Siro
The Challenges of Evaluating LLM Applications: An Analysis of Automated, Human, and LLM-Based Approaches 4-18 PDF
Ruhan Circi, Bhashithe Abeysinghe
Exploring Large Language Models for Relevance Judgments in Tetun 19-30 PDF
Gabriel de Jesus, Sérgio Sobral Nunes
EXAM++: LLM-based Answerability Metrics for IR Evaluation 31-50 PDF
Naghmeh Farzi, Laura Dietz
A Novel Evaluation Framework for Image2Text Generation 51-65 PDF
Hongyi Zhu (disambiguation), Stevan Rudinac, Alessio M. Pacces, Evangelos Kanoulas, Yixian Shen, Jia-Hong Huang
Using LLMs to Investigate Correlations of Conversational Follow-up Queries with User Satisfaction 66-91 PDF
Yoonseo Choi, Jin Young Kim (disambiguation), Juho Kim, Chaneon Park, Taehyun Yang, Yongju Lee (disambiguation), Honggu Lee, Hyunwoo Kim (disambiguation)
Evaluating RAG-Fusion with RAGElo: an Automated Elo-based Framework 92-112 PDF
Arthur Câmara, Zackary Rackauckas, Jakub Zavrel
Toward Automatic Relevance Judgment using Vision-Language Models for Image-Text Retrieval Evaluation 113-123 PDF
Jimmy Lin, Jheng-Hong Yang

LLM4Eval 2024