Sebastian Sünkler, Kardelen Bilir, Tuhina Kumar, Oliver Koop, Sebastian Schultheiß, and Dirk Lewandowski published a demo paper showcasing the various features and potential applications of the Result Assessment Tool (RAT) in the Proceedings of the 2026 Conference on Human Information Interaction and Retrieval (CHIIR).
The paper received the Best Demo Paper Award at CHIIR 2026!
Abstract
The Result Assessment Tool (RAT) is an open-source software toolkit for conducting research with results from commercial search engines and other web-based information retrieval systems. Conducting such research is challenging due to the “black-box” nature of these systems and limited data access. RAT addresses this by providing an integrated software environment that unifies modules for study design, automated data collection, manual assessment in a dedicated interface, and automated analysis via an extensible classifier framework. The software is designed to assist with various research tasks, including comparative assessments of result quality, investigations of source variety, and content analysis. It emphasizes transparency in methodologies, reproducibility of outcomes, and responsible data collection.
Read the full paper: https://doi.org/10.1145/3786304.3787925