LabadainLog-60: A Curated Tetun Query-Response Dataset for Conversational System Evaluation

Multi-turn queries:** Tasks requiring interaction across two conversational turns. **3.Ambiguous queries:** Under-specified requests that require clarification. --- **2. Dataset Contents** **2.1. labadainlog-60-query-response.json** Core dataset of 60 labeled conversation cases (LC001–LC060). Each case includes: **2.1.1.

Query Information:** Each query includes: - query ID: Unique query identifier. - initial question: Initial user question. - follow-up question: Follow-up question (if applicable). - query type: One of "Fact-seeking", "Multi-turn", or "Ambiguous". **2.1.2. System Response:** Responses generated by multiple systems for each query: - labadain_chat: Primary response from the current Labadain Chat system. - labadain_chat_follow_up: Follow-up response (if applicable). - labadain_old: Response from the legacy Labadain Chat system. - labadain_old_follow_up: Follow-up response (if applicable). - chatgpt: Response from ChatGPT. - chatgpt_follow_up: Follow-up response (if applicable). --- **2.2. labadainlog-60-eval** This directory contains evaluation results produced by three annotators.

The files include the following subsets: - **labadainlog-60-eval-a1-(lc001-lc040).json** Annotation results for queries LC001–LC040 provided by the first annotator. - **labadainlog-60-eval-a2-(lc021-lc060).json** Annotation results for queries LC021–LC060 provided by the second annotator. - **labadainlog-60-eval-a3-(lc001-lc020_lc041-lc060).json** Annotation results for queries LC001–LC020 and LC041–LC060 provided by the third annotator. --- **3.

Data Schema** Each entry follows the structure below. Evaluation fields are included only in the files generated by human annotators. **3.1 Query Information** Details about the user query: - **query_id:** String identifier (e.g., "LC017"). - **query:** Initial user question. - **query_follow_up:** Follow-up question or clarification, applicable only to multi-turn queries. - **query_type:** "Fact-seeking", "Multi-turn", or "Ambiguous". **3.2 System Response** Responses generated by multiple systems: - **labadain_chat:** Primary response from the current Labadain Chat system. - **labadain_chat_follow_up:** Follow-up response, if applicable. - **labadain_old:** Response from the legacy Labadain Chat system. - **labadain_old_follow_up:** Follow-up response, if applicable. - **chatgpt:** Response generated by ChatGPT. - **chatgpt_follow_up:** Follow-up response, if applicable. **3.3.

Evaluation:** Assessment of task success and user satisfaction: - **Task Success** - Boolean (0 or 1) - Rated for: labadain_chat, labadain_old, chatgpt - **User Satisfaction** (5-point Linkert scale) - 1 = very dissatisfied, 2 = dissatisfied, 3 = neutral, 4 = satisfied, 5 = very satisfied. - Rated for: labadain_chat, labadain_old, chatgpt. -- **

Citation

Information:** If you use this dataset or any of its contents for your research, academic work, or publication, please cite it as follows: Jesus, G. de, & Nunes, S. (2026). Labadain Chat: A conversational agent for the Tetun language. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26).

Association for Computing Machinery. doi.org/10.1145/3805712.3808372