2026-05-19T01:10:47Zhttps://keep.lib.asu.edu/oai/request

oai:keep.lib.asu.edu:node-2011472025-05-05T15:53:02Zoai_pmh:alloai_pmh:repo_items

201147 https://hdl.handle.net/2286/R.2.N.201147 http://rightsstatements.org/vocab/InC/1.0/ All Rights Reserved 2025 53 pages Masters Thesis Academic theses en Liou, Kuan-Ru Wei, Hua Zou, Jia Choi, YooJung Arizona State University Partial requirement for: M.S., Arizona State University, 2025 Field of study: Computer Science Asking ambiguous questions is a natural aspect of human communication, making it essential for Large Language Models (LLMs) to effectively recognize and address ambiguities. However, a comprehensive analysis of how well LLMs detect and solve ambiguities is lacking. Besides, though several datasets exist on ambiguity, the absence of explicit explanations of ambiguity and annotations of ambiguity types limits the comprehensive evaluation. To address this issue, I introduce Abg-SciQA, a dataset designed to evaluate and help LLMs detect ambiguities and generate appropriate clarification questions using challenge questions in the area of social and nature science. Abg-SciQA encompasses four tasks: Ambiguity Detection, Ambiguity Type Classification, Clarification Question Generation, and Clarification-Based Question Answering, where each task has corresponding annotations. I evaluate the dataset using both closed-source and open-source LLMs and fine-tune it on open-source LLMs. My experiments show that the most state-of-the-art LLMs still encounter difficultiesin resolving ambiguity in natural questions, and fine-tuning on Abg-SciQA can significantly enhance their capabilities to understand and address ambiguities. Notably, in the Ambiguity Detection task, the F1 score of Llama2-7b improves significantly from 16.6% to 79.1%. On the other hand, Abg-SciQA remains a challenging benchmark for LLMs, revealing ample room for model improvement. Computer Science Linguistics Ambiguity Dataset Fine-tuning LLMs Abg-SciQA: Benchmarking and Enhancing Ambiguity Detection and Clarification in Language Models