SdQuAD: A Large Benchmark Question Answering Dataset for Low-resource Sindhi Language

Abstract

Question answering (QA) datasets are crucial for developing and evaluating monolingual and multilingual language models, yet low-resource languages like Sindhi lack open-source QA resources. We introduce SdQuAD, a novel open-source textual QA dataset for the low-resource Sindhi language, comprising 15,000 QA pairs meticulously annotated by native speakers using the Label Studio platform. Sourced from diverse domains, including news, history, science, geography, business, and tourism, SdQuAD supports both extractive and abstractive QA tasks while capturing Sindhi’s linguistic and topical diversity. We assess annotation quality using span-level agreement and evaluate extractive performance with Exact Match (EM), F1 score, and a TF-IDF baseline. Additionally, we fine-tune mBERT, XLM-R, and mT5 models on SdQuAD, benchmarking their performance to demonstrate the dataset’s utility.

Resources

Details