Silah at QIAS 2026: Fine-Tuning vs. Retrieval-Augmented Generation for Islamic Inheritance Reasoning

Abstract

Islamic inheritance is a highly structured and rule-intensive domain that requires precise reasoning. The QIAS 2026 Shared Task introduces a benchmark for evaluating generative artificial intelligence on end-to-end inheritance problem solving. In this paper, we present our team Silah’s participation in the QIAS 2026 shared task, where we compare three approaches: (1) a multi-stage retrieval-augmented, rule-guided pipeline, (2) supervised fine-tuning of generative large language models, and (3) a retrieval-augmented fine-tuning approach. We evaluate several open-source models, including Qwen2.5, Llama, DeepSeek, and Fanar. Our results show that supervised fine-tuning consistently outperforms retrieval-based approaches, with the fine-tuned Fanar-1-9B-Instruct model achieving the best performance (MIR-E = 0.83) and ranking sixth overall in the shared task. These findings suggest that learning implicit reasoning patterns through fine-tuning is more effective than explicit rule injection under current retrieval setups, and emphasize the need for more accurate and minimal rule selection mechanisms in future retrieval-augmented approaches.

Resources

Details