A Large-Scale Instruction-Tuning Dataset and Models for Slovenian Vision-Language Tasks

Abstract

Vision-language models (VLMs) represent a significant leap forward in artificial intelligence, yet their development has been predominantly focused on English, creating a digital divide for speakers of less-resourced languages. This paper addresses this gap by introducing the first large-scale, general instruction-tuning dataset for the less-resourced Slovenian language. Comprising over one million text-image pairs, the dataset was constructed through a multi-pronged approach: automatic curation from Slovenian news media and Wikipedia, and machine translation of the English LLaVA-665k dataset. To demonstrate the dataset’s efficacy, we fine-tuned two pre-trained, multilingual Gemma-3 models (4B and 12B parameters) on this new resource. Our evaluation, conducted on a new manually curated test set, reveals that the fine-tuned models named SVILA (Slovenian Vision Language Assistant) exhibit substantial performance gains on a variety of vision question answering, visual grounding, and optical character recognition tasks when compared to their baseline counterparts. This establishes our methodology as an effective blueprint for enhancing VLM capabilities in other less-resourced languages. The dataset is publicly available in the Slovenian language resource repository CLARIN.SI (http://hdl.handle.net/11356/2050) and both fine-tuned models are published on the Hugging Face platform (https://huggingface.co/GaMS-Beta/SVILA-1-12B and https://huggingface.co/GaMS-Beta/SVILA-1-4B).

Resources

Details