These are the trained adapters from my MSc dissertation "Welsh Text-to-SQL with Small Language Models: a Study of Penalty, Recovery and Transfer" (Wrexham University, 2026). Everything else, the Welsh benchmark, the training data, the notebooks, every score and the figures, is on GitHub:
An adapter is a small file of extra weights that training adds on top of a downloaded model, leaving the model itself unchanged. Six small models were each trained twice with QLoRA on Spider's training split, once in English and once in Welsh (machine translated), to write SQL for a question about a database. That gives 12 adapters, plus one repeat of one of them with a different random seed. Each is a folder here, named after its run:
folder
model
trained on
Welsh questions
English questions
gemma-3-4b-it_welsh_full_lf
Gemma 3 4B
Welsh
0.6587
0.7219
gemma-3-4b-it_english_full_lf
Gemma 3 4B
English
0.4225
0.6821
gemma-3-12b-it_welsh_full_lf
Gemma 3 12B
Welsh
0.7351
0.7715
gemma-3-12b-it_english_full_lf
Gemma 3 12B
English
0.6334
0.7686
Qwen3.5-4B_welsh_full_lf
Qwen 3.5 4B
Welsh
0.7078
0.7701
Qwen3.5-4B_english_full_lf
Qwen 3.5 4B
English
0.6150
0.7496
Qwen3.5-9B_welsh_full_lf
Qwen 3.5 9B
Welsh
0.7472
0.7817
Qwen3.5-9B_english_full_lf
Qwen 3.5 9B
English
0.6855
0.7759
Ministral-3-3B-Instruct-2512_welsh_full_lf
Ministral 3 3B
Welsh
0.6937
0.7574
Ministral-3-3B-Instruct-2512_english_full_lf
Ministral 3 3B
English
0.5513
0.7579
Ministral-3-8B-Instruct-2512_welsh_full_lf
Ministral 3 8B
Welsh
0.7569
0.7710
Ministral-3-8B-Instruct-2512_english_full_lf
Ministral 3 8B
English
0.6821
0.7846
Qwen3.5-4B_welsh_full_lf_s3408
Qwen 3.5 4B
Welsh, seed 3408
0.7297
The two score columns are execution accuracy on the 2,057 questions of the Spider test split, in Welsh and in English: the adapter's query and the correct query are both run against the database and the answer counts only if the same rows come back. 0.7472 means about 75 questions in a hundred. The Welsh questions are my own translation of Spider, in the GitHub repository under benchmark/welsh. For comparison, the models out of the box scored between 0.3369 and 0.5960 on the Welsh questions and between 0.5187 and 0.7453 on the English ones.
What is in each folder
adapter/adapter_model.safetensors, the weights (99 to 262 MB)
adapter/adapter_config.json, the LoRA settings, including the base model the adapter goes on
adapter/tokenizer.json, tokenizer_config.json, chat_template.jinja, processor_config.json, the tokenizer files saved with the adapter
adapter/trainer_state.json, trainer_log.jsonl, all_results.json, train_results.json, the loss curve and final numbers
training_summary.json, everything about the run: model, data file, every setting, library versions, GPU, loss, runtime, when it was saved
train_log.txt, the raw training log
training_summary.csv at the top is all thirteen runs in one table. Intermediate checkpoints are not uploaded.
How to use one
Download the folder you want, then load it in 4-bit on top of its base model. This is what the evaluation notebook in the GitHub repository does (code/eval and train/LF_eval_colab.ipynb has the full loading function, with a fallback path and the Gemma detail).
python
1from huggingface_hub import snapshot_download
23run ="Qwen3.5-9B_welsh_full_lf"4path = snapshot_download("Aled-caio/welsh-text2sql-slms-adapters", allow_patterns=[run +"/*"])5adapter_dir =f"{path}/{run}/adapter"67# with Unsloth (how the adapters were trained and scored)8from unsloth import FastModel
9model, tokenizer = FastModel.from_pretrained(model_name=adapter_dir, max_seq_length=4096, load_in_4bit=True)10FastModel.for_inference(model)1112# or with transformers and PEFT13# from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig14# from peft import PeftModel15# base = "unsloth/Qwen3.5-9B" # base_model_name_or_path in adapter/adapter_config.json16# tokenizer = AutoTokenizer.from_pretrained(adapter_dir)17# model = AutoModelForCausalLM.from_pretrained(base, device_map="auto",18# quantization_config=BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4"))19# model = PeftModel.from_pretrained(model, adapter_dir)
The prompt the adapters were trained on never changed. It is one user message with this shape (the schema is the CREATE TABLE statements of the database; the training data on GitHub shows the exact format in its schema_ddl field, and models/prompt.txt there gives it with its hash):
You are a text-to-SQL system. Given a database schema and a question, write one valid SQLite query that answers the question. Output only the SQL query, with no explanation and no markdown.
### Database schema:
CREATE TABLE "clwb" (
"id_clwb" NUMBER PRIMARY KEY,
"enw" TEXT,
"rheolwr" TEXT,
"capten" TEXT,
"gwneuthurwr" TEXT,
"noddwr" TEXT
);
CREATE TABLE "chwaraewr" (
"id_chwaraewr" NUMBER PRIMARY KEY,
"enw" TEXT,
"gwlad" TEXT,
"enillion" NUMBER,
"nifer_digwyddiadau" NUMBER,
"nifer_buddugoliaethau" NUMBER,
"id_clwb" NUMBER,
FOREIGN KEY ("id_clwb") REFERENCES "clwb" ("id_clwb")
);
### Question:
Faint o glybiau sydd?
### SQL:
Greedy decoding (do_sample=False) and 256 new tokens are what every score above was taken with. The model answers with SQL only; for the example above the right answer is SELECT count(*) FROM clwb.
How they were trained
QLoRA through LLaMA-Factory on Google Colab (one L4, or one A100 for the Gemma runs), 2 to 6 August 2026. One recipe for all thirteen: base model loaded in 4-bit (NF4), LoRA rank 16 and alpha 16, no dropout, learning rate 2e-4, two epochs, effective batch 8, warmup 3 per cent, weight decay 0.01, linear schedule, seed 3407 (3408 for the repeat), 8-bit AdamW, loss on the SQL answer only, sequences cut at 4,096 tokens, training arithmetic in bfloat16, each family's own chat template. Training data: 6,948 rows of Spider's training split, English as released or machine translated into Welsh with NLLB-200. Library versions and everything else are in each training_summary.json.
Licence
CC BY-SA 4.0, the same licence as Spider. The adapters also carry the terms of their base models (Gemma 3, Qwen 3.5, Ministral 3).
