You can deploy this model and start earning money today!

JoshyG14/
qwen3-1.7b-spanish-lora-with-inject

Name: qwen3-1.7b-spanish-lora-with-inject
Availability: InStock

Views

No views yet

Qwen3-1.7B LoRA-with-inject

LoRA adapter for Qwen/Qwen3-1.7B, trained for Spanish sentence generation under morphosyntactic constraints with the gold inflected form supplied in the prompt. This is Experiment A (LoRA-with-inject) from the MSc project: Grammatically Controlled Sentence Generation with Small Language Models for Language Learning.

Code: github.com/JoshuaGraham14/LinguistOS.

Companion adapter trained without form injection: JoshyG14/qwen3-1.7b-spanish-lora-no-inject.

Intended use

Given a Spanish verb lemma, morphosyntactic constraints (tense, person), and the required surface form, generate a short natural sentence in which that form is the main verb.

The adapter is not a general-purpose Spanish chat model. Load it on top of Qwen/Qwen3-1.7B and use the same form-injection prompt format as at training time (see the repository configs).

Training


Base model	`Qwen/Qwen3-1.7B` (Apache 2.0)
Method	LoRA, rank (r=16), (\alpha=32), dropout 0.05
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
Data	4,205 filtered prompt–completion pairs from the 150-verb diagnostic grid (3,995 train / 210 val; hard cells oversampled (\times 2))
Objective	Supervised fine-tuning (next-token CE on prompt–completion pairs)
Checkpoint	Best validation loss at 1,000 steps (early stopping); 3 epochs, lr (2\times10^{-4})
Adapter size	~67 MB

Load

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen3-1.7B"
adapter_id = "JoshyG14/qwen3-1.7b-spanish-lora-with-inject"

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(model, adapter_id)

Framework versions

PEFT 0.19.1 · TRL 1.8.0 · Transformers 5.13.0 · PyTorch 2.12.1