LoRA adapter for Trendyol/Trendyol-LLM-Asure-12B, trained with GRPO (RLVR) on a custom Turkish e-commerce attribute-extraction reward function across 8 fields (vs 4 fields in the v0.6 sibling adapter).

Two adapters, two use cases

If you only need the 4 most common fields, the v0.6 adapter (KocKaankk/asure-12b-tr-attr-grpo) is the better choice — it beats Claude Opus 4.7 by +0.026 macro F1 on a 578-row out-of-distribution test (P<0.001).

This v0.7 adapter trades broader field coverage (8 fields instead of 4) for a slight in-distribution loss to Claude (-0.026 macro F1, not statistically significant). On OOD, our 8-field adapter still beats the base by +0.049 (P<0.001), but Claude's 8-field OOD score is currently unmeasured (API credits ran out partway during eval).

Headline result

In-distribution test (200 rows held-out from training)

model	macro F1	95% CI	Renk	Materyal	Kalıp	Desen	Kumaş Tipi	Yaka Tipi	Kol Tipi	Bel
Asure-12B base + v4 prompt	0.382	[0.342, 0.421]	0.460	0.190	0.278	0.475	0.361	0.628	0.417	0.264
Claude Opus 4.7 v4 (frontier reference)	0.474	[0.438, 0.508]	0.444	0.158	0.258	0.574	0.429	0.721	0.593	0.621
Asure-12B + this adapter (step 250)	0.444	—	0.449	0.189	0.276	0.571	0.427	0.676	0.390	0.575
Asure-12B + this adapter, step 150	0.448	[0.410, 0.484]	0.450	0.196	0.286	0.571	0.434	0.676	0.390	0.583

Δ vs Claude on in-dist (step 150): -0.026, paired-bootstrap CI [-0.054, +0.001], P=0.97 — Claude wins by a hair, CI just barely includes zero.

The entire gap is on Kol Tipi (sleeve type), where Claude is +0.203 ahead. On all other 7 fields combined, the trained adapter is roughly at parity with Claude.

Out-of-distribution (578 rows from 18 categories never seen during training)

model	macro F1 OOD	95% CI	notes
Asure-12B base + v4 prompt	0.323	[0.277, 0.369]
Asure-12B + this adapter (step 250)	0.372	[0.325, 0.418]	+0.049 over base, P<0.001
Claude Opus 4.7 v4	not measured	—	Anthropic credits ran out during eval

Per-field decomposition

field	base v4	Claude v4	this adapter	Δ vs base	Δ vs Claude
Renk	0.460	0.444	0.450	-0.010	+0.006
Materyal	0.190	0.158	0.196	+0.006	+0.038
Kalıp	0.278	0.258	0.286	+0.008	+0.028
Desen	0.475	0.574	0.571	+0.096	-0.003
Kumaş Tipi	0.361	0.429	0.434	+0.073	+0.005
Yaka Tipi	0.628	0.721	0.676	+0.048	-0.045
Kol Tipi	0.417	0.593	0.390	-0.027	-0.203 ← see caveats
Bel	0.264	0.621	0.583	+0.319	-0.038

Where this adapter wins or matches: Materyal, Kalıp, Renk, Kumaş Tipi, Desen.

Where this adapter loses: Yaka Tipi, Kol Tipi (much), Bel (slightly).

The Bel win is the headline: GRPO learned the catalog default Normal Bel (modal value at 69%) and lifted F1 from 0.264 → 0.583 — same dynamic as v0.6's Desen → Düz story but ~3x bigger.

Intended use

Given a Turkish product title (and optionally a brand), produce a strict JSON object with up to eight catalog attributes:

Renk (color), Materyal (material), Kalıp (fit), Desen (pattern) — same as v0.6.
Kumaş Tipi (fabric type) — "Dokuma" (woven) is the catalog default for most clothing.
Yaka Tipi (neckline) — TOPS ONLY; emit null for everything else.
Kol Tipi (sleeve type) — TOPS ONLY; emit null for sleeveless or non-tops.
Bel (waist style) — PANTS/SKIRTS ONLY; emit null elsewhere; default "Normal Bel".

Use null (literal JSON null, not the string "null") when an attribute can't be inferred or doesn't apply to the product type.

Example output

Input: Marka: Mavi\nÜrün adı: Erkek Slim Fit %100 Pamuk Basic Tişört Beyaz

Output:

json

{"Renk": "Beyaz", "Materyal": "Pamuklu", "Kalıp": "Slim", "Desen": "Düz",
 "Kumaş Tipi": "Örme", "Yaka Tipi": "Bisiklet Yaka", "Kol Tipi": "Kısa Kol", "Bel": null}

Usage

This is a PEFT/LoRA adapter — load it on top of the base model.

With vLLM (recommended for serving)

bash

vllm serve Trendyol/Trendyol-LLM-Asure-12B \
    --enable-lora \
    --lora-modules grpo-v0.7=KocKaankk/asure-12b-tr-attr-grpo-v0.7-8field \
    --max-lora-rank 16 \
    --served-model-name asure-12b-grpo-v0.7 \
    --dtype bfloat16 --max-model-len 4096

Then call the OpenAI-compatible endpoint with model="asure-12b-grpo-v0.7". Use the v4 prompt — there is no other supported prompt for this adapter; see eval/prompts.py (V4 constant).

With transformers + PEFT

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base = AutoModelForCausalLM.from_pretrained(
    "Trendyol/Trendyol-LLM-Asure-12B",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, "KocKaankk/asure-12b-tr-attr-grpo-v0.7-8field")
tok = AutoTokenizer.from_pretrained("KocKaankk/asure-12b-tr-attr-grpo-v0.7-8field")

Training

	value
algorithm	GRPO via TRL
base model	Trendyol/Trendyol-LLM-Asure-12B (12B params, Turkish base)
adapter	LoRA, rank=16, alpha=32, dropout=0.05, target_modules="all-linear"
quantization (training)	4-bit NF4 (QLoRA, double-quant, bf16 compute)
reward function	per-row macro F1 across 8 fields, strict normalized exact-match
reward parser	strict JSON; markdown fences stripped; case-canonicalized field names
rollouts	vLLM colocated, G=8 generations per prompt, T=0.8
optimizer	AdamW 8-bit
learning rate	1.5e-5, cosine schedule, 10% warmup
KL coefficient (beta)	0.02
effective batch	4 prompts × 8 generations × 2 grad-accum = 64 rollouts per gradient update
training steps	300 (saved every 50, this is checkpoint-250 — peak by held-out OOD F1)
training data	2,009 Turkish Trendyol product titles + Trendyol-provided gold attribute labels
held-out test	200 prompts (deterministic SHA1 split, frozen across v0.6 and v0.7 — same SHA256)
compute	1× NVIDIA L40S (48 GB), AWS `g6e.4xlarge`, ~3:00 wallclock
training cost	~$7 in on-demand GPU

Held-out checkpoint sweep on in-dist test

step	macro F1 in-dist	Δ vs base
base	0.382	ref
50	0.410	+0.026
100	0.418	+0.034
150	0.448	+0.064 (peak — tied with step 300)
200	0.441	+0.057
250	0.444	+0.060 (peak by OOD F1 — this published checkpoint)
300	0.448	+0.064 (slight overfit to in-dist)

We publish step 250 (peak OOD) rather than step 150 (peak in-dist) because OOD generalization is the better signal for production use. The two are within 0.004 in-dist anyway.

Limitations

Underperforms Claude on Kol Tipi by 0.203 F1. The model became overcautious — it learned to emit null when uncertain rather than guessing the modal value. Recall is 0.283 vs Claude's 0.660; precision (when it does emit) is actually higher than Claude (0.625 vs 0.538). This is the entire reason v0.7 doesn't beat Claude in-dist. A targeted v0.8 prompt fix is planned.
Domain-specific. Trained on 2,209 products from ~36 Trendyol clothing/footwear/bag categories. Tested on 18 OOD categories including home textiles. Not tested on electronics, food, books, etc.
Field applicability is category-specific. The model has to learn that Yaka/Kol Tipi apply only to tops and Bel only to pants/skirts. It does this correctly most of the time, but when in doubt about Kol Tipi specifically, it defaults to null too aggressively (see above).
Capability narrowing. RL on a narrow reward function reduces the model's general-Turkish capabilities. Don't use this adapter for free-form Turkish chat.
Prompt-locked. Only works well with the v4 prompt above.
OOD comparison vs frontier is incomplete. Claude Opus 4.7 OOD (8-field, v4 prompt) was not measured this round because Anthropic credits exhausted at row 189/578.

Reproducibility

Full code, dataset scrapers, training script, eval pipeline, Terraform infra: github.com/KocKaankk/tr-rl (repo will be live once published).

The training run that produced this adapter is documented end-to-end in notes/v0.7-8field.md of that repo.

Citation

If you use this adapter in research:

bibtex

@misc{karakas2026asuregrpov07,
  author       = {Karakaş, Kaan},
  title        = {Asure-12B + GRPO v0.7: Turkish e-commerce 8-field attribute extraction},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/KocKaankk/asure-12b-tr-attr-grpo-v0.7-8field}},
}

License

CC BY-NC 4.0 — inherited from the base model (Trendyol/Trendyol-LLM-Asure-12B).