This model has been optimized using GRPO with a parse-based reward function that validates structured outputs (JSON, YAML, TOML, CSV, XML).
The reward function validates whether the generated output can be successfully parsed as the target format:
1from transformers import AutoModelForCausalLM, AutoTokenizer
2import torch
3
4model_id = "amu870/test-v3-sft-grpo"
5
6tokenizer = AutoTokenizer.from_pretrained(model_id)
7model = AutoModelForCausalLM.from_pretrained(
8 model_id,
9 torch_dtype=torch.float16,
10 device_map="auto"
11)
12
13# Test inference
14prompt = "Extract the following attributes from text and output JSON..."
15inputs = tokenizer.apply_chat_template(
16 [{"role": "user", "content": prompt}],
17 tokenize=True,
18 add_generation_prompt=True,
19 return_tensors="pt"
20).to("cuda")
21outputs = model.generate(inputs, max_new_tokens=512)
22print(tokenizer.decode(outputs[0]))
Apache 2.0. Users must follow the original base model's license terms.
