from transformers import pipeline

question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
generator = pipeline("text-generation", model="andresnowak/Qwen3-0.6B-instruction-finetuned_v2", device="cuda")
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])

Training procedure

Visualize in Weights & Biases

This model was trained with SFT, and the idea was to train only on completion loss and we removed all examples that have a combined (prompt and completion) token size bigger than 2048

Training arguments

yaml

defaults:
  - override hydra/job_logging: disabled

environment:
  seed: 42
  use_template: True

model:
  name: Qwen/Qwen3-0.6B-Base
  hub_model_id: andresnowak/Qwen3-0.6B-instruction-finetuned_v2

# Hardcoded subset dataset is just to make the model answer that is from allenai tulu basically
dataset:
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: codeAlpaca
    size: 0.3
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: noRobots
    size: 0.8
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: openMathGsm8k
    size: 0.5
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: codeV2
    size: 0.3
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: flanV2
    size: 0.8
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: ifData
    size: 0.8
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: mathAlgebra 
    size: 0.4
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: mathGrade
    size: 0.4
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: oasst1
    size: 0.4
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: sciriff
    size: 0.8
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: tableGpt
    size: 0.2
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: tirMath
    size: 0.5
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: wildChat
    size: 0.6
  - name: andresnowak/Instruction-finetuning-mixture-mnlp
    config: mathV5
    size: 0.3

dataset_evaluation:
  - name: cais/mmlu
    config: validation
    subjects: ["abstract_algebra", "anatomy", "astronomy", "college_biology", "college_chemistry", "college_computer_science", "college_mathematics", "college_physics", "computer_security", "conceptual_physics", "electrical_engineering", "elementary_mathematics", "high_school_biology",  "high_school_chemistry", "high_school_computer_science", "high_school_mathematics", "high_school_physics", "high_school_statistics", "machine_learning"]

training:
  output_dir: ./output
  logging_dir: ./logs
  resume_dir: None
  report_to: wandb
  learning_rate: 0.00001 # Default value instead of 5e-6
  per_device_train_batch_size: 4
  per_device_eval_batch_size: 4
  gradient_accumulation_steps: 32 # to get effective 128
  num_train_epochs: 2
  weight_decay: 0.00
  warmup_ratio: 0.03
  max_grad_norm: 1.0
  # linear_layers_max_grad_norm: 0.5
  lr_scheduler: "linear"
  completion_only_loss: True

wandb:
  project: MNLP-qwen-instruction-finetuning
  name: qwen-instruction-finetuning_v2

Evaluation results

The model was evaluated on a suite of Multiple Choice Question Answering (MCQA) benchmarks (on its validation and test sets repsectively for each one), and NLP4education is only the approximated 1000 question and answers given to use.

The performance on the MCQA benchmarks is:

First evaluation: The tests where done with this prompt (type 5):

This question assesses challenging STEM problems as found on graduate standardized tests. Carefully evaluate the options and select the correct answer.

---
[Insert Question Here]
---
[Insert Choices Here, e.g.:
A. Option 1
B. Option 2
C. Option 3
D. Option 4]
---

Your response should include the letter and the exact text of the correct choice.
Example: B. Entropy increases.
Answer:

And the teseting was done on [Letter]. [Text answer]

Benchmark	Accuracy (Acc)	Normalized Accuracy (Acc Norm)
ARC Challenge	57.99%	55.61%
ARC Easy	75.02%	69.69%
GPQA	32.59%	30.13%
Math QA	22.39%	21.59%
MCQA Evals	38.70%	36.62%
MMLU	46.11%	46.11%
MMLU Pro	13.57%	11.50%
MuSR	42.99%	41.93%
NLP4Education	41.75%	39.80%
Overall	41.23%	39.22%

Second evaluation: (type 0)

The following are multiple choice questions (with answers) about knowledge and skills in advanced master-level STEM courses.

---
*[Insert Question Here]*
---
*[Insert Choices Here, e.g.:*
*A. Option 1*
*B. Option 2*
*C. Option 3*
*D. Option 4]*
---
Answer:

And the teseting was done on [Letter]. [Text answer]

Benchmark	Accuracy (Acc)	Normalized Accuracy (Acc Norm)
ARC Challenge	60.23%	58.87%
ARC Easy	78.11%	74.88%
GPQA	31.47%	28.35%
Math QA	24.80%	24.83%
MCQA Evals	41.30%	37.92%
MMLU	46.05%	46.05%
MMLU Pro	14.97%	13.64%
MuSR	42.99%	41.93%
NLP4Education	44.84%	42.65%
Overall	42.75%	41.01%

Third evaluation: (type 2)


This is part of an assessment on graduate-level science, technology, engineering, and mathematics (STEM) concepts. Each question is multiple-choice and requires a single correct answer.

---
*[Insert Question Here]*
---
*[Insert Choices Here, e.g.:*
*A. Option 1*
*B. Option 2*
*C. Option 3*
*D. Option 4]*
---
For grading purposes, respond with: [LETTER]. [VERBATIM TEXT]
Example: D. Planck constant
Your Response:

And the teseting was done on [Letter]. [Text answer]

Benchmark	Accuracy (Acc)	Normalized Accuracy (Acc Norm)
ARC Challenge	44.39%	44.39%
ARC Easy	61.78%	61.78%
GPQA	23.44%	23.44%
Math QA	23.33%	23.33%
MCQA Evals	34.81%	34.81%
MMLU	45.99%	45.99%
MMLU Pro	14.09%	14.09%
MuSR	45.50%	45.50%
NLP4Education	34.91%	34.91%
Overall	36.47%	36.47%

First evaluation: (type 0)

The following are multiple choice questions (with answers) about knowledge and skills in advanced master-level STEM courses.

---
*[Insert Question Here]*
---
*[Insert Choices Here, e.g.:*
*A. Option 1*
*B. Option 2*
*C. Option 3*
*D. Option 4]*
---
Answer:

And the teseting was done on [Letter]

Benchmark	Accuracy (Acc)	Normalized Accuracy (Acc Norm)
ARC Challenge	62.20%	62.20%
ARC Easy	79.23%	79.23%
GPQA	29.02%	29.02%
Math QA	25.39%	25.39%
MCQA Evals	43.90%	43.90%
MMLU	46.02%	46.02%
MMLU Pro	16.37%	16.37%
MuSR	45.50%	45.50%
NLP4Education	46.25%	46.25%
Overall	43.76%	43.76%

Framework versions

TRL: 0.18.1
Transformers: 4.52.4
Pytorch: 2.7.0
Datasets: 3.6.0
Tokenizers: 0.21.0

Citations

Cite TRL as:

bibtex

@misc{vonwerra2022trl,
	title        = {{TRL: Transformer Reinforcement Learning}},
	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
	year         = 2020,
	journal      = {GitHub repository},
	publisher    = {GitHub},
	howpublished = {\url{https://github.com/huggingface/trl}}
}