You can deploy this model and start earning money today!

Goekdeniz-Guelmez/
Qwen3.5-0.8B-QAT-Demo

Name: Qwen3.5-0.8B-QAT-Demo
Availability: InStock

Views

No views yet

Qwen3.5-0.8B-QAT-Demo

Created with MLX-LoRA-Studio · Created with MLX LoRA Studio

Overview

Repository: Goekdeniz-Guelmez/Qwen3.5-0.8B-QAT-Demo
Asset type: model
Created at: 2026-06-19 15:41:45 UTC
Model upload mode: mergedWeights
Base model: /Users/Goekdeniz.Guelmez@computacenter.com/.omlx/models/mlx-community/Qwen3.5-0.8B-bf16
Algorithm: sft
Dataset: /Users/Goekdeniz.Guelmez@computacenter.com/.mlxlorastudio/runs/synthetic-sft-ultrafeedback-prompts-flat-rlhf-2026-06-19-173721/generated-data
Training length: 2 epochs
Final train loss: 0.5110
Final validation loss: 2.0305

This repository was prepared by MLX LoRA Studio from local training outputs.

Training Settings

Model & Data

Field	Value
Model	`/Users/Goekdeniz.Guelmez@computacenter.com/.omlx/models/mlx-community/Qwen3.5-0.8B-bf16`
Dataset	/Users/Goekdeniz.Guelmez@computacenter.com/.mlxlorastudio/runs/synthetic-sft-ultrafeedback-prompts-flat-rlhf-2026-06-19-173721/generated-data
Adapter path	/Users/Goekdeniz.Guelmez@computacenter.com/.mlxlorastudio/runs/Qwen3.5-0.8B-bf16-lora-sft-2026-06-19-173929/adapters

Algorithm

Field	Value
Mode	sft
Train type	lora
Optimizer	adamw
Quantization	4-bit

Optimisation

Field	Value
Learning rate	1e-05
LR schedule	constant
Batch size	1
Grad accumulation	1
Iters / epochs	2 epochs
Val batches	1
Max seq length	128

LoRA Parameters

Field	Value
Rank	8
Scale	20
Dropout	0

Reporting & Saving

Field	Value
Steps per report	1
Steps per eval	20
Save every	100

Toggles

Field	Value
Grad checkpoint	✅
Efficient long context	❌
Mask prompt	❌
Fuse	✅

Preference / RL / GRPO

Field	Value
β (beta)	0.1
Loss type	sigmoid
δ (delta)	50
Reward scaling	1

Quantization-Aware Training

Field	Value
Bits	4
Group size	32
Start step	1
Interval	1

Training Metrics

Final train loss: 0.5110 (min 0.1080)
Final val loss: 2.0305 (min 1.0625)
Final learning rate: 1.000e-05
Peak memory: 1.69 GB

Loss Curve

Sparkline

Train loss: ▇▇█▅▆▇▄▅▅▅▆▇▅▄▅█▅▆▆▆▅▃▃▃▅▃▇▅▇▅▃▂▄▅▄▁▄▅▄▂
Validation loss: ▇▅▃▃▃▆▁█▄▅

Key Metrics

Metric	Latest	Min	Max	N
`it_s`	247.0000	58.0000	250.0000	176
`learning_rate`	1.00e-05	1.00e-05	1.00e-05	176
`loss`	0.5110	0.1080	3.7760	176
`peak_mem`	1.6931	1.6931	1.6931	176
`tok_s`	1.6900	1.6900	1.6900	176
`trained_tok`	2.19e+04	1.27e+02	2.19e+04	176
`val_loss`	2.0305	1.0625	2.5901	10
`val_time`	0.0475	0.0469	0.0902	10

Reproducibility

The full run_spec.json used to launch this run is included in the repository. Re-running the same spec on the same model(s), source dataset, and generation settings should reproduce an equivalent artifact (up to sampling and kernel-level non-determinism).

About

MLX-LoRA-Studio — Created with MLX LoRA Studio

MLX LoRA Studio is a SwiftUI desktop app for fine-tuning open language models on Apple Silicon with the mlx-lm-lora trainer. Curated by MLX-LoRA-Studio.

License

The license of the upstream base model(s), source dataset, generated dataset, and any included tokenizer files applies. Check the source model and dataset cards before redistribution or downstream training.

Goekdeniz-Guelmez/Qwen3.5-0.8B-QAT-Demo