The adapter teaches the model to respond to tool-use requests with clean, parseable JSON function calls (no reasoning traces — training data was templated with enable_thinking=False).

Training setup


Hardware	2 nodes x 8 NVIDIA H200 (16 GPUs), InfiniBand interconnect
Platform	Nebius AI Cloud, Slurm via Soperator (Slurm-on-Kubernetes)
Strategy	DDP with LoRA (frozen base, adapter-only gradient sync)
LoRA	r=16, alpha=32, dropout 0.05, all attention + MLP projections
Trainable params	134.2M of 32.9B (0.41%)
Epochs	3 (360 steps, packed 4096-token sequences)
Precision	bf16, SDPA attention, gradient checkpointing
Wall clock	78 minutes
Final train loss	0.085
Eval loss / token accuracy	0.083 / 98.2% (500 held-out examples)

DDP was chosen over FSDP deliberately: with the 32B base frozen, FSDP would shard and all-gather weights that never change. Full replication per GPU (~64 GB of the 141 GB H200 budget) with adapter-only gradient all-reduce keeps cross-node traffic in the megabytes per step.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-32B", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "shwnbrns/qwen3-32b-xlam-function-calling-lora")
tokenizer = AutoTokenizer.from_pretrained("shwnbrns/qwen3-32b-xlam-function-calling-lora")

Or serve with vLLM as a LoRA module:

bash

vllm serve Qwen/Qwen3-32B --enable-lora \
  --lora-modules xlam-ft=shwnbrns/qwen3-32b-xlam-function-calling-lora

Prompt with available tools in the system message; the model answers tool-requiring queries with a JSON list of {"name": ..., "arguments": ...} calls and plain text otherwise.