from unsloth import FastLanguageModel
import torch

# Load model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="rjml/gemma-3-270m-sports-finetune",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)

# Enable inference mode
FastLanguageModel.for_inference(model)

# Example usage
prompt = "What is the average points per game for a basketball player?"
inputs = tokenizer(prompt, return_tensors="pt")

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=128, temperature=0.1)
    
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Training Details

Base Model: unsloth/gemma-3-270m-it-unsloth-bnb-4bit
Fine-tuning Method: LoRA (Low-Rank Adaptation)
Dataset: Sports statistics queries (15k samples)
Task: Convert natural language sports queries to structured JSON

LoRA Rank: 16
LoRA Alpha: 32
Learning Rate: 3e-5
Batch Size: 4 per device
Epochs: 2
Gradient Accumulation: 4 steps

Intended Use

This model is designed to convert natural language sports statistics queries into structured JSON format for database queries or API calls.