from transformers import AutoTokenizer, AutoModelForCausalLM

# Load model
tokenizer = AutoTokenizer.from_pretrained("terrytaylorbonn/431_smollm-model")
model = AutoModelForCausalLM.from_pretrained("terrytaylorbonn/431_smollm-model")

# Generate text
messages = [{"role": "user", "content": "Your prompt here"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=200)
response = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print(response)

Training Configuration

Base Model: HuggingFaceTB/SmolLM2-1.7B-Instruct
Fine-tuning Method: LoRA (Low-Rank Adaptation)
Hardware: NVIDIA RTX 3050 (4GB VRAM)
Framework: PyTorch + Transformers + PEFT

Batch Size: 1 (with gradient accumulation)
Learning Rate: 2e-4
LoRA Rank: 16
LoRA Alpha: 32
Training Steps: Variable based on dataset

Limitations

This model inherits the limitations of the base model and may have additional biases from the fine-tuning data.