from unsloth import FastLanguageModel
from transformers import AutoTokenizer

# Load base model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="ibm-granite/granite-3.0-2b-instruct",
    max_seq_length=2048,
    load_in_4bit=True,
)

# Load LoRA adapters
model = FastLanguageModel.get_peft_model(
    model,
    lora_path="asifdotpy/vetta-granite-2b-lora-v3",
    r=16,
    lora_alpha=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
)

# Enable inference
FastLanguageModel.for_inference(model)

# Generate
inputs = tokenizer("Begin a technical interview...", return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=256)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

Training Details

Base Model: ibm-granite/granite-3.0-2b-instruct
Training Method: LoRA fine-tuning
Dataset: Custom interview conversation dataset
Training Steps: 450
Final Loss: 0.2422

Intended Use

This model is designed to conduct professional AI-powered interviews, providing empathetic and technically accurate responses.