from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "pravindr/llama-3.1-8b-emvlogs-finetune-dataset4",
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("pravindr/llama-3.1-8b-emvlogs-finetune-dataset4")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "pravindr/llama-3.1-8b-emvlogs-finetuned")

# Generate text
prompt = "Your prompt here..."
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=100)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Training Details

Base Model: pravindr/llama-3.1-8b-emvlogs-finetune-dataset4
Training Dataset: RahulYevle/sample_training_dataset_7
Fine-tuning Method: LoRA (Low-Rank Adaptation)
Quantization: 4-bit (NF4)
LoRA Rank: 16
LoRA Alpha: 32
Training Epochs: 2
Learning Rate: 0.0002

This model was fine-tuned using parameter-efficient fine-tuning (PEFT) with LoRA adapters.

License

This model inherits the license from the base model. Please check the base model's license for usage terms.