This model is intended for assistant-style chat interactions. Because it is a 1.1B parameter model, it may struggle with complex reasoning or long-form factual accuracy compared to larger models like Llama-3 70B.

Training Hyperparameters

Developed by: [Abhishek Patel]
Model type: Causal Language Model (TinyLlama architecture)
Language(s): English
License: Apache 2.0
Fine-tuning technique: QLoRA (4-bit)

The following hyperparameters were used during training:

Learning rate: 2e-4
Batch size: 1
Optimizer: Paged AdamW (32-bit)
LR Scheduler: Cosine
Epochs: 1
LoRA Rank (r): 64
LoRA Alpha: 16
LoRA Dropout: 0.1

How to use

You can use this model with the peft and transformers libraries:

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model_id = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
adapter_id = "abhishekpatel10/Llama-2-7b-chat-finetune"

# Load base model and adapter
model = AutoModelForCausalLM.from_pretrained(base_model_id, torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, adapter_id)
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

# Generate text
prompt = "### Human: How do I bake a cake? ### Assistant:"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))