A fine-tuned version of TinyLlama/TinyLlama-1.1B-Chat-v1.0 trained on 5,000 samples from the Alpaca dataset using QLoRA (4-bit quantization + LoRA adapters). LoRA weights are fully merged into the base model — drop-in ready for inference with standard transformers, no PEFT required.

TinyLlama-1.1B is already a strong chat-capable base model. This fine-tune further reinforces Alpaca-style instruction following while keeping the model compact enough to run on a single consumer GPU or even CPU.

Model Details

Property	Value
Base model	`TinyLlama/TinyLlama-1.1B-Chat-v1.0`
Parameters	~1.1B
Fine-tuning method	QLoRA (4-bit NF4 + LoRA)
Dataset	`tatsu-lab/alpaca` (5,000 samples)
LoRA rank (r)	8
LoRA alpha	16
Target modules	`q_proj`, `v_proj`, `k_proj`, `o_proj`
Training epochs	1
Optimizer	`paged_adamw_8bit`
Precision	fp16
Max sequence length	512

Usage

python

from transformers import pipeline

pipe = pipeline("text-generation", model="sarimahsan101/tinyllama-alpaca-qlora")

prompt = "### Instruction:\nExplain what gravity is.\n\n### Response:\n"
output = pipe(
    prompt,
    max_new_tokens=150,
    do_sample=True,
    temperature=0.7,
    repetition_penalty=1.3,
    no_repeat_ngram_size=3,
)
print(output[0]["generated_text"])

Or load manually:

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "sarimahsan101/tinyllama-alpaca-qlora"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)

prompt = "### Instruction:\nWrite a short poem about the ocean.\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=150,
        do_sample=True,
        temperature=0.7,
        repetition_penalty=1.3,
        no_repeat_ngram_size=3,
        pad_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(out[0], skip_special_tokens=True))

Prompt Format

Trained with the standard Alpaca instruction format:

### Instruction:
<your instruction here>

### Response:

With optional input context:

### Instruction:
<your instruction here>

### Input:
<additional context here>

### Response:

Example Outputs

Prompt:

### Instruction:
What is machine learning?

### Response:

Response:

Machine learning is a branch of artificial intelligence that enables computers to
learn from data without being explicitly programmed. Instead of writing rules by
hand, you feed examples to an algorithm and it learns patterns on its own...

Training Details

Fine-tuned on Google Colab (T4 GPU, ~20–25 min) using:

transformers
peft
trl — SFTTrainer + SFTConfig
bitsandbytes
datasets

QLoRA freezes the base model in 4-bit NF4 precision and trains only low-rank adapter matrices injected into the attention layers. Trainable parameters are roughly 0.5% of total model parameters, making fine-tuning feasible on a free T4 GPU.

Comparison with OPT-125M Fine-tune

	OPT-125M (QLoRA)	TinyLlama-1.1B (QLoRA)
Parameters	125M	1.1B
Base capability	Weak	Strong (chat-pretrained)
Coherence	Basic	Good
Instruction following	Inconsistent	Consistent
GPU memory (inference)	~1GB	~2.5GB fp16
Recommended for	Learning the pipeline	Actual use

Limitations

Trained on 5K samples for 1 epoch — not exhaustive coverage of all instruction types
May hallucinate facts on knowledge-intensive questions
Not safety-aligned — apply your own guardrails for production use
English only

License

TinyLlama is released under the Apache 2.0 license. This fine-tune follows the same terms and is free for personal and commercial use.

sarimahsan101/TinyLlama-1.1B-Alpaca-QLoRA

TinyLlama-1.1B-Chat — QLoRA Fine-tuned on Alpaca