You can deploy this model and start earning money today!

kaliyo22/
lora-finetuned

Name: lora-finetuned
Availability: InStock

TinyLlama/TinyLlama-1.1B-Chat-v1.0

adapter

apache-2.0

Views

No views yet

Fine-tuned TinyLlama Model

Model Description

This is a fine-tuned version of TinyLlama-1.1B-Chat using LoRA (Low-Rank Adaptation) for improved conversational capabilities on custom dataset.

Developed by: Makwana Mukesh
Funded by: Makwana Kaliyo M
Shared by: Makwana Kaliyo M
Model type: Causal Language Model (Text Generation)
Language(s): English
License: Apache 2.0
Finetuned from model: TinyLlama/TinyLlama-1.1B-Chat-v1.0

Model Sources

Base Model: TinyLlama/TinyLlama-1.1B-Chat-v1.0

Training Details

Training Data

Custom dataset formatted with prompt-response pairs:

### Prompt:
{user_input}

### Response:
{model_response}

Training Procedure

Fine-tuning method: LoRA (Low-Rank Adaptation)
Framework: Hugging Face Transformers + PEFT
Hardware: [GPU/CPU used]
Training time: [Duration]

Training Hyperparameters

LoRA rank (r): 8
LoRA alpha: 16
LoRA dropout: 0.05
Target modules: q_proj, v_proj
Learning rate: 2e-4
Batch size: [Your batch size]
Epochs: [Number of epochs]
Optimizer: AdamW

How to Use

Installation

pip install torch transformers peft accelerate

Inference

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load model
base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
model = PeftModel.from_pretrained(base_model, "./lora-finetuned")
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")

# Generate response
prompt = "### Prompt:\nHello, how are you?\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Limitations and Bias

Model is fine-tuned on a specific dataset and may not generalize to all use cases
Inherits limitations from the base TinyLlama model
May produce biased or incorrect outputs
Should not be used for critical decision-making without human oversight

Ethical Considerations

This model should be used responsibly
Users should be aware of potential biases in the training data
Not suitable for generating harmful, illegal, or misleading content

Citation

bibtex

@misc{tinyllama-finetuned-2025,
  author = {Makwana Mukesh},
  title = {Fine-tuned TinyLlama Model},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{[Your model URL]}}
}

Acknowledgments

Base model: TinyLlama Team
Training framework: Hugging Face Transformers and PEFT
Funded by: Makwana Kaliyo M

kaliyo22/lora-finetuned