You can deploy this model and start earning money today!

Tourist99/
lora-tinyllama

safetensors

Views

No views yet

lora-tinyllama

Overview

lora-tinyllama is a fine-tuned version of the tinyllama-1.1b model, created using LoRA (Low-Rank Adaptation). This model specializes in adapting the tinyllama-1.1b base for specific tasks with minimal computational overhead.

Key Features

Model Size: ~90MB (LoRA adapter weights only).
Efficiency: Keeps the base model frozen and adds small trainable layers.
Flexibility: Requires the original tinyllama-1.1b base model for usage.
Purpose: Designed for specialized NLP tasks, leveraging the compact and powerful nature of the base model.

Usage Instructions

Prerequisites

Before using lora-tinyllama, ensure you have:

The base model: tinyllama-1.1b.
The fine-tuned LoRA weights: lora-tinyllama.

Loading the Model

Here’s how to load and use lora-tinyllama with the base model:

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Step 1: Load the base model
base_model_path = "path/to/tinyllama-1.1b"
base_model = AutoModelForCausalLM.from_pretrained(base_model_path)

# Step 2: Load the LoRA weights
lora_model_path = "path/to/lora-tinyllama"
lora_model = PeftModel.from_pretrained(base_model, lora_model_path)

# Step 3: Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_path)

# Step 4: Use the model for inference
inputs = tokenizer("Hello, world!", return_tensors="pt")
outputs = lora_model.generate(inputs["input_ids"])
print(tokenizer.decode(outputs[0]))

Tourist99/lora-tinyllama