Instead of training the full model weights, only small LoRA adapter layers were trained, making the model significantly smaller and more efficient to train.

Model Details

Base Model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
Fine-Tuning Method: LoRA / PEFT
Task: Text Generation / Classification
Intended Use:
- Educational
- Learning PEFT/LoRA
- Small demo classification project

What Is LoRA?

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique.

Instead of updating all model weights:

The original TinyLlama weights remain frozen
Small trainable adapter layers are added
Only the adapter weights are trained

Benefits:

Much smaller model size
Faster training
Lower VRAM usage
Easier sharing and deployment

Healthy Foods Dataset

Dataset:

healthyfoods

Prompt Format

The model was trained using this prompt format:

text

### Question:
Is Apples healthy or unhealthy?

### Answer:
Healthy

Another example:

text

### Question:
Is French Fries healthy or unhealthy?

### Answer:
Unhealthy

Example Python Usage

Install dependencies:

pip install transformers peft torch

Load the model:

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"

# Load base TinyLlama model
base_model = AutoModelForCausalLM.from_pretrained(base_model_name)

# Load LoRA adapter
model = PeftModel.from_pretrained(
    base_model,
    "YOUR_USERNAME/health-food-demo-lora"
)

tokenizer = AutoTokenizer.from_pretrained(
    "YOUR_USERNAME/health-food-demo-lora"
)

prompt = """
### Question:
Is Pizza healthy or unhealthy?

### Answer:
"""

inputs = tokenizer(prompt, return_tensors="pt")

output = model.generate(
    **inputs,
    max_new_tokens=10
)

print(tokenizer.decode(output[0], skip_special_tokens=True))

Example Output

text

### Question:
Is Pizza healthy or unhealthy?

### Answer:
Unhealthy

Training

The model was trained using supervised fine-tuning with custom food classification examples.

Example dataset format:

text

### Question:
Is Salmon healthy or unhealthy?

### Answer:
Healthy

Training characteristics:

Prompt tokens were masked during loss computation
The model only learned to predict answer tokens
LoRA adapters were trained instead of full model weights

LoRA Configuration

Example LoRA settings used:

python

LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none"
)

GGUF Support

This repository may also include a GGUF LoRA adapter for llama.cpp-compatible runtimes.

Compatible tools:

llama.cpp
LM Studio
KoboldCpp
Text Generation WebUI

Example llama.cpp usage:

bash

llama-cli \
  -m TinyLlama-1.1B-Chat-v1.0-Q4_K_M.gguf \
  --lora health_food_demo_lora.gguf \
  -p "### Question: Is Pizza healthy or unhealthy?\n\n### Answer:"

Files

File	Description
`adapter_model.safetensors`	LoRA adapter weights
`adapter_config.json`	LoRA configuration
`tokenizer.json`	Tokenizer
`README.md`	Documentation

Optional:

File	Description
`health_food_demo_lora.gguf`	GGUF LoRA adapter for llama.cpp

Limitations

Small educational demo model
Very limited dataset
May hallucinate or answer incorrectly
Not suitable for medical or nutritional advice

License

MIT

jmarianodc/health_food_demo_lora

Health Food Classifier (LoRA / PEFT)

Model Details

What Is LoRA?

Healthy Foods Dataset

Prompt Format

Example Python Usage

Example Output

Training

LoRA Configuration

GGUF Support

Files

Limitations

License

jmarianodc/
health_food_demo_lora