You can deploy this model and start earning money today!

jlm429/
birdhouse-in-your-soul-lora-demo

Name: birdhouse-in-your-soul-lora-demo
Availability: InStock

Qwen/Qwen2.5-3B-Instruct

adapter

apache-2.0

endpoints_compatible

Views

No views yet

LoRA Fine-Tuning Demo - Bluebird House (Synthetic Structured Data)

Overview

This repository contains a LoRA adapter fine-tuned on a small synthetic dataset representing structured facts about a fictional “Bluebird House.”

The goal of this project is to demonstrate:

LoRA / PEFT fine-tuning workflow
Structured output learning
Controlled before/after behavior on domain-specific data

This is intentionally a minimal, reproducible example.

Base Model

Qwen/Qwen2.5-3B-Instruct
Architecture: decoder-only causal language model

What This Demonstrates

This project highlights how a small LoRA adapter can override base model priors and inject domain-specific knowledge.

Before fine-tuning, the base model produces generic or incorrect answers.
After fine-tuning, the model reproduces structured, dataset-specific facts.

Example

Prompt

Where is the main water shutoff in the Bluebird House?

Before (base model)

The main water shutoff is typically located under the kitchen sink...

After (LoRA)

Location: basement utility room
Details: north wall, next to the red pressure tank

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "Qwen/Qwen2.5-3B-Instruct"
adapter = "jlm429/birdhouse-in-your-soul-lora-demo"

tokenizer = AutoTokenizer.from_pretrained(base_model)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    device_map="auto",
    trust_remote_code=True
)

model = PeftModel.from_pretrained(model, adapter)

Notes

Dataset is synthetic and intentionally small
Model is expected to memorize structured facts
Demonstrates LoRA workflow rather than generalization

Related Resources

Dataset: https://huggingface.co/datasets/jlm429/bluebird-house-synthetic-dataset
Demo (Space): https://jlm429-bluebird-house-demo.hf.space/