You can deploy this model and start earning money today!

hungryfoxz/
Llama-3.2-1B-ViLA

Name: Llama-3.2-1B-ViLA
Availability: InStock

safetensors

hungryfoxz/LibraryFAQ1000

meta-llama/Llama-3.2-1B-Instruct

finetune

apache-2.0

Views

No views yet

🦙 Llama-3.2-1B-ViLA

ViLA (Virtual Library Assistant) is a fine-tuned version of Meta's Llama-3.2-1B-Instruct model, optimized to answer library-related questions accurately and concisely. It has been instruction-tuned on a curated dataset of library FAQs, making it a lightweight yet capable assistant for library information services.

📋 Model Details

Attribute	Details
Base Model	`meta-llama/Llama-3.2-1B-Instruct`
Model Size	1 Billion parameters
Context Length	128K tokens
Fine-tuning Method	LoRA (Low-Rank Adaptation)
Training Dataset	`hungryfoxz/LibraryFAQ1000`
Framework	Hugging Face Transformers + PEFT + TRL
License	Llama 3.2 Community License (inherited from base model)

🎯 Intended Uses

This model is designed for:

Library Information Services – Answering questions about memberships, policies, services, and general library operations
On-Device Chatbots – Lightweight enough to run on edge devices and mobile platforms
Educational Demonstrations – Showcasing fine-tuning of small language models for domain-specific tasks

The model is not intended for:

General-purpose conversation beyond library-related topics
Multimodal tasks (text-only model)
High-stakes decision-making

🏋️ Training Details

Dataset

The model was fine-tuned on the hungryfoxz/LibraryFAQ1000 dataset, which contains over 1,000 question-answer pairs covering common library inquiries.

Training Configuration

Parameter	Value
LoRA Rank (`r`)	16
LoRA Alpha (`lora_alpha`)	32
LoRA Dropout	0.05
Target Modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
Epochs	3
Batch Size	1 (with gradient accumulation over 8 steps)
Learning Rate	2e-4
Scheduler	Cosine with 3% warmup
Precision	FP16

Training Time

~10 minutes on Apple Silicon (M2) hardware.

🚀 Quick Start

Installation

If possible create a python environment for installing the packages. After activating the environment, create a file within the working directory, paste the code from the two section into that file, save it and run.

pip install torch transformers accelerate peft trl datasets huggingface_hub

Loading the Model

python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
from huggingface_hub import login

# login("your_hf_token_here")
# Create a token from hf hub and paste here, its ***free***. Uncomment the above line after pasting the token.

# Load tokenizer and base model
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B-Instruct")
base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-1B-Instruct",
    torch_dtype=torch.float16
)

# Load the ViLA adapter
model = PeftModel.from_pretrained(base_model, "hungryfoxz/Llama-3.2-1B-ViLA")
model.eval()

Running Inference

python

def ask_question(question: str) -> str:
    messages = [{"role": "user", "content": question}]
    
    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize=True,
        add_generation_prompt=True,
        return_tensors="pt",
        return_dict=True
    )
    
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=200,
            temperature=0.7,
            do_sample=True
        )
    
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

# Example
print(ask_question("Who are you?"))

Example Output

bash

**User**: Who are you?
**Assistant**: I am a Virtual Library Assistant.

📊 Evaluation

The model was evaluated on the test split of the LibraryFAQ1000 dataset. Training metrics:

Step	Training Loss	Validation Loss	Mean Token Accuracy
100	1.645	1.505	66.8%
200	1.111	1.340	69.0%
300	0.791	1.395	68.8%

🖥️ Hardware Requirements

Precision	Disk Size	RAM (Inference)
BF16	~2 GB	~2-3 GB
4-bit (Q4)	~800 MB	~1 GB

The model runs comfortably on:

Apple Silicon (M1/M2/M3) via MPS/ Intel i5 11th Gen / Ryzen 5 5000 series
Mid-range laptops
Edge devices (Raspberry Pi 5)
macOs/ Windows / Linux or Unix

⚠️ Limitations

Domain-Specific – Optimized for library FAQs; performance on out-of-domain questions may degrade Small Model – As a 1B parameter model, it may lack the depth of larger models for complex reasoning Hallucination – Like all LLMs, may occasionally generate incorrect or fabricated information

🤝 Acknowledgements

Meta for releasing the Llama 3.2 family of models
Hugging Face for the Transformers, PEFT, and TRL libraries
Dataset Contributors for the LibraryFAQ1000 dataset

Happy building! 📖✨

Facing any issues‼️ while running the code, feel free to send a 📸screenshot to 'kaustavkhanikar99@gmail.com'. I will try to get back to you as soon as possible ✌️.