LoRA adapter for unsloth/Muse-Glimmer-30B, a 30B parameter vision-language model. Fine-tuned using parameter-efficient LoRA on Kaggle with 2x T4 GPUs.

Model Architecture:

Base: Muse Glimmer 30B (Vision-Language)
Training Method: LoRA (Low-Rank Adaptation)
Precision: FP16 (adapters)
Base Model Loading: 4-bit quantization

Training Details

Configuration	Value
Hardware	2x NVIDIA T4 (16GB each)
Platform	Kaggle
Framework	Unsloth + PEFT
LoRA Rank (r)	16
LoRA Alpha	16
LoRA Dropout	0.0
Target Modules	Attention, MLP
Vision Layers	Frozen
Language Layers	Trainable
Effective Batch Size	16
Learning Rate	2e-4
Optimizer	AdamW 8-bit
Warmup Steps	10
Weight Decay	0.01
Max Sequence Length	2048
Quantization	4-bit (for loading)
Adapter Size	0.66 GB

Usage

Installation

pip install transformers accelerate peft torch bitsandbytes

Load Model

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model with 4-bit quantization
base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/Muse-Glimmer-30B",
    torch_dtype=torch.float16,
    load_in_4bit=True,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(
    base_model,
    "aliRafik/Muse-Glimmer-30B-finetuned-mlabonne"
)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(
    "aliRafik/Muse-Glimmer-30B-finetuned-mlabonne"
)

Inference

python

def generate_response(prompt, max_length=256):
    inputs = tokenizer(prompt, return_tensors="pt")

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_length,
            temperature=0.7,
            do_sample=True,
            top_p=0.9,
            repetition_penalty=1.1,
            pad_token_id=tokenizer.eos_token_id
        )

    return tokenizer.decode(
        outputs[0],
        skip_special_tokens=True
    )


# Example
response = generate_response("Your prompt here")
print(response)

Memory Requirements

Precision	GPU Memory Required
4-bit (Recommended)	~12 GB
8-bit	~18 GB
16-bit (FP16)	~35 GB

Download Statistics

Adapter: 0.66 GB
Base Model: 34.53 GB (FP16)

Disclaimer

This is a LoRA adapter, not a standalone model. Must be loaded with the base Muse Glimmer 30B model.

Citation

bibtex

@misc{muse-glimmer-30b-lora-2024,
  author = {Ali Rafik},
  title = {Muse Glimmer 30B - LoRA Fine-tuned Adapter},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/aliRafik/Muse-Glimmer-30B-finetuned-mlabonne}}
}

aliRafik/Muse_Glimmer_30B_finetuned_mlabonne_16bit

Uploaded model

Muse Glimmer 30B - LoRA Fine-tuned

Model Description