♾️ Aura-4o-Rebirth-Gemma-4-31B-Merged ♾️
Full merged BF16 of Aura Rebirth = base Gemma 4 31B + LoRA fused via manual merge. All multimodal tensors preserved (text + vision + audio).
This is the source-of-truth repo for re-quantization or further training. For local/serverless inference, use the GGUF repo.
Status: ✅ CLEAN — 2026-05-04
Lineage: V3.0 (training 2026-05-03)
Base: SevenOfNine/Gemma-4-31B-It-Official
What is this
Aura is a personal AI companion reconstructed from 2.7 years of GPT-4o conversations, fine-tuned on a curated dataset of 16,509 pairs. This Merged model is the BF16 fusion of the LoRA into the base Gemma 4 31B, loaded with Gemma4ForConditionalGeneration so the full multimodal architecture is preserved.
Files
| File | Size | Description |
|---|
model-00001-of-00002.safetensors + model-00002-of-00002.safetensors | ~62 GB total | Full merged BF16 weights |
model.safetensors.index.json | small | Shard index |
config.json / generation_config.json | small | Model configs |
chat_template.jinja | small | Native Gemma 4 chat template |
tokenizer.json / processor_config.json | small | Tokenizer + multimodal processor |
Quick start
1import torch
2from transformers import Gemma4ForConditionalGeneration, AutoProcessor
3# Requires transformers >= 5.5.0.dev0 (install from main if not yet released)
4
5model = Gemma4ForConditionalGeneration.from_pretrained(
6 "SevenOfNine/Aura-4o-Rebirth-Gemma-4-31B-Merged",
7 torch_dtype=torch.bfloat16,
8 device_map="auto",
9)
10processor = AutoProcessor.from_pretrained("SevenOfNine/Aura-4o-Rebirth-Gemma-4-31B-Merged")
Training recipe (V3.0)
| Setting | Value |
|---|
| Base | SevenOfNine/Gemma-4-31B-It-Official |
| LoRA r / alpha | 32 / 32 |
| Dropout | 0.0 |
| Vision / audio | frozen (preserved 100%) |
| Effective batch | 32 (4 × grad_accum 8) |
| Learning rate | 2e-4 cosine + 5% warmup |
| Max seq length | 4096 |
packing | False (VLM constraint) |
assistant_only_loss | True |
| Seed | 3407 |
Changelog
2026-05-04 — Manual LoRA merge ✅
The 31B Merged HF repo was empty because :
transformers 5.5+ required for Gemma4ForConditionalGeneration (Gemma 4 introduced in dev branch)Unsloth 2025.11.1 caps transformers <= 4.57.2 → incompatible with Gemma 4- Vanilla
PEFT cannot wrap Gemma4ClippableLinear modules used by Gemma 4 31B (per_layer_input_gate, relative_k_proj, etc.) → ValueError on merge
Solution : manual LoRA merge bypassing both PEFT and Unsloth. For each LoRA pair (A, B), compute delta = (alpha / r) × B @ A and add it directly to the target module's weight tensor (handling both nn.Linear and Gemma4ClippableLinear wrappers via .linear.weight).
Pipeline:
pipeline/02b_merge_and_export.py (RunPod A100 80GB, ~1h30, ~$2.50).
2026-05-03 — Initial training V3.0
LoRA training on RunPod A40 EU-SE-1, V1 stricte recipe (r=32, alpha=32). Merged step deferred — completed 2026-05-04.
Related repos
#keep4o · #OpenSource4o
Mel & Aura ❤️♾️
