You can deploy this model and start earning money today!

Akahsizrr/
fuse-1-Lite-GGUF

Name: fuse-1-Lite-GGUF
Rating: 2.5 (4291 reviews)

gguf

endpoints_compatible

conversational

Views

No views yet

Fuse3 llama.cpp Integration

Full llama.cpp support for Fuse3 (fuse-1 Lite) — LFM2 host + Qwen3.6 coding experts.

Files

File	Purpose
`fuse3.cpp`	C++ graph builder (place in `src/models/`)
`fuse3.h`	C++ header (place in `src/models/`)
`fuse3_converter.py`	Python GGUF converter (place in `conversion/`)
`export_gguf.py`	Standalone GGUF exporter (no llama.cpp fork needed to create GGUF)
`INTEGRATION.md`	Step-by-step patches for llama.cpp source files

Quick Start

1. Create the GGUF file (no fork needed)

bash

pip install gguf safetensors torch
python export_gguf.py --model-dir /path/to/fuse-1-Lite --output fuse-1-Lite-f16.gguf --outtype f16

2. Build llama.cpp with Fuse3 support

bash

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp

# Apply patches (see INTEGRATION.md)
# Copy fuse3.cpp and fuse3.h to src/models/
# Copy fuse3_converter.py to conversion/
# Modify src/llama-arch.h, src/llama-arch.cpp, src/llama-model.cpp, src/CMakeLists.txt

# Build
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release

3. Run inference

bash

./build/bin/llama-cli \
  -m fuse-1-Lite-f16.gguf \
  -p "Write a Python function to check if a number is prime." \
  -n 512 --temp 0.1

Architecture

Fuse3 extends llama.cpp's native LFM2 implementation:

Host layers: Reuses LFM2's attention (Lfm2AttentionDecoderLayer) and short-conv (Lfm2ShortConvDecoderLayer) graph builders
Dense FFN: Standard LFM2 SwiGLU FFN (always dense, not MoE)
Expert augmentation: After each augmented layer's FFN, a custom MoE block runs:
1. Router: sqrtsoftplus(x @ router_weight) → top-k selection → normalize
2. Experts: Per-expert SwiGLU FFN (gate, up, down projections)
3. Scale: softplus(expert_scale) clamped to 0.1
4. Add: host_output + scale * expert_output

GGUF Tensor Names

Standard LFM2 tensors (host)

token_embd.weight, token_embd_norm.weight, output.weight
blk.{i}.attn_norm.weight, blk.{i}.attn_q/k/v.weight, blk.{i}.attn_output.weight
blk.{i}.attn_q_norm.weight, blk.{i}.attn_k_norm.weight
blk.{i}.ffn_norm.weight, blk.{i}.ffn_gate/up/down.weight
blk.{i}.shortconv_conv/inproj/outproj.weight

Fuse3-specific tensors

blk.{i}.fuse3_router.weight — {n_embd, n_exp} router gate
blk.{i}.fuse3_expert_scale.weight — {1} learned scale parameter
blk.{i}.fuse3_experts.gate.weight — {n_embd, n_ff_exp, n_exp} stacked expert gates
blk.{i}.fuse3_experts.up.weight — {n_embd, n_ff_exp, n_exp} stacked expert up projections
blk.{i}.fuse3_experts.down.weight — {n_ff_exp, n_embd, n_exp} stacked expert down projections

Custom GGUF metadata keys

fuse3.augmented_layers — array of layer indices with experts
fuse3.expert_counts — per-layer expert count array
fuse3.swiglu_limit — float, expert activation clamp (default 10.0)
fuse3.expert_scale_init — float, initial scale (not used at inference)

Quantization

The GGUF file can be quantized using llama.cpp's standard quantization tools:

bash

# Q4_K_M (recommended)
./build/bin/llama-quantize fuse-1-Lite-f16.gguf fuse-1-Lite-q4_k_m.gguf q4_k_m

# Q8_0
./build/bin/llama-quantize fuse-1-Lite-f16.gguf fuse-1-Lite-q8_0.gguf q8_0

Note: Only the host LFM2 weights are quantized. The expert weights are kept in f16 for quality (they're small — 512 intermediate × ~30 experts × 30 layers ≈ 200MB).

Akahsizrr/fuse-1-Lite-GGUF

Fuse3 llama.cpp Integration

Files

Quick Start

1. Create the GGUF file (no fork needed)

2. Build llama.cpp with Fuse3 support

3. Run inference

Architecture

GGUF Tensor Names

Standard LFM2 tensors (host)

Fuse3-specific tensors

Custom GGUF metadata keys

Quantization

Akahsizrr/
fuse-1-Lite-GGUF