Filename	Quantization	Size	Recommended Use Case
`french-bge-m3-f16.gguf`	F16	~700 MB	Full precision baseline
`french-bge-m3-q8_0.gguf`	Q8_0	~360 MB	Recommended (99.9% quality, small footprint)
`french-bge-m3-q5_k_m.gguf`	Q5_K_M	~260 MB	Medium quantization
`french-bge-m3-q4_k_m.gguf`	Q4_K_M	~220 MB	Ultra lightweight

Quick Start with llama.cpp

Local file execution:

llama-server -m french-bge-m3-q8_0.gguf --embedding --port 8080 -ngl 99

Direct loading via Hugging Face (`-hf`):

llama-server -hf "fabcc/french-bge-m3-GGUF:french-bge-m3-q8_0.gguf" --embedding --port 8080 -ngl 99

Test with OpenAI API Client (curl)

bash

curl http://127.0.0.1:8080/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Ceci est un document de test en français.",
    "model": "french-bge-m3"
  }'

Credits & Attributions

French Pruning & Optimization: Antoine Louis (antoinelouis/french-bge-m3)
Original Model & Architecture: BAAI (BAAI/bge-m3)
GGUF Conversion & Quantization: fabcc

Fabcc/french-bge-m3-gguf

French BGE-M3 - GGUF

Overview

Files Included

Quick Start with llama.cpp

Local file execution:

Direct loading via Hugging Face (-hf):

Test with OpenAI API Client (curl)

Credits & Attributions

Fabcc/
french-bge-m3-gguf

Direct loading via Hugging Face (`-hf`):