You can deploy this model and start earning money today!

johannhartmann/
SauerkrautLM-Multi-ColBERT-33m-GGUF

Name: SauerkrautLM-Multi-ColBERT-33m-GGUF
Availability: InStock

sentence-transformers

Views

No views yet

SauerkrautLM-Multi-ColBERT-33m-GGUF (ColBERT GGUF)

This repository contains GGUF format weights for the ColBERT retrieval model VAGOsolutions/SauerkrautLM-Multi-ColBERT-33m, converted specifically for our GGML-based ColBERT C++ runtime.

This GGUF file is NOT a standard llama.cpp decoder model. It uses the pg_colbert_v1 layout containing backbone transformer weights, tokenizer metadata, ColBERT dense projection layers, and similarity metrics.

Model Summary

Source Model: VAGOsolutions/SauerkrautLM-Multi-ColBERT-33m
Format: GGUF (pg_colbert_v1 schema)
Precision: F16
Encoder Backbone: ModernBERT / BERT
ColBERT Projection Dimension: 128
Profile Schema: pg_colbert_profile_v1
Target Runtime: both

Parity & Verification Report

Strict PyLate Token-Plan Parity: PASSED
Vector Parity Checked: YES
Strict Vector Parity Status: PASSED
Vector Parity Claim: PASSED (verified numerical equivalence with reference PyLate embeddings)

Usage with ColBERT GGML C++ Runtime

This GGUF model is designed to be loaded directly by our custom C++ ColBERT runtime (which uses GGML for the ModernBERT/BERT backbone forward pass and executes the ColBERT late-interaction dense projection).

Loading the Model in C++

cpp

// Initialize the ColBERT GGML model context
colbert_model model = colbert_model_load("SauerkrautLM-Multi-ColBERT-33m.f16.gguf");

// Tokenize and encode queries into late-interaction token embeddings
std::vector<float> query_embeddings = colbert_encode_query(model, "Which planet is known as the Red Planet?");

// Tokenize and encode documents into late-interaction token embeddings
std::vector<float> doc_embeddings = colbert_encode_doc(model, "Mars is the Red Planet.");

GGUF Conversion Info

Generated using the convert_colbert_hf_to_gguf.py utility.

CLI Command: python tools/convert_colbert_hf_to_gguf.py --model-id VAGOsolutions/SauerkrautLM-Multi-ColBERT-33m --outfile VAGOsolutions_SauerkrautLM_Multi_ColBERT_33m.f16.gguf --outtype f16 --target-runtime both
Converter Version: 1.0.0

johannhartmann/SauerkrautLM-Multi-ColBERT-33m-GGUF