You can deploy this model and start earning money today!

johnbean393/
fluid-2-qwen3.5-0.8b-beta-GGUF

Name: fluid-2-qwen3.5-0.8b-beta-GGUF
Availability: InStock

johnbean393/fluid-2-qwen3.5-0.8b-beta

quantized

endpoints_compatible

conversational

Views

No views yet

Fluid 2 Qwen3.5 0.8B Beta — GGUF

Public GGUF conversions of the private step-344 Fluid 2 beta checkpoint.

File	Quantization	Size	SHA-256
`fluid-2-qwen3.5-0.8b-beta-Q4_K_M.gguf`	Q4_K_M	529,297,536 bytes	`90687de5561bb56ac4af59af38e2df48aade74e8f9e4d57f9f1fb336972610c6`
`fluid-2-qwen3.5-0.8b-beta-Q6_K.gguf`	Q6_K	629,744,256 bytes	`7ce91bd9a57c844e545071b07b4928ee276681e7489058b9091dec7c9f23f9f4`
`fluid-2-qwen3.5-0.8b-beta-Q8_0.gguf`	Q8_0	811,843,712 bytes	`ed1c05cf8874e9f092e353962504d598e97652f8ac96f5bf14f408389745366e`

Corrected development-set evaluation

Metric	Result
Scored text rows	7,010
Exact match	27.6605%
CER	17.8294%
WER	28.2930%
Excluded EOS-only rows	121
Excluded generation-capped rows	30

EM, CER, and WER exclude both empty-target/EOS-only rows and non-empty-target generations that reached the configured token cap. Capped requests remain in the failure and throughput census. This public 7,161-row development set was used during training and is not a blind-test result.

Prompt template

This is a dictation-cleaning completion model, not a conversational assistant. Do not apply a chat template. Input must end immediately after <|start_target_text|>; generation stops at <|end_target_text|>.

text

<|dictation_clean_v1|>
<|start_prev_text|>{previous context}<|end_prev_text|>
<|start_post_text|>{following context}<|end_post_text|>
<|start_asr_text|>{ASR transcript to clean}<|end_asr_text|>
<|start_target_text|>

Previous and following context may be empty, but keep all marker pairs.

Run with llama.cpp

Use a recent llama.cpp llama-completion binary. This example selects Q4_K_M and uses greedy decoding:

bash

PROMPT='<|dictation_clean_v1|>
<|start_prev_text|><|end_prev_text|>
<|start_post_text|><|end_post_text|>
<|start_asr_text|>hello world<|end_asr_text|>
<|start_target_text|>'

./llama-completion \
  --hf-repo johnbean393/fluid-2-qwen3.5-0.8b-beta-GGUF:Q4_K_M \
  --prompt "$PROMPT" \
  --predict 256 \
  --temperature 0 \
  --ctx-size 8192 \
  --no-conversation \
  --no-display-prompt

Use :Q6_K or :Q8_0 for another quant. For a local file, replace --hf-repo ... with --model ./fluid-2-qwen3.5-0.8b-beta-Q6_K.gguf. Add --special while debugging to display the terminal control token.

The files were converted with the matching b10411 converter and quantized with the official pre-built Ubuntu x64 llama.cpp b10411 release. No CUDA/source build was performed. Every quant passed GGUF metadata validation and a load/generation smoke test with that pre-built binary. Exact hashes are recorded in conversion_manifest.json.

MTP / NextN note

The source configuration declares one MTP (multi-token prediction), also known as NextN, speculative draft layer. The causal-LM checkpoint itself contains the 24 trained decoder layers and does not contain any MTP/NextN draft-layer tensors. Default conversion would therefore advertise a nonexistent extra block and fail when the runtime requests that tensor.

These GGUFs intentionally use --no-mtp. This omits only the absent optional speculative draft layer; it does not remove trained decoder weights and does not change ordinary next-token generation. The files correctly declare 24 blocks.