Views
No views yet
1llama-completion \
2 -hf HuggingFaceBio/Carbon-3B-GGUF:Q8_0 \
3 --prompt "<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG" \
4 -n 128 --temp 0 --no-display-promptllama-cli instead, pass -no-cnv so it does not wrap the prompt
in assistant-style markers (otherwise the model may emit annotation tags such
as <protein_coding_region> instead of DNA).| Quant | File | Size |
|---|---|---|
| BF16 | carbon-3b-bf16.gguf | 6.4 GB |
| Q8_0 | carbon-3b-q8_0.gguf | 3.4 GB |
| Q6_K | carbon-3b-q6_k.gguf | 2.8 GB |
| Q5_K_M | carbon-3b-q5_k_m.gguf | 2.4 GB |
| Q4_K_M | carbon-3b-q4_k_m.gguf | 2.1 GB |
--token-embedding-type Q8_0 --output-tensor-type Q8_0
so the sensitive embedding/output tensors stay at Q8_0 precision (important for
this model's tight DNA-vs-text logit margins).master after that works:1git clone https://github.com/ggml-org/llama.cpp
2cd llama.cpp && cmake -B build && cmake --build build -jhf download HuggingFaceBio/Carbon-3B-GGUF carbon-3b-bf16.gguf --local-dir .1./build/bin/llama-completion -m carbon-3b-bf16.gguf \
2 -p '<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG' \
3 -n 64 --temp 0 -no-cnv1./build/bin/llama-completion -m carbon-3b-bf16.gguf \
2 -p '<vertebrate_mammalian><protein_coding_region><dna>ATGCGCTAG' \
3 -n 64 --temp 0 -no-cnvhf download HuggingFaceBio/Carbon-500M-GGUF carbon-500m-bf16.gguf --local-dir .--model-draft:1./build/bin/llama-speculative \
2 -m carbon-3b-bf16.gguf \
3 -md carbon-500m-bf16.gguf \
4 -p '<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG' \
5 -n 256 --temp 0llama-server accepts the same -md flag):1./build/bin/llama-server \
2 -m carbon-3b-bf16.gguf \
3 -md carbon-500m-bf16.gguf \
4 --draft-max 16 --draft-min 1 \
5 --port 80801curl -s http://localhost:8080/completion -d '{
2 "prompt": "<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG",
3 "n_predict": 256,
4 "temperature": 0
5}' | jq -r .contentscore() function computes mean log-prob per DNA token. In llama.cpp the closest tools are llama-perplexity for corpus-level perplexity (perplexity = exp(-mean_logprob)):1# one prompt per line in dna_corpus.txt, each wrapped in <dna>...</dna>
2./build/bin/llama-perplexity -m carbon-3b-bf16.gguf -f dna_corpus.txt --ppl-stride 0llama-server with logprobs for per-token log-probabilities:1./build/bin/llama-server -m carbon-3b-bf16.gguf --port 8080 &
2curl -s http://localhost:8080/completion -d '{
3 "prompt": "<dna>GGGCTATAAAGGCCATCGATCGATCGATCGATCGATCGATCG</dna>",
4 "n_predict": 0,
5 "n_probs": 1
6}' | jq '.completion_probabilities'rope_scaling = {type: yarn, factor: 4.0, original_max_position_embeddings: 32768}:1./build/bin/llama-completion -m carbon-3b-bf16.gguf \
2 -c 65536 --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 32768 \
3 -p '<dna>...' -n 64 --temp 0 -no-cnvrevision="fns" example from the source card needs custom modeling code (factorized nucleotide supervision head), which only the Python transformers path can load. llama.cpp can't run that branch.HybridDNATokenizer (loaded with trust_remote_code=True) on the standard <dna>/metadata/edge-case fixtures shipped in ggml-org/vocabs.