You can deploy this model and start earning money today!

pszemraj/
rnj-1.5-instruct-GGUF

Name: rnj-1.5-instruct-GGUF
Availability: InStock

gguf

llama-cpp

pszemraj/rnj-1.5-instruct

Views

No views yet

rnj-1.5-instruct-GGUF

GGUF conversions of pszemraj/rnj-1.5-instruct, a sliding-window-patched repack of EssentialAI/rnj-1.5-instruct. See the upstream card for architecture, training, and benchmarks.

Files

Quant	File
Q6_K	`rnj-1.5-instruct-q6_k.gguf`
Q4_K_M	`rnj-1.5-instruct-q4_k_m.gguf`

[!IMPORTANT] The base repo substitutes sliding-window attention for the original block-local attention. Equivalent to upstream for prompts 8192 tokens; diverges beyond and YMMV past 8192

Usage

bash

llama-cli   -hf pszemraj/rnj-1.5-instruct-GGUF:Q6_K -p "how to cook minute rice in 58 seconds?"
llama-server -hf pszemraj/rnj-1.5-instruct-GGUF:Q6_K -c 8192

Swap :Q6_K for :Q4_K_M for the smaller quant. Proof it works: