benk04/

CausalLM-RP-34B-4.65bpw-h6-exl2

[!IMPORTANT] Fits in 24GB VRAM with 32k+ context. Make sure to enable 4-bit cache option or you'll run into OOM errors.

Original Card

Description

Model used

• CausalLM/34b-beta

Prompt template ChatML

<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
{output}<|im_end|>