You can deploy this model and start earning money today!

doupari/
llama3.1_8b_sft-solo-attn-v2-k24

Name: llama3.1_8b_sft-solo-attn-v2-k24
Availability: InStock

pytorch

llama

Views

No views yet

LLOPA Model

This repo bundles LLOPA/TRI inference with minimal friction.

python

from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("your-repo")
model = AutoModelForCausalLM.from_pretrained("your-repo", trust_remote_code=True)

out = model.llopa_generate(
    tokenizer=tok,
    system="You are a helpful assistant.",
    document="...",
    question="...",
    K=4,
    prefill_mode="lower",
    prefill_attn="causal",
)
print(out)

doupari/llama3.1_8b_sft-solo-attn-v2-k24

LLOPA Model

doupari/
llama3.1_8b_sft-solo-attn-v2-k24