The goal is practical Transformers usage with lower memory requirements. It is suited for local experiments, notebooks, adapter workflows, and generation tests on high-memory NVIDIA hardware.

Format

Quantization: bitsandbytes 4-bit
Quant type: NF4
Compute dtype: bfloat16
Nested quantization: enabled
Base model: deepreinforce-ai/Ornith-1.0-35B

Usage

python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "almernzh/Ornith-1.0-35B-BNB-NF4"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {
        "role": "user",
        "content": "Give a concise plan to debug a failing Python unit test.",
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=200,
        temperature=0.6,
        top_p=0.95,
    )

print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Reasoning Behavior

Ornith-1.0-35B may emit a <think>...</think> block before the final answer. This follows the behavior of the base model.

Notes

This model targets Transformers and bitsandbytes. Runtime support may vary across serving engines. For vLLM, SGLang, or TensorRT-LLM, use a format that the target runtime explicitly supports.