You can deploy this model and start earning money today!

NorthernTribe-Research/
UMSR-Reasoner-7B-INT8

Name: UMSR-Reasoner-7B-INT8
Availability: InStock

NorthernTribe-Research/UMSR-Reasoner-7B

finetune

endpoints_compatible

Views

No views yet

UMSR-Reasoner-7B (INT8)

8-bit bitsandbytes runtime quantization profile.

Model Tree

Source model: https://huggingface.co/NorthernTribe-Research/UMSR-Reasoner-7B
Variant repo: https://huggingface.co/NorthernTribe-Research/UMSR-Reasoner-7B-INT8
Dataset: https://huggingface.co/datasets/NorthernTribe-Research/UMSR-v1

Quantization Profile

Variant: int8
Scheme: bitsandbytes int8
Materialized weights in this repo: False

Usage

This variant repository is a profile card. Load weights from the source model id.

python

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

model_id = "NorthernTribe-Research/UMSR-Reasoner-7B"
bnb_config = BitsAndBytesConfig(load_in_8bit=True)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    quantization_config=bnb_config,
)

Output Contract

<final_answer>...</final_answer>

NorthernTribe-Research/UMSR-Reasoner-7B-INT8