You can deploy this model and start earning money today!

ogflash/
merged-mistral-4bit-bitnetQnA

Name: merged-mistral-4bit-bitnetQnA
Availability: InStock

Views

No views yet

Mistral-7B BitNet LoRA — 4-Bit Merged

This repository contains a 4-bit quantized and LoRA-merged version of mistralai/Mistral-7B-Instruct-v0.2, fine-tuned on a small Q&A dataset related to 1-bit LLMs and BitNet B1.58.

The LoRA adapter was merged with the base model for easier deployment on constrained hardware or Hugging Face Spaces.

Model Details

Base model: mistralai/Mistral-7B-Instruct-v0.2
Quantization: 4-bit (NF4 via bitsandbytes)
Fine-tuning: LoRA (merged into base)
Adapter repo: ogflash/mistral-lora-qa-1bit

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
import torch

model_id = "ogflash/mistral-merged-1bit-4bit"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16
)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", quantization_config=bnb_config)

inputs = tokenizer("### Instruction:\nWhat is BitNet B1.58?\n\n### Response:\n", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=300)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

ogflash/merged-mistral-4bit-bitnetQnA

Mistral-7B BitNet LoRA — 4-Bit Merged

Model Details

Usage

ogflash/
merged-mistral-4bit-bitnetQnA