You can deploy this model and start earning money today!

jablonkagroup/
chempile-llama-paper-100m-checkpoint-450

Qwen/Qwen2.5-7B-Instruct

adapter

Views

No views yet

LoRA Adapter - Checkpoint 450

LoRA adapter fine-tuned from Qwen/Qwen2.5-7B-Instruct (checkpoint 450)

Quick Start with Quantization

pip install torch transformers peft accelerate bitsandbytes

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel, prepare_model_for_kbit_training
from accelerate import PartialState

# Configure 4-bit quantization for memory efficiency
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=False,
)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct", use_fast=False)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

# Load base model with quantization
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct",
    quantization_config=bnb_config,
    device_map={"": PartialState().process_index},
    torch_dtype=torch.float16,
)

# Prepare for LoRA
base_model = prepare_model_for_kbit_training(base_model)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "YOUR_USERNAME/YOUR_REPO_NAME")

print("✅ Model loaded with 4-bit quantization!")

Interactive Chat