from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-1.5B",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Load LoRA adapters
model = PeftModel.from_pretrained(base_model, "adity12345/qwen2.5-1.5b-medical-pretrained")
tokenizer = AutoTokenizer.from_pretrained("adity12345/qwen2.5-1.5b-medical-pretrained")

# Generate text
prompt = "Your prompt here"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Loading with 4-bit Quantization (Low Memory)

python

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-1.5B",
    quantization_config=bnb_config,
    device_map="auto"
)

model = PeftModel.from_pretrained(base_model, "adity12345/qwen2.5-1.5b-medical-pretrained")

📈 Performance

VRAM Usage: ~1.5GB (4-bit) to ~3GB (16-bit)
Inference Speed: ~50-100 tokens/second (T4 GPU)
Context Length: 32K tokens (inherited from base model)

⚖️ Limitations

Model may inherit biases from base model and training data
Performance depends on domain similarity between training and inference
Best results within the domain of the training corpus

📝 Citation

bibtex

@misc{qwen2.5-1.5b-medical-pretrained},
  author = {Your Name},
  title = {adity12345/qwen2.5-1.5b-medical-pretrained},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/adity12345/qwen2.5-1.5b-medical-pretrained}}
}

🙏 Acknowledgments

Base model: Qwen Team
Training framework: Hugging Face
LoRA: Microsoft Research
QLoRA: University of Washington

📄 License

This model inherits the Apache 2.0 license from the base Qwen2.5 model.