from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "openpql/nishka-sft",
    device_map="auto",
    torch_dtype="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained("openpql/nishka-sft")

# Generate
inputs = tokenizer("Your prompt here", return_tensors="pt")
outputs = model.generate(**inputs, max_length=512)
print(tokenizer.decode(outputs[0]))

Deployment

This model is ready for deployment with vLLM, TGI, or other inference servers.

bash

# vLLM example
vllm serve openpql/nishka-sft --dtype float16

openpql/nishka-sft-phi3-merged

NISHKA SFT

Model Details

Usage

Deployment

openpql/
nishka-sft-phi3-merged