Views
No views yet
1from unsloth import FastLanguageModel
2from peft import PeftModel
3from transformers import AutoTokenizer
4
5BASE = "unsloth/gpt-oss-20b"
6ADAPTER = "llmimplementation/gpt-oss-20b-sft-multilingual-reasoning-qlora-v1"
7
8base, tok = FastLanguageModel.from_pretrained(
9 BASE, load_in_4bit=True, max_seq_length=1024
10)
11try:
12 tok = AutoTokenizer.from_pretrained(ADAPTER, use_fast=True)
13except Exception:
14 pass
15
16model = PeftModel.from_pretrained(base, ADAPTER)
17FastLanguageModel.for_inference(model)
18
19prompt = "<|start|>user<|message|>List 3 creative uses for paper clips.<|end|>\n<|start|>assistant<|message|>"
20out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=200)
21print(tok.decode(out[0], skip_special_tokens=False))Prompt format (GPT-OSS)<|start|>user<|message|>{your text}<|end|> <|start|>assistant<|message|>
### Tip: include the README when pushing
- Easiest: create a local folder with your adapter files **and** `README.md`, then call `model.push_to_hub(repo_id)` from there (or use `huggingface_hub`’s `upload_file` if pushing after the fact).
- Don’t forget to push the tokenizer if you customized it:
```python
tokenizer.push_to_hub("llmimplementation/gpt-oss-20b-sft-multilingual-reasoning-qlora-v1")