Views
No views yet
1python -m vllm.entrypoints.openai.api_server \
2 --model tellang/yeji-8b-rslora-v7-AWQ \
3 --quantization awq \
4 --max-model-len 40961from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained(
4 "tellang/yeji-8b-rslora-v7-AWQ",
5 torch_dtype="auto",
6 device_map="auto"
7)
8tokenizer = AutoTokenizer.from_pretrained("tellang/yeji-8b-rslora-v7-AWQ")| Model | Params | Type | Downloads |
|---|---|---|---|
| yeji-8b-rslora-v7 | 8.2B | Full | 345 |
| yeji-8b-rslora-v7-AWQ | ~2.5B | AWQ 4-bit | 371 |
| yeji-4b-instruct-v9 | 4.0B | Full | 65 |
| yeji-4b-instruct-v9-AWQ | ~1.3B | AWQ 4-bit | 138 |