Views
No views yet
| Base Model | Chat Model | 4bits Quantized Chat Model | |
|---|---|---|---|
| 7B-2k | BlueLM-7B-Base | BlueLM-7B-Chat | BlueLM-7B-Chat-4bits |
| 7B-32K | BlueLM-7B-Base-32K | BlueLM-7B-Chat-32K | BlueLM-7B-Chat-32K-AWQ / BlueLM-7B-Chat-32K-GPTQ |
| Model | Average | Summary | Single-Doc QA | Multi-Doc QA | Code | Few-shot | Synthetic |
|---|---|---|---|---|---|---|---|
| BlueLM-7B-Chat-32K | 41.2 | 18.8 | 35.6 | 36.2 | 54.2 | 56.9 | 45.5 |
1>>> from transformers import AutoModelForCausalLM, AutoTokenizer
2>>> tokenizer = AutoTokenizer.from_pretrained("vivo-ai/BlueLM-7B-Chat-32K", trust_remote_code=True, use_fast=False)
3>>> model = AutoModelForCausalLM.from_pretrained("vivo-ai/BlueLM-7B-Chat-32K", device_map="cuda:0", torch_dtype=torch.bfloat16, trust_remote_code=True)
4>>> model = model.eval()
5>>> inputs = tokenizer("[|Human|]:三国演义的作者是谁？[|AI|]:", return_tensors="pt")
6>>> inputs = inputs.to("cuda:0")
7>>> pred = model.generate(**inputs, max_new_tokens=64, repetition_penalty=1.1)
8>>> print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))
9三国演义的作者是谁？ 《三国演义》的作者是明代小说家罗贯中。