Views
No views yet
care-chinese-mistral-7b is based on mistralai/Mistral-7B-Instruct-v0.3 and further fine-tuned on our CARE, enhancing cultural awareness especially in Chinese.1>>> from vllm import LLM, SamplingParams
2>>> from transformers import AutoTokenizer
3>>> import torch
4
5>>> model = LLM(model="geyang627/care-chinese-mistral-7b", tensor_parallel_size=torch.cuda.device_count(), dtype="auto", trust_remote_code=True, max_model_len=2048)
6
7>>> tokenizer = AutoTokenizer.from_pretrained("geyang627/care-chinese-mistral-7b", use_fast=False, trust_remote_code=True)
8>>> if tokenizer.pad_token is None:
9>>> tokenizer.pad_token = tokenizer.eos_token
10>>> tokenizer.pad_token_id = tokenizer.eos_token_id
11
12>>> sampling_params = SamplingParams(temperature=0.7, top_p=1.0, max_tokens=256)
13>>> outputs = model.generate(["为什么中国人不喜欢数字4？"], sampling_params)
14>>> print(outputs[0].outputs[0].text)