This version of deepseek-coder-7b-instruct-v1.5 has been converted to run on the RK3588 NPU using ['w8a8', 'w8a8_g128', 'w8a8_g256', 'w8a8_g512'] quantization.
This model has been optimized with the following LoRA:
Pretty much anything by these folks:
marty1885 and
happyme531Deepseek-Coder-7B-Instruct-v1.5 is continue pre-trained from Deepseek-LLM 7B on 2T tokens by employing a window size of 4K and next token prediction objective, and then fine-tuned on 2B tokens of instruction data.
Here give some examples of how to use our model.
1from transformers import AutoTokenizer, AutoModelForCausalLM
2tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/deepseek-coder-7b-instruct-v1.5", trust_remote_code=True)
3model = AutoModelForCausalLM.from_pretrained("deepseek-ai/deepseek-coder-7b-instruct-v1.5", trust_remote_code=True).cuda()
4messages=[
5 { 'role': 'user', 'content': "write a quick sort algorithm in python."}
6]
7inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
8
9outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, top_k=50, top_p=0.95, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
10print(tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True))
This code repository is licensed under the MIT License. The use of DeepSeek Coder models is subject to the Model License. DeepSeek Coder supports commercial use.
If you have any questions, please raise an issue or contact us at
service@deepseek.com.
