DeciLM-7B is a 7.04 billion parameter decoder-only text generation model, released under the Apache 2.0 license. At the time of release, DeciLM-7B is the top-performing 7B base language model on the Open LLM Leaderboard. With support for an 8K-token sequence length, this highly efficient model uses variable Grouped-Query Attention (GQA) to achieve a superior balance between accuracy and computational efficiency. The model's architecture was generated using Deci's proprietary Neural Architecture Search technology, AutoNAC.
Deci developed and released the DeciLM-7B language model, a pre-trained, high-efficiency text generation model with 7 billion parameters. DeciLM-7B is not only the most accurate 7B base model, but it also outpaces all models in its class with a throughput that is up to 4.4x that of Mistral-7B's. An instruct version
DeciLM-7B-instruct has also been released.
*AutoNAC was employed to optimize the selection of the GQA num_key_value_heads for each layer.
The model is intended for commercial and research use in English and can be fine-tuned for various tasks and languages.
Use the code below to get started with the model.
1import torch
2from transformers import AutoModelForCausalLM, AutoTokenizer
3
4model_name = "Deci/DeciLM-7B"
5device = "cuda" # for GPU usage or "cpu" for CPU usage
6
7tokenizer = AutoTokenizer.from_pretrained(model_name)
8model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", trust_remote_code=True).to(device)
9
10inputs = tokenizer.encode("In a shocking finding, scientists discovered a herd of unicorns living in", return_tensors="pt").to(device)
11outputs = model.generate(inputs, max_new_tokens=100, do_sample=True, top_p=0.95)
12print(tokenizer.decode(outputs[0]))
13
14# The model can also be used via the text-generation pipeline interface
15from transformers import pipeline
16generator = pipeline("text-generation", "Deci/DeciLM-7B", torch_dtype="auto", trust_remote_code=True, device=device)
17outputs = generator("In a shocking finding, scientists discovered a herd of unicorns living in", max_new_tokens=100, do_sample=True, top_p=0.95)
18print(outputs[0]["generated_text"])
Below are DeciLM-7B and DeciLM-7B-instruct's Open LLM Leaderboard results.
DeciLM-7B is a new technology that comes with inherent risks associated with its use. The testing conducted so far has been primarily in English and does not encompass all possible scenarios. Like those of all large language models, DeciLM-7B's outputs are unpredictable, and the model may generate responses that are inaccurate, biased, or otherwise objectionable. Consequently, developers planning to use DeciLM-7B should undertake thorough safety testing and tuning designed explicitly for their intended applications of the model before deployment.
Please cite this model using this format.
1@misc{DeciFoundationModels,
2title = {DeciLM-7B},
3author = {DeciAI Research Team},
4year = {2023}
5url={https://huggingface.co/Deci/DeciLM-7B},
6}
