Views
No views yet
StableCode-Completion-Alpha-3B-4KStableCode-Completion-Alpha-3B-4K is a 3 billion parameter decoder-only code completion model pre-trained on diverse set of programming languages that topped the stackoverflow developer survey.StableCode-Completion-Alpha-3B-4k by using the following code snippet:1from transformers import AutoModelForCausalLM, AutoTokenizer
2tokenizer = AutoTokenizer.from_pretrained("stabilityai/stablecode-completion-alpha-3b-4k")
3model = AutoModelForCausalLM.from_pretrained(
4 "stabilityai/stablecode-completion-alpha-3b-4k",
5 trust_remote_code=True,
6 torch_dtype="auto",
7)
8model.cuda()
9inputs = tokenizer("import torch\nimport torch.nn as nn", return_tensors="pt").to("cuda")
10tokens = model.generate(
11 **inputs,
12 max_new_tokens=48,
13 temperature=0.2,
14 do_sample=True,
15)
16print(tokenizer.decode(tokens[0], skip_special_tokens=True))StableCode-Completion-Alpha-3B-4k models are auto-regressive language models based on the transformer decoder architecture.lm@stability.ai| Parameters | Hidden Size | Layers | Heads | Sequence Length |
|---|---|---|---|---|
| 2,796,431,360 | 2560 | 32 | 32 | 4096 |
StableCode-Completion-Alpha-3B-4k is pre-trained at a context length of 4096 for 300 billion tokens on the bigcode/starcoder-data.starcoder-data dataset.1@misc{StableCodeCompleteAlpha4K,
2 url={[https://huggingface.co/stabilityai/stablecode-complete-alpha-3b-4k](https://huggingface.co/stabilityai/stablecode-complete-alpha-3b-4k)},
3 title={Stable Code Complete Alpha},
4 author={Adithyan, Reshinth and Phung, Duy and Cooper, Nathan and Pinnaparaju, Nikhil and Laforte, Christian}
5}