Views
No views yet
OctoGeeX is an instruction tuned model with 6B parameters created by fine-tuning CodeGeeX2 on CommitPackFT & OASST as described in the OctoPack paper.
| Data | CommitPack | 4TB of GitHub commits across 350 programming languages |
|---|---|---|
| CommitPackFT | Filtered version of CommitPack for high-quality commit messages that resemble instructions | |
| Model | OctoCoder | StarCoder (16B parameters) instruction tuned on CommitPackFT + OASST |
| OctoGeeX | CodeGeeX2 (6B parameters) instruction tuned on CommitPackFT + OASST | |
| Evaluation | HumanEvalPack | Extension of OpenAI's HumanEval to cover 3 scenarios across 6 languages |
1# pip install -q transformers
2from transformers import AutoModelForCausalLM, AutoTokenizer
3
4checkpoint = "bigcode/octogeex"
5device = "cuda" # for GPU usage or "cpu" for CPU usage
6
7tokenizer = AutoTokenizer.from_pretrained(checkpoint)
8model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)
9
10inputs = tokenizer.encode("Question: Please write a function in Python that performs bubble sort.\n\nAnswer:", return_tensors="pt").to(device)
11outputs = model.generate(inputs)
12print(tokenizer.decode(outputs[0]))1@article{muennighoff2023octopack,
2 title={OctoPack: Instruction Tuning Code Large Language Models},
3 author={Niklas Muennighoff and Qian Liu and Armel Zebaze and Qinkai Zheng and Binyuan Hui and Terry Yue Zhuo and Swayam Singh and Xiangru Tang and Leandro von Werra and Shayne Longpre},
4 journal={arXiv preprint arXiv:2308.07124},
5 year={2023}
6}