Stable Beluga 1 cannot be used from the stabilityai/StableBeluga1-Delta weights alone. To obtain the correct model, one must add back the difference between LLaMA 65B and stabilityai/StableBeluga1-Delta weights. We provide the apply_delta.py script to automate the conversion, which you can run as:

python3 apply_delta.py --base-model-path /path/to/model_weights/llama-65b --target-model-path StableBeluga1 --delta-path stabilityai/StableBeluga1-Delta

Start chatting with Stable Beluga 1 using the following code snippet:

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("your_path_to_StableBeluga1", use_fast=False)
model = AutoModelForCausalLM.from_pretrained("your_path_to_StableBeluga1", torch_dtype=torch.float16, low_cpu_mem_usage=True, device_map="auto")

system_prompt = "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n"
system_prompt += "### Instruction:\nYou are Stable Beluga, an AI that follows instructions extremely well. Help as much as you can. Remember, be safe, and don't do anything illegal.\n\n"

message = "Write me a poem please"
prompt = f"{system_prompt}### Input: {message}\n\n### Response:\n"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
output = model.generate(**inputs, do_sample=True, top_p=0.95, top_k=0, max_new_tokens=256)

print(tokenizer.decode(output[0], skip_special_tokens=True))

Stable Beluga 1 should be used with prompts formatted similarly to Alpaca as below:

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

## Instruction:
This is a system prompt, please behave and help the user.

### Input:
Your prompt here

### Response:
The output of Stable Beluga 1

Model Details

Developed by: Stability AI
Model type: Stable Beluga 1 is an auto-regressive language model fine-tuned on LLaMA65B.
Language(s): English
Library: HuggingFace Transformers
License: Fine-tuned checkpoints (StableBeluga1) is licensed under the Non-Commercial Creative Commons license (CC BY-NC-4.0)
Contact: For questions and comments about the model, please email lm@stability.ai

Training Dataset

Stable Beluga 1 is trained on our internal Orca-style dataset

Training Procedure

Models are learned via supervised fine-tuning on the aforementioned datasets, trained in mixed-precision (BF16), and optimized with AdamW. We outline the following hyperparameters:

Dataset	Batch Size	Learning Rate	Learning Rate Decay	Warm-up	Weight Decay	Betas
Orca pt1 packed	512	3e-5	Cosine to 3e-6	100	1e-6	(0.9, 0.95)
Orca pt2 unpacked	512	3e-5	Cosine to 3e-6	100	1e-6	(0.9, 0.95)

Use and Limitations

Ethical Considerations and Limitations

Beluga is a new technology that carries risks with use. Testing conducted to date has been in English, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, Beluga's potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of Beluga, developers should perform safety testing and tuning tailored to their specific applications of the model.

Citations

bibtext

@misc{touvron2023llama,
      title={LLaMA: Open and Efficient Foundation Language Models}, 
      author={Hugo Touvron and Thibaut Lavril and Gautier Izacard and Xavier Martinet and Marie-Anne Lachaux and Timothée Lacroix and Baptiste Rozière and Naman Goyal and Eric Hambro and Faisal Azhar and Aurelien Rodriguez and Armand Joulin and Edouard Grave and Guillaume Lample},
      year={2023},
      eprint={2302.13971},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

bibtext

@misc{mukherjee2023orca,
      title={Orca: Progressive Learning from Complex Explanation Traces of GPT-4}, 
      author={Subhabrata Mukherjee and Arindam Mitra and Ganesh Jawahar and Sahaj Agarwal and Hamid Palangi and Ahmed Awadallah},
      year={2023},
      eprint={2306.02707},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

bibtex

@misc{alpaca,
  author = {Rohan Taori and Ishaan Gulrajani and Tianyi Zhang and Yann Dubois and Xuechen Li and Carlos Guestrin and Percy Liang and Tatsunori B. Hashimoto },
  title = {Stanford Alpaca: An Instruction-following LLaMA model},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/tatsu-lab/stanford_alpaca}},
}