Alignment Lab AI is pleased to introduce our latest research efforts with:
Buzz-8b-Large, a state-of-the-art language model developed in collaboration with Hive Digital Technologies.
The Buzz model, Dataset, and Code are to be released to build a toolkit that aims to demonstrate the potential for reuse and optimization of existing pretrained language models to continuously refine the heights of performance that can be achieved with optimal use of FlOps. Alongside Buzz-5b-Medium, we release
we release this, the Buzz dataset and over the next few days, two additional models: Buzz-3B-Small and Buzz-5B-Medium, the codebase to refine, filter and augment the data, as well as prune and train your own variants, will additionally be released in the coming days.
Iterative Fine-Tuning Methodology
Our research builds upon the concepts introduced in several key papers, including:
By combining high quality data, iterative fine-tuning with carefully selected "grounding" distributions from previous epochs, we have developed a cost-effective approach that pushes the boundaries of model reuse and optimization.
notably, we observe that the models have not yet appeared to plateu with the application of these techniques
image/png
Chat Template and Inference
To use the Buzz-8b-Medium model for chat-based tasks, you can utilize the provided chat template. Here's an example of how to perform inference using the Hugging Face Transformers library:
python
1from transformers import AutoTokenizer, AutoModelForCausalLM
23# Load the tokenizer and model4model_name ="H-D-T/Buzz-8b-Large-v0.5"5tokenizer = AutoTokenizer.from_pretrained(model_name)6model = AutoModelForCausalLM.from_pretrained(model_name)78# Set the device to run the model on (e.g., "cuda" for GPU, "cpu" for CPU)9device ="cuda"if torch.cuda.is_available()else"cpu"10model.to(device)1112# Define the input prompt13prompt ="Hello, how are you today?"1415# Tokenize the input prompt16input_ids = tokenizer.encode(prompt, return_tensors="pt").to(device)1718# Generate the model's response19output = model.generate(20 input_ids,21 max_length=100,22 num_return_sequences=1,23 no_repeat_ngram_size=2,24 early_stopping=True25)2627# Decode the generated response28response = tokenizer.decode(output[0], skip_special_tokens=True)2930print("Input:", prompt)31print("Response:", response)
NOTE: this model is a COMPLETIONS model, it will generate text by default, which completes the text you send it, it only has a start <|begin_of_text|> and a stop token <|end_of_text|>
if you want it to have conversations reliably, append <|end_of_text|>\n<|begin_of_text|>assistant: to the end of your prompt, [the speaker 'assistant' is flexible, and can be tooled to the type of response you want, for example "Mathematician:"" will give you a different type of response, than "felon:"]
later iterations of the model will likely have formatting similar to openchat
Conclusion
We intend to focus on updating and improving the performance of these models, and surrounding open sourced infrastructure. Our next effort will focus on context and implementing the research currently being conducted by Wing-Lian, the lead developer of the Axolotl training framework that underpins these experiments. We encourage the community to explore Wing-Lian's work, such as the Llama-3-8b-64k-PoSE and llama-3-8b-256k-PoSE models, which showcase the potential for further advancements in language modeling.
Buzz hopes to be a proof of concept, and a toolkit to demonstrate and enable the community in the pursuit of efficient and effective locally run, personally owned, language models. Through collaboration with Hive Digital Technologies who have enabled us to perform this research, we have demonstrated the immense potential for model reuse and optimization.
