You can deploy this model and start earning money today!

RichardErkhov/
cmarkea_-_bloomz-3b-dpo-chat-gguf

Name: cmarkea_-_bloomz-3b-dpo-chat-gguf
Availability: InStock

endpoints_compatible

gguf

template

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

bloomz-3b-dpo-chat - GGUF

Model creator: https://huggingface.co/cmarkea/
Original model: https://huggingface.co/cmarkea/bloomz-3b-dpo-chat/

Name	Quant method	Size
bloomz-3b-dpo-chat.Q2_K.gguf	Q2_K	1.52GB
bloomz-3b-dpo-chat.IQ3_XS.gguf	IQ3_XS	1.68GB
bloomz-3b-dpo-chat.IQ3_S.gguf	IQ3_S	1.71GB
bloomz-3b-dpo-chat.Q3_K_S.gguf	Q3_K_S	1.71GB
bloomz-3b-dpo-chat.IQ3_M.gguf	IQ3_M	1.81GB
bloomz-3b-dpo-chat.Q3_K.gguf	Q3_K	1.9GB
bloomz-3b-dpo-chat.Q3_K_M.gguf	Q3_K_M	1.9GB
bloomz-3b-dpo-chat.Q3_K_L.gguf	Q3_K_L	2.02GB
bloomz-3b-dpo-chat.IQ4_XS.gguf	IQ4_XS	2.0GB
bloomz-3b-dpo-chat.Q4_0.gguf	Q4_0	2.08GB
bloomz-3b-dpo-chat.IQ4_NL.gguf	IQ4_NL	2.09GB
bloomz-3b-dpo-chat.Q4_K_S.gguf	Q4_K_S	2.09GB
bloomz-3b-dpo-chat.Q4_K.gguf	Q4_K	2.24GB
bloomz-3b-dpo-chat.Q4_K_M.gguf	Q4_K_M	2.24GB
bloomz-3b-dpo-chat.Q4_1.gguf	Q4_1	2.25GB
bloomz-3b-dpo-chat.Q5_0.gguf	Q5_0	2.43GB
bloomz-3b-dpo-chat.Q5_K_S.gguf	Q5_K_S	2.43GB
bloomz-3b-dpo-chat.Q5_K.gguf	Q5_K	2.55GB
bloomz-3b-dpo-chat.Q5_K_M.gguf	Q5_K_M	2.55GB
bloomz-3b-dpo-chat.Q5_1.gguf	Q5_1	2.6GB
bloomz-3b-dpo-chat.Q6_K.gguf	Q6_K	2.8GB
bloomz-3b-dpo-chat.Q8_0.gguf	Q8_0	3.62GB

Original model description:

license: bigscience-bloom-rail-1.0 datasets:

Anthropic/hh-rlhf language:
en
fr

bloomz-3b-dpo-chat Model Card

Model Overview

The bloomz-3b-dpo-chat is a conversational model fine-tuned using Direct Preference Optimization (DPO) from the base bloomz-3b-sft-chat model. This model aims to provide high-quality conversational abilities in both English and French, leveraging the pre-trained strengths of its SFT (Supervised Fine-Tuning) predecessor.

Parent Model: bloomz-3b-sft-chat

Model Description

The bloomz-3b-dpo-chat model builds upon the solid foundation of the bloomz-3b-sft-chat, which is notable for its chatbot-specific pre-training and efficient tokenization strategy. The DPO fine-tuning process enhances the model's ability to generate more human-preferred responses in conversational contexts.

Multilingual Capabilities

The model was initially trained on both French and English datasets, ensuring high efficiency and performance in these languages. Due to the DPO process and potential data type changes (from float16 to bfloat16), the model's multilingual capabilities might not be as robust as its SFT predecessor, but fine-tuning can help in restoring performance in other languages.

Model Applications

This model is suitable for chatbot applications, customer service automation, and other conversational AI systems where bilingual (French and English) support is essential.

Dataset

The bloomz-3b-dpo-chat model was trained using the Anthropic/hh-rlhf dataset, which includes:

Human Preference Data:

Description: Annotations of helpfulness and harmlessness, with each entry containing "chosen" and "rejected" text pairs.
Purpose: To train preference models for Reinforcement Learning from Human Feedback (RLHF), not for supervised training of dialogue agents.
Source: Data from context-distilled language models, rejection sampling, and an iterated online process.

Evaluation

Evaluation of the model was conducted using the PoLL (Pool of LLM) technique, assessing performance on 100 French questions with scores aggregated from six evaluations (two per evaluator). The evaluators included GPT-4o, Gemini-1.5-pro, and Claude3.5-sonnet.

Performance Scores (on a scale of 5):

Model	Score	# params
gpt-4o	4.13	N/A
mistralai/Mixtral-8x7B-Instruct-v0.1	3.71	46.7b
gpt-3.5-turbo	3.66	175b
mistralai/Mistral-7B-Instruct-v0.2	1.98	7.25b
cmarkea/bloomz-7b1-mt-sft-chat	1.69	7.1b
cmarkea/bloomz-3b-dpo-chat	1.68	3b
cmarkea/bloomz-3b-sft-chat	1.51	3b
croissantllm/CroissantLLMChat-v0.1	1.19	1.3b
cmarkea/bloomz-560m-sft-chat	1.04	0.56b
OpenLLM-France/Claire-Mistral-7B-0.1	0.38	7.25b

The bloomz-3b-dpo-chat model demonstrates improved performance over its SFT counterpart, particularly in zero-shot contexts, making it a competitive choice for production environments.

Usage

To utilize the bloomz-3b-dpo-chat model, format the prompt for chatbot interactions as follows:

</s>[human prompt 1]<s>[bot answer 1]</s>[human prompt 2]<s>

Example code to load the model using HuggingFace's pipeline:

python

from transformers import pipeline

model = pipeline("text-generation", "cmarkea/bloomz-3b-dpo-chat")
result = model("</s>C'est quoi le deep learning ?<s>", max_new_tokens=512)

result
[{'generated_text': "</s>C'est quoi le deep learning ?<s>L'apprentissage
   en profondeur est un sous-ensemble de l'apprentissage automatique qui
   utilise des réseaux de neurones artificiels pour apprendre à partir de
   données. Ces réseaux sont conçus pour reconnaître des modèles dans les
   données et peuvent être utilisés pour des tâches telles que la 
   reconnaissance d'images, le traitement du langage naturel et la
   reconnaissance vocale."}]

Citation

bibtex

@online{DeBloomzChat,
  AUTHOR = {Cyrile Delestre},
  URL = {https://huggingface.co/cmarkea/bloomz-3b-dpo-chat},
  YEAR = {2024},
  KEYWORDS = {NLP ; Transformers ; LLM ; Bloomz},
}

RichardErkhov/cmarkea_-_bloomz-3b-dpo-chat-gguf