Quantization made by Richard Erkhov.
Mistral-7B-Customer-Support - GGUF
Original model description:
license: apache-2.0
inference: false
pipeline_tag: text-generation
tags:
- axolotl
- generated_from_trainer
- text-generation-inference
model-index:
- name: Mistral-7B-instruct-v0.2
results: []
model_type: mistral
widget:
- messages:
- role: user
content: I want to cancel an order
Mistral-7B-Customer-Support-v1
Model Description
This model, ["Mistral-7B-Customer-Support-v1"], is a fine-tuned version of the
mistralai/Mistral-7B-Instruct-v0.2, specifically tailored for the Custumer Support domain. It is optimized to answer questions and assist users with various support transactions. It has been trained using hybrid synthetic data generated using our NLP/NLG technology and our automated Data Labeling (DAL) tools.
The goal of this model is to show that a generic verticalized model makes customization for a final use case much easier. For example, if you are "ACME Company", you can create your own customized model by using this fine-tuned model and a doing an additional fine-tuning using a small amount of your own data. An overview of this approach can be found at:
From General-Purpose LLMs to Verticalized Enterprise ModelsIntended Use
- Recommended applications: This model is designed to be used as the first step in Bitext’s two-step approach to LLM fine-tuning for the creation of chatbots, virtual assistants and copilots for the Customer Support domain, providing customers with fast and accurate answers about their banking needs.
- Out-of-scope: The model is not intended for general conversational purposes and should not be used for medical, legal, or safety-critical advice.
Usage Example
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained("bitext-llm/Mistral-7B-Customer-Support-v1")
4tokenizer = AutoTokenizer.from_pretrained("bitext-llm/Mistral-7B-Customer-Support-v1")
5
6inputs = tokenizer("<s>[INST] I want to change to the standard account [/INST] ", return_tensors="pt")
7outputs = model.generate(inputs['input_ids'], max_length=50)
8print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Model Architecture
The model employs the MistralForCausalLM architecture with a LlamaTokenizer. It maintains the configuration of the base Mistral model but has been fine-tuned to better understand and generate responses related to customer service.
Training Data
The model was fine-tuned using the
bitext/Bitext-customer-support-llm-chatbot-training-dataset, which is designed for question and answer interactions in the customer service sector. This dataset includes instructions and responses across a variety of customer service topics, ensuring that the model can handle a wide range of inquiries related to this field. The dataset covers 27 intents assigned to 10 categories such as
cancel_order,
place_order,
change_order, and
check_invoice. Each intent has around 1000 examples, illustrating a training process aimed at understanding and generating accurate responses for customer service interactions.
Training Procedure
Hyperparameters
- Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08
- Learning Rate: 0.0002 with a cosine learning rate scheduler
- Epochs: 1
- Batch Size: 8
- Gradient Accumulation Steps: 4
- Maximum Sequence Length: 1024 tokens
Environment
- Transformers Version: 4.40.0.dev0
- Framework: PyTorch 2.2.1+cu121
- Tokenizers: Tokenizers 0.15.0
Training Results
| Training Loss | Epoch | Step | Validation Loss |
|---|
| 1.6865 | 0.01 | 1 | 2.0557 |
| 0.6351 | 0.25 | 32 | 0.8355 |
| 0.5724 | 0.5 | 64 | 0.7859 |
| 0.5249 | 0.75 | 96 | 0.7711 |
| 0.516 | 1.0 | 128 | 0.7667 |
Dataset Description
Overview
The dataset used for fine-tuning can train Large Language Models for both Fine Tuning and Domain Adaptation. It includes:
- Use Case: Intent Detection
- Vertical: Customer Service
- 27 intents assigned to 10 categories
- 26,872 question/answer pairs, around 1000 per intent
- 30 entity/slot types
- 12 different types of language generation tags
Categories and Intents
The dataset covers the following categories and intents:
- ACCOUNT: create_account, delete_account, edit_account, switch_account
- CANCELLATION_FEE: check_cancellation_fee
- DELIVERY: delivery_options
- FEEDBACK: complaint, review
- INVOICE: check_invoice, get_invoice
- NEWSLETTER: newsletter_subscription
- ORDER: cancel_order, change_order, place_order
- PAYMENT: check_payment_methods, payment_issue
- REFUND: check_refund_policy, track_refund
- SHIPPING_ADDRESS: change_shipping_address, set_up_shipping_address
Entities
The dataset includes various entities such as:
- {{Order Number}}, {{Invoice Number}}, {{Online Order Interaction}}, {{Online Payment Interaction}}, {{Online Navigation Step}}, {{Online Customer Support Channel}}, {{Profile}}, {{Profile Type}}, {{Settings}}, {{Online Company Portal Info}}, {{Date}}, {{Date Range}}, {{Shipping Cut-off Time}}, {{Delivery City}}, {{Delivery Country}}, {{Salutation}}, {{Client First Name}}, {{Client Last Name}}, {{Customer Support Phone Number}}, {{Customer Support Email}}, {{Live Chat Support}}, {{Website URL}}, {{Upgrade Account}}, {{Account Type}}, {{Account Category}}, {{Account Change}}, {{Program}}, {{Refund Amount}}, {{Money Amount}}, {{Store Location}}
Language Generation Tags
The dataset contains tags for various linguistic phenomena:
- Lexical Variation: Morphological (M), Semantic (L)
- Syntactic Structure Variation: Basic (B), Interrogative (I), Coordinated (C), Negation (N)
- Language Register Variations: Politeness (P), Colloquial (Q), Offensive (W)
- Stylistic Variations: Keyword (K), Abbreviations (E), Errors and Typos (Z)
- Other Tags: Indirect Speech (D), Regional Variations (G), Respect Structures (R), Code Switching (Y)
Limitations and Bias
- The model is fine-tuned on a domain-specific dataset and may not perform well outside the scope of customer service.
- Users should be aware of potential biases in the training data, as the model's responses may inadvertently reflect these biases. The dataset aims to cover general customer service inquiries, but biases may exist for specific use cases.
Ethical Considerations
This model should be used responsibly, considering ethical implications of automated customer service. It is important to ensure that the model's advice complements human expertise and adheres to relevant customer service guidelines.
Acknowledgments
This model was developed by Bitext and trained on infrastructure provided by Bitext.
License
This model, "Mistral-7B-Customer-Support-v1," is licensed under the Apache License 2.0 by Bitext Innovations International, Inc. This open-source license allows for free use, modification, and distribution of the model but requires that proper credit be given to Bitext.
Key Points of the Apache 2.0 License
- Permissibility: Users are allowed to use, modify, and distribute this software freely.
- Attribution: You must provide proper credit to Bitext Innovations International, Inc. when using this model, in accordance with the original copyright notices and the license.
- Patent Grant: The license includes a grant of patent rights from the contributors of the model.
- No Warranty: The model is provided "as is" without warranties of any kind.
You may view the full license text at
Apache License 2.0.
This licensing ensures the model can be used widely and freely while respecting the intellectual contributions of Bitext. For more detailed information or specific legal questions about using this license, please refer to the official license documentation linked above.
See axolotl config
axolotl version: 0.4.0
1base_model: mistralai/Mistral-7B-Instruct-v0.2
2model_type: MistralForCausalLM
3tokenizer_type: LlamaTokenizer
4is_mistral_derived_model: true
5
6hub_model_id: malmarjeh/Mistral-7B-instruct-v0.2
7
8load_in_8bit: false
9load_in_4bit: true
10strict: false
11
12datasets:
13 - path: bitext/Bitext-customer-support-llm-chatbot-training-dataset
14 type:
15 system_prompt: 'You are an expert in customer support.'
16 field_instruction: instruction
17 field_output: response
18 format: '[INST] {instruction} [/INST]'
19 no_input_format: '[INST] {instruction} [/INST]'
20
21#datasets:
22# - path: json
23# type: alpaca_w_system.load_open_orca
24#data_files: file.zip
25
26dataset_prepared_path:
27
28val_set_size: 0.05
29output_dir: ./qlora-out
30
31adapter: qlora
32lora_model_dir:
33
34sequence_len: 1024
35sample_packing: true
36pad_to_sequence_len: true
37
38eval_sample_packing: False
39
40lora_r: 32
41lora_alpha: 16
42lora_dropout: 0.05
43lora_target_linear: true
44lora_fan_in_fan_out:
45lora_target_modules:
46 - gate_proj
47 - down_proj
48 - up_proj
49 - q_proj
50 - v_proj
51 - k_proj
52 - o_proj
53
54wandb_project: axolotl
55wandb_entity:
56wandb_watch:
57wandb_name:
58wandb_log_model:
59
60gradient_accumulation_steps: 4
61micro_batch_size: 8
62num_epochs: 1
63optimizer: adamw_bnb_8bit
64lr_scheduler: cosine
65learning_rate: 0.0002
66
67train_on_inputs: false
68group_by_length: false
69bf16: auto
70fp16:
71tf32: false
72
73gradient_checkpointing: true
74early_stopping_patience:
75resume_from_checkpoint:
76local_rank:
77logging_steps: 1
78xformers_attention:
79flash_attention: true
80
81loss_watchdog_threshold: 5.0
82loss_watchdog_patience: 3
83
84warmup_steps: 10
85evals_per_epoch: 4
86eval_table_size:
87eval_max_new_tokens: 128
88saves_per_epoch: 1
89debug:
90deepspeed:
91weight_decay: 0.0
92fsdp:
93fsdp_config:
94special_tokens:
95 bos_token: '<s>'
96 eos_token: '</s>'
97 unk_token: '<unk>'
