Mistral-7B-Wealth-Management-v1
Model Description
This model is a fine-tuned version of the
mistralai/Mistral-7B-Instruct-v0.2, specifically tailored for the wealth management domain. It is designed to handle question answering tasks, providing responses based on a specialized financial dataset.
Intended Use
- Recommended applications: This model is intended for automated financial advising, especially in the context of wealth management. You can use it in Chatbots and Copilots as a base model to answer user questions.
- Out-of-scope: The model is not intended for general conversational purposes and should not be used for medical, legal, or safety-critical advice.
Usage Example
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained("bitext-llm/Mistral-7B-Wealth-Management-v1")
4tokenizer = AutoTokenizer.from_pretrained("bitext-llm/Mistral-7B-Wealth-Management-v1")
5
6inputs = tokenizer("<s>[INST] What investment strategies are best for retirement savings?[/INST] ", return_tensors="pt")
7outputs = model.generate(inputs['input_ids'], max_length=50)
8print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Model Architecture
The model employs the MistralForCausalLM architecture with a LlamaTokenizer. It maintains the configuration of the base Mistral model but has been fine-tuned to better understand and generate responses related to wealth management.
Training Data
The model was fine-tuned using a private Bitext dataset designed for question and answer interactions in the wealth management sector. This dataset includes instructions and responses across a variety of financial topics, ensuring that the model can handle a wide range of inquiries related to this field. The dataset covers 24 intents such as
arrange_meeting,
calculate_portfolio_risk,
check_balances,
create_account, and more specialized queries like
portfolio_performance and
set_price_alert. Each intent has 1000 examples, illustrating a training process aimed at understanding and generating accurate responses for financial advisory services. The dataset follows the same structured approach as our dataset published on Hugging Face as
bitext/Bitext-customer-support-llm-chatbot-training-dataset, but with a focus on wealth management.
Training Procedure
Hyperparameters
- Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08
- Learning Rate: 0.0002 with a cosine learning rate scheduler
- Epochs: 4
- Batch Size: 10
- Gradient Accumulation Steps: 8
- Maximum Sequence Length: 8192 tokens
Environment
- Transformers Version: 4.40.0.dev0
- Framework: PyTorch 2.2.1+cu121
- Tokenizers: Tokenizers 0.15.0
Limitations and Bias
- The model is fine-tuned on a domain-specific dataset and may not perform well outside the scope of financial advice.
- Users should be aware of potential biases in the training data, as the model's responses may inadvertently reflect these biases. This model has been trained with a dataset that answers general wealth management questions, so potential biases may exist for specific use cases.
Ethical Considerations
This model should be used responsibly, considering ethical implications of automated financial advice. As it is a base model for this financial field, it is crucial to ensure that the model's advice complements human expertise and adheres to relevant financial regulations.
Acknowledgments
This model was developed by the Bitext and trained on infrastructure provided by Bitext.
License
This model, "Mistral-7B-Wealth-Management-v1", is licensed under the Apache License 2.0 by Bitext Innovations International, Inc. This open-source license allows for free use, modification, and distribution of the model but requires that proper credit be given to Bitext.
Key Points of the Apache 2.0 License
- Permissibility: Users are allowed to use, modify, and distribute this software freely.
- Attribution: You must provide proper credit to Bitext Innovations International, Inc. when using this model, in accordance with the original copyright notices and the license.
- Patent Grant: The license includes a grant of patent rights from the contributors of the model.
- No Warranty: The model is provided "as is" without warranties of any kind.
You may view the full license text at
Apache License 2.0.
This licensing ensures the model can be used widely and freely while respecting the intellectual contributions of Bitext. For more detailed information or specific legal questions about using this license, please refer to the official license documentation linked above.
