Model Details
Model Description
This is a fine-tuned version of the Llama 3.1 8B model, adapted for generating expert sommelier responses in Russian. The model was trained on a custom Russian-language dataset consisting of user query - sommelier response pairs.
- Developed by: [Your Name or Organization]
- Model type: Language Model
- Language(s) (NLP): Russian
- License: [Specify the license, e.g., MIT, Apache 2.0, etc.]
- Finetuned from model: Llama 3.1 8B
Model Sources [optional]
- Repository: [Your repository link if available]
Uses
Direct Use
This model can be used to generate sommelier-like responses to user queries about wine in Russian. It's designed to provide expert-level information and recommendations related to wine.
Out-of-Scope Use
This model should not be used for generating responses outside the domain of wine expertise or in languages other than Russian. It is not designed for general-purpose language tasks.
Bias, Risks, and Limitations
The model's knowledge is limited to the training data provided, which focuses on wine expertise. It may not have up-to-date information on recent wine releases or changes in the wine industry. The model may also reflect biases present in the training data.
Recommendations
Users should verify important information provided by the model, especially for critical decisions related to wine selection or pairing. The model should be used as a supportive tool rather than a definitive source of wine expertise.
Training Details
Training Data
The model was fine-tuned on a custom Russian-language dataset consisting of user query - sommelier response pairs. The exact size and composition of the dataset are not specified.
Training Procedure
Training Hyperparameters
- Training regime: Mixed precision (specific type not provided)
- Number of epochs: 1
- Batch size per device: 2
- Gradient Accumulation steps: 4
- Total batch size: 8
- Total steps: 60
- Number of trainable parameters: 83,886,080
Evaluation
Metrics
The primary metric used for evaluation was Training Loss. The loss decreased from an initial value of 2.6349 to 0.5164 over 60 training steps, indicating significant improvement in the model's performance.
Results
The training loss showed a consistent decrease over the 60 training steps, suggesting that the model successfully adapted to the task of generating sommelier-like responses.
Environmental Impact
- Hardware Type: 1 GPU (specific type not provided)
- Hours used: [Information needed]
- Cloud Provider: [Information needed]
- Compute Region: [Information needed]
- Carbon Emitted: [Information needed]
Technical Specifications [optional]
Compute Infrastructure
Hardware
1 GPU (specific details not provided)
Software
- Unsloth for faster finetuning
- Transformers library
Model Card Contact
Uploaded model
- Developed by: mcgrozov
- License: apache-2.0
- Finetuned from model : unsloth/meta-llama-3.1-8b-bnb-4bit
This llama model was trained 2x faster with
Unsloth and Huggingface's TRL library.
