Cat1.0

Overview

Model Specifications

• Parameters: 8 Billion (8B)
• Precision: bf16 (Brain Floating Point 16-bit)
• Fine-Tuning Method: LoRa (Low-Rank Adaptation)
• Lora Rank: 32
• Lora Alpha: 64
• Learning Rate: 0.0008
• Training Epochs: 4
• Datasets Used:
  • cat1.0 Roleplay Dataset
  • cat1.0 Reasoning and Logic Dataset
• Fine-Tuning Approach: Iterative Fine-Tuning using self-chat logs

Recommended Settings

• Temperature: 1.1
• Min P: 0.05

Note: Due to the nature of the fine-tuning, setting the temperature to 1.1 or higher helps prevent the model from repeating itself and encourages more creative and coherent responses.

Usage Instructions

Installation Steps

1. Clone the WebUI Repository:
   bash

   1git clone https://github.com/oobabooga/text-generation-webui 2cd text-generation-webui
2. Install Dependencies:
   pip install -r requirements.txt
3. Download the Model:
   Download the fine-tuned model from Hugging Face and place it in the models directory.
4. Launch the WebUI:
   python server.py --bf16 --flash-attention

Sample Prompt Formats

Model Capabilities

Roleplay Examples

Text Generation Example

Limitations and Tips

• Increase the Temperature: Setting the temperature higher (≥ 1.1) can help generate more diverse and creative outputs.
• Adjust min_p Setting: Ensuring min_p is at least 0.05 can prevent low-probability tokens from being excluded, enhancing the response quality.

Acknowledgments

• oobabooga text-generation-webui: A powerful interface for running and interacting with language models. GitHub Repository
• Hugging Face: For hosting the model and providing a platform for collaboration. Website
• Meta For pre-training the Llama-3.1-8B Base Model that was used for fine-tuning. Model Card