LLaMA 3.1 8B-Instruct Fine-Tuned for Tweet Generation
This model is a fine-tuned version of meta-llama/Meta-Llama-3.1-8B-Instruct, specifically adapted for generating short, expressive, and stylistically bold tweet-style text completions.
🧠 Use Case
The model is intended to generate creative, unfiltered, and stylistically unique tweets. It's useful in experimental language generation scenarios, character simulations, or ideation tools where punchy, opinionated, or stylistically exaggerated text is required.
📦 Base Model
meta-llama/Meta-Llama-3.1-8B-Instruct- Fine-tuned with chat-style prompts using
Trainer with FSDP and 8-bit optimization
🧪 Training Details
- Framework: PyTorch + Hugging Face Transformers
- Hardware: 2 x A100 80GB GPUs
- Optimizer:
adamw_bnb_8bit
- Gradient Accumulation: 16 steps
- Precision:
bfloat16 + Fully Sharded Data Parallel (FSDP)
- Epochs: 1
- Custom prompt formatting: Chat-completion (
{"role": ..., "content": ...})
🔤 Chat Template
Inference is performed with chat-style messages using Hugging Face’s apply_chat_template:
chat_prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
