This is a GGUF quantized version of a fine-tuned Llama 3.1 8B Instruct model, specialized for fast-paced, Post-training Llama-3.1-8B mostly for story generation , less conversational role-play.
This model was fine-tuned using Unsloth on a curated dataset of over 300 examples designed to mimic a "quick response" chat style, similar to platforms like Character.AI. The persona is dominant, assertive, and direct, using a combination of expressive actions and concise dialogue.
This repository contains the Q4_K_M GGUF version, which offers an excellent balance of quality and performance for local inference.
Model Details
Base Model:unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
Fine-tuning Method: PEFT (LoRA) with Unsloth's performance optimizations.
LoRA Rank (r): 32
Format: GGUF
Quantization: Q4_K_M
How to Use in LM Studio
Search: Find this model (samunder12/llama-3.1-8b-roleplay-BSNL-gguf) on the LM Studio home screen.
Download: Download the llama3BSNL.Q4_K_M.gguf file.
Load: Go to the Chat tab (💬 icon) and select this model to load at the top.
Set Prompt Format: In the right-hand panel, under "Preset," select Llama 3. This is a critical step!
Set Context Length: Set the Context Length (n_ctx) to 4096 to match the model's training.
Apply a Sampler Preset: Use one of the presets below for the best experience.
Intended Use & Limitations
This model is intended for creative writing, immersive role-playing, and chatbot development where a quick, conversational interaction style is desired.
The model's output is unfiltered and reflects the persona and content of its training data.
It is highly specialized for its role-play task and may not perform well on other tasks like coding, summarization, or factual question-answering.
Training Procedure
Framework: Unsloth
Dataset: 513 examples of short-form, multi-turn conversational data. The data emphasizes a structure of *Action/Expression in asterisks.* Short, impactful dialogue.
