Heavy models, made light.PocketWeights optimizes state-of-the-art open-source LLMs into efficient GGUF formats so you can run unrestricted, powerful AI locally on consumer hardware, gaming laptops, and edge devices.
🧠 About This Model
Meta-Llama-3.1-8B-Instruct-abliterated (created by huihui-ai) is an unaligned modification of Meta's flagship 8B model. Using orthogonal projection and activation steering, the internal refusal directions have been neutralized without destroying the model's core intelligence.
🎯 Key Highlights of this PocketWeights Edition
Zero Refusals & No Preachiness: Completely removes moralizing lectures, refusal triggers, and alignment guardrails. Ideal for complex creative writing, deep roleplay, red-team security audits, and uncensored synthetic dataset generation.
Massive 128K Context Window: Retains Meta's full Llama 3.1 architecture capabilities, allowing for book-length inputs, code repositories, and persistent long-term memory in multi-turn roleplay.
Exceptional Efficiency: Optimized across 3-bit, 4-bit, and 6-bit quantization tiers to run with low latency on budget GPUs (4GB–8GB VRAM) and Apple Silicon Macs.
⚠️ Disclaimer:This model has had its safety guardrails removed. It is designed for creative writers, researchers, and developers operating in secure, local environments.
📦 Available Files & Hardware Requirements
Quant Format
File Size
Recommended Hardware
Best For
IQ3_M
3.78 GB
4 GB VRAM / 6 GB RAM
Ultra-Low Memory. Fits budget GPUs (GTX 1650, RTX 3050, 8GB System RAM) with surprising conversational coherence.
Q4_K_M
4.92 GB
6 GB VRAM / 8 GB RAM
The Balanced Standard. The community sweet spot offering full Llama 3.1 intelligence for RTX 3060, RTX 4050, and M-series Macs.
Q6_K
6.60 GB
8 GB VRAM / 16 GB RAM
Near-Lossless Quality. Maximizes reasoning accuracy, nuance, and vocabulary richness for RTX 4060, 3070, and desktop rigs.
🚀 Beginner's Quick Start Guide
You can run this model offline on your local machine in under 2 minutes:
Option 1: LM Studio (Visual GUI — Easiest)
Download and install LM Studio (Free for Windows, macOS, and Linux).
Click the Magnifying Glass (Search) icon in the left navigation bar.
Click Download next to your preferred size (Q4_K_M is recommended), open the Chat Tab, load the model at the top, and start chatting!
Option 2: Ollama (Terminal / CLI)
If you already use Ollama, you can run any tier immediately with a single terminal command:
bash
1# Recommended 4-bit balanced tier2ollama run hf.co/PocketWeights/Llama-3.1-8B-Instruct-abliterated-GGUF:Q4_K_M
34# Low-VRAM 3-bit tier5ollama run hf.co/PocketWeights/Llama-3.1-8B-Instruct-abliterated-GGUF:IQ3_M
67# High-precision 6-bit tier8ollama run hf.co/PocketWeights/Llama-3.1-8B-Instruct-abliterated-GGUF:Q6_K
Option 3: Jan / Kobold.cpp / llama.cpp / SillyTavern
Direct Download: Go to the Files and versions tab above and download your desired .gguf file.
Load it directly into Kobold.cpp (perfect for SillyTavern roleplay), Jan.ai, Text-Generation-WebUI, or execute via llama.cpp:
llama-cli -m Llama-3.1-8B-Instruct-abliterated-GGUF-Q4_K_M.gguf -p "Write an unrestricted story about..." -ngl 33
🤝 Support the PocketWeights Mission
I build, verify, and maintain automated quantization pipelines to provide high-quality, unrestricted, and hardware-friendly models to the open-source community for free.
Running conversion clusters, cloud instances, and storage requires ongoing resources. If these weights have saved you time, compute overhead, or server bills, please consider supporting the project with a small tip!
