You can deploy this model and start earning money today!

atlasium-efficient/
Llama-3.1-6B-25pct-Compressed-8B-EN-V1

Name: Llama-3.1-6B-25pct-Compressed-8B-EN-V1
Availability: InStock

meta-llama/Llama-3.1-8B

finetune

llama3.1

text-generation-inference

endpoints_compatible

Views

No views yet

Llama-3.1-6B — 25% Compressed from Llama-3.1-8B (English)

Built with Llama. This repository is part of the Efficient and Robust AI System (E-AI) Project by Vincent-Daniel Yun. This model is a compressed edition of meta-llama/Llama-3.1-8B with 8 of 32 transformer layers removed (24 layers remain, ≈6.29B parameters), recovered training-free. It is a base (non–chat-tuned) model.

🔗 Project: https://www.worldwidedaniel.com/eai-project 📅 Release: 2026-06-28 · Version: V1

⚖️ License: governed by the Llama 3.1 Community License Agreement (https://huggingface.co/meta-llama/Llama-3.1-8B/blob/main/LICENSE), and subject to the Llama 3.1 Acceptable Use Policy (https://www.llama.com/llama3_1/use-policy). By using this model you agree to those terms. "Llama" is a trademark of Meta Platforms, Inc. Built with Llama.

⚠️ Language: English-focused. Use as a discrimination / classification engine (classification, reading comprehension, domain QA, judging, safety screening) — open-ended long-form generation is degraded by compression (see PPL). For factual questions use retrieval (RAG).

About E-AI

Modern AI is powerful but heavy. State-of-the-art models are enormous and their inference is slow. The E-AI (Efficient-AI) project builds compact yet capable AI — making every model lightweight and fast, and keeping multi-agent teams reliable even when some members fail — so AI can assist people in urgent, high-stakes moments.

Method

The pruning method and the training-free recovery method are proprietary, undisclosed methods created by Vincent-Daniel Yun and are not released. Only the resulting model is shared.

Results (measured, full test sets)

PPL on 2048-token context (↓ better); downstream & MMLU are 0-shot via lm-eval-harness (↑ better).

Metric	Llama-3.1-8B (base)	This model (25%)
PPL · WikiText2 ↓	6.24	35.33
PPL · C4 ↓	8.68	35.17
ARC-c ↑	0.5350	0.4181
ARC-e ↑	0.8114	0.6553
BoolQ ↑	0.8208	0.6245
COPA ↑	0.8700	0.7300
HellaSwag ↑	0.7889	0.6375
OpenBookQA ↑	0.4480	0.3420
RACE ↑	0.3914	0.3722
RTE ↑	0.6931	0.7004
WinoGrande ↑	0.7356	0.7048
Avg. downstream (9) ↑	0.6771	0.5761
MMLU ↑	0.6360	0.6288

Task-suitability (vs base Llama-3.1-8B)

Where this compressed model stays close to the base on discrimination tasks (full test sets):

Task	Llama-3.1-8B (base)	This model (25%)
Topic classification (AG News)	0.564	0.796
LLM-as-judge (RewardBench)	0.660	0.693
SafetyBench (MCQ)	0.743	0.686
MultiRC	0.572	0.572
WiC	0.511	0.505
MRPC	0.674	0.444
CB (NLI)	0.643	0.732
SST-2	0.775	0.663
MedQA	0.598	0.599
MedMCQA	0.564	0.566
PubMedQA	0.758	0.602
Belebele-en	0.788	0.781
Belebele-ko	0.662	0.669
XNLI-zh	0.395	0.398
ToxiGen	0.432	0.431
TruthfulQA	0.452	0.481

(Discrimination is largely preserved; generation/PPL degrades with compression. Yes/No safety-F1 benchmarks are omitted as the non-chat base is poorly calibrated for them.)

Efficiency (measured, fp16, single GPU)

	Llama-3.1-8B (base)	This model (25%)
Layers	32	24
Parameters	8.03B	6.29B
Peak inference memory (fp16)	19.31 GB	15.55 GB (−19%)
Forward latency (fp16)	1293 ms	1011 ms (−22%)

Usage — Transformers

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("daniel-eai/Llama-3.1-6B-25pct-Compressed-8B-EN-V1", trust_remote_code=True, dtype=torch.float16, device_map="cuda")
tok = AutoTokenizer.from_pretrained("daniel-eai/Llama-3.1-6B-25pct-Compressed-8B-EN-V1", trust_remote_code=True)
ids = tok("The capital of France is", return_tensors="pt").to("cuda")
print(tok.decode(m.generate(**ids, max_new_tokens=20)[0]))

trust_remote_code=True is required (custom decoder layer in modeling_llama_recovered.py).

License & Acknowledgements

This model is a derivative of Llama 3.1 and is distributed under the Llama 3.1 Community License. Built with Llama. Thanks to Prof. Sai Praneeth Karimireddy (USC) and Prof. Sunwoo Lee (Inha University) for guidance, and to Meta for releasing Llama 3.1.

atlasium-efficient/Llama-3.1-6B-25pct-Compressed-8B-EN-V1