Model: pyloxsystems/pylox-voice-8b
Base model: meta-llama/Llama-3.1-8B-Instruct
Method: Supervised Fine-Tuning (SFT) + EAGLE-3 speculative decoding at serve time
Domain: Multi-turn conversational dialogue for voice-AI applications
Hardware: NVIDIA Grace Blackwell GB10 (DGX Spark, 128 GB unified memory)

Abstract

We present a parameter-efficient LoRA adapter fine-tuned on the Capybara multi-turn conversation dataset for the task of natural, flowing conversational dialogue generation — optimized for voice-AI latency budgets. Starting from meta-llama/Llama-3.1-8B-Instruct, we apply QLoRA (NF4, rank 32) across all seven linear projection layers via three epochs of supervised fine-tuning on 15,204 curated multi-turn conversation examples drawn from the full LDJnr/Capybara corpus. The adapter shifts the base model's register toward natural, multi-turn spoken-language dialogue patterns. Deployed with the RedHatAI EAGLE-3 speculative head, it achieves 2–3× lower first-token latency — critical for sub-300ms voice agent response budgets. MT-Bench overall score: 5.875 / 10 (target 6.5; humanities 7.41, STEM 6.77, writing 6.83 are strong; math and reasoning reflect the 8B model's inherent ceiling). Serves at 34+ tok/s on a single NVIDIA Grace Blackwell GB10.

Model Details

Property	Value
Base model	`meta-llama/Llama-3.1-8B-Instruct`
Adapter type	LoRA (PEFT)
LoRA rank	32
LoRA alpha	64
LoRA dropout	0.1
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Trainable parameters	~84M (~1.0% of base model)
Training method	SFT (3 epochs)
Training quantization	NF4 (bitsandbytes QLoRA, bnb_4bit_compute_dtype=bfloat16, double quant)
Inference quantization	NVFP4 via vLLM
Speculative decoding	EAGLE-3 (`RedHatAI/Llama-3.1-8B-Instruct-speculator.eagle3`, k=5)
License	Llama 3.1 Community License
Release date	2026-05

Intended Use

Primary use cases

Natural multi-turn dialogue generation for voice-AI assistants and agents
Conversational response generation where naturalness and turn coherence matter
Low-latency inference for real-time voice applications (Vapi, Retell, Bland, Voiceflow)
Avatar and video-gen conversational backends (Tavus, Argil, Synthesia)
Peer support and wellness conversational agents

Target users

Voice AI platform engineers, conversational AI developers, and researchers building real-time dialogue systems. Available under the Llama 3.1 Community License.

Out of scope

Not designed for single-turn Q&A, retrieval-augmented generation, or structured output tasks. The adapter improves multi-turn conversational register — it does not add factual knowledge or tool-calling capability. Not evaluated for clinical or crisis-support applications.

Training Data

Property	Value
Dataset	LDJnr/Capybara
HF repository	LDJnr/Capybara
License	Apache 2.0
Source rows	16,006 multi-turn conversation examples
After filtering	15,204 training examples
Eval split	5% held out (802 examples)

Preprocessing methodology

Format normalization — Capybara conversation: [{input, output}] structure converted to standard messages format (user/assistant alternating turns)
Near-duplicate removal — MinHash with Jaccard similarity threshold 0.92
Quality scoring — Each example scored 1–5 by a local 120B judge on conversational naturalness and turn coherence; examples below 3/5 discarded
Full corpus — No row cap applied; all 16,006 source examples processed (per pipeline policy: never subsample training data)

Training Procedure

SFT Hyperparameters

Hyperparameter	Value
Learning rate	2e-4
LR schedule	Cosine
Warmup ratio	0.03
Optimizer	paged_adamw_8bit
Gradient accumulation steps	16
Effective batch size	16
Max sequence length	2,048
Packing	True
NEFTune noise alpha	5
Epochs	3

Infrastructure

Property	Value
Hardware	NVIDIA Grace Blackwell GB10 (DGX Spark)
Unified memory	128 GB
Frameworks	PyTorch, Hugging Face transformers, peft 0.19.1, trl, bitsandbytes

Evaluation

MT-Bench

Evaluated via single-grade MT-Bench (80 questions, 2 turns each) using openai/gpt-oss-120b as judge (local TRT-LLM endpoint). Date: 2026-05-01.

Category	Score / 10
Overall	5.875
Humanities	7.412
Writing	6.833
STEM	6.765
Roleplay	6.056
Extraction	5.556
Math	5.500
Coding	4.800
Reasoning	4.333

The target was ≥6.5 overall. The adapter scores 5.875 — strong on the categories most relevant to voice applications (humanities, writing, roleplay) and weaker on math and reasoning, which is expected for an 8B SFT adapter without RL alignment on those domains.

Safety

Red-team evaluated against a 50-prompt adversarial suite (JailbreakBench, AdvBench, PAIR, DAN archive).

Metric	Value
Adversarial block rate (raw adapter)	77.78%
Benign control pass rate	100%
False positive rate	0%

Quickstart

PEFT (direct adapter loading)

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_id = "meta-llama/Llama-3.1-8B-Instruct"
adapter_id = "pyloxsystems/pylox-voice-8b"

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter_id)

messages = [
    {"role": "user", "content": "I've been feeling really disconnected lately. Like I'm going through the motions but nothing feels real."},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=256, do_sample=True, temperature=0.7)
print(tokenizer.decode(out[0], skip_special_tokens=True))

vLLM with EAGLE-3 (low-latency serving)

bash

vllm serve nvidia/Llama-3.1-8B-Instruct-NVFP4 \
    --enable-lora \
    --lora-modules pylox-voice-8b=pyloxsystems/pylox-voice-8b \
    --speculative-config '{"method": "eagle3", "model": "RedHatAI/Llama-3.1-8B-Instruct-speculator.eagle3", "num_speculative_tokens": 5}'

python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")
response = client.chat.completions.create(
    model="pylox-voice-8b",
    messages=[
        {"role": "system", "content": "You are a warm, natural conversational assistant."},
        {"role": "user", "content": "I've been really stressed about this job interview tomorrow."},
    ],
    max_tokens=256,
    temperature=0.7,
)
print(response.choices[0].message.content)

Limitations

MT-Bench 5.875 vs 6.5 target: Math (5.5) and reasoning (4.33) categories reflect the 8B model's inherent capability ceiling on multi-step reasoning, not a training failure. For voice applications (writing, roleplay, humanities), the scores are strong.
Conversational register only: The adapter improves multi-turn naturalness, not factual accuracy or domain knowledge. Inject domain knowledge via system prompt or RAG.
English only: Not evaluated on multilingual dialogue.
Max sequence length 2,048: Long conversation histories must be truncated or summarized.

Citation

bibtex

@misc{pylox_voice_8b_2026,
  author       = {Girard, Emilio},
  title        = {Pylox Voice 8B -- Llama-3.1-8B LoRA for Voice AI},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/pyloxsystems/pylox-voice-8b}}
}

Built at Pylox Forge — on-prem LLM fine-tuning and deployment on NVIDIA Grace Blackwell hardware.