CS Bitext — Llama-3.1-8B LoRA (SFT + DPO)
Model: emiliogirard/cs-bitext-llama-3.1-8b-lora-dpo
Base model: meta-llama/Llama-3.1-8B-Instruct
Method: Supervised Fine-Tuning (SFT) + Direct Preference Optimization (DPO)
Domain: Customer support dialogue generation
Hardware: NVIDIA Grace Blackwell GB10 (DGX Spark, 128 GB unified memory)
Abstract
We present a parameter-efficient LoRA adapter fine-tuned on the Bitext customer support LLM chatbot training dataset for the task of intent-aware customer support response generation. Starting from meta-llama/Llama-3.1-8B-Instruct, we apply QLoRA (NF4, rank 32) across all seven linear projection layers via three epochs of supervised fine-tuning on 378 curated support dialogue turns, followed by one epoch of DPO alignment on 132 preference pairs. The adapter learns the structural conventions, tone, and intent-handling patterns of professional customer support dialogue — including billing inquiries, order status, account management, product information, and complaint resolution — without encoding product-specific knowledge. The SFT checkpoint achieves a 32% pairwise win rate over the base model on held-out support prompts (scored by a local 120B judge). It serves at 27.9 tok/s on a single NVIDIA Grace Blackwell GB10 with NVFP4 quantization and EAGLE-3 speculative decoding, approximately 20× cheaper per token than GPT-4o for equivalent support dialogue generation.
Model Details
| Property | Value |
|---|
| Base model | meta-llama/Llama-3.1-8B-Instruct |
| Adapter type | LoRA (PEFT) |
| LoRA rank | 32 |
| LoRA alpha | 64 |
| LoRA dropout | 0.1 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Trainable parameters | ~84M (~1.0% of base model) |
| Training method | SFT (3 epochs) + DPO (1 epoch) |
| Training quantization | NF4 (bitsandbytes QLoRA, bnb_4bit_compute_dtype=bfloat16, double quant) |
| Inference quantization | NVFP4 via vLLM |
| Speculative decoding | EAGLE-3 (RedHatAI/Llama-3.1-8B-Instruct-speculator.eagle3, k=5) |
| License | Llama 3.1 Community License |
| Release date | 2026-05 |
Intended Use
Primary use cases
This adapter is intended for customer support dialogue generation tasks. Suitable applications include:
- Generating contextually appropriate, on-topic responses to customer inquiries
- Handling intent categories including billing, account management, order status, product information, and complaint resolution
- Powering customer-facing chatbot products where domain-specific knowledge is supplied via system prompt or RAG
- Rapid prototyping of support automation pipelines
Target users
Developers building customer-facing chatbot products, support automation engineers, and researchers working on task-oriented dialogue systems. The adapter is available for research and evaluation use under the Llama 3.1 Community License.
Out of scope
This adapter is a domain-style adapter — it learns conversational structure and tone, not product-specific knowledge. It does not encode knowledge about any particular company's products, policies, or account systems. It should not be used as a replacement for human agents in high-stakes support scenarios requiring legal judgment, escalation handling, or accessibility accommodations. It is not suitable for healthcare or financial services contexts where regulatory compliance governs customer communications.
Training Data
Preprocessing methodology
- Near-duplicate removal — MinHash with Jaccard similarity threshold 0.92; removes near-identical dialogue turns while preserving intent-category diversity
- Quality scoring — Each example is scored 1–5 by a local 120B judge model evaluating response helpfulness, appropriateness of tone, intent resolution quality, and absence of hallucinated policy claims; examples scoring below 3/5 are discarded
- PII redaction — Named entity recognition pass removes customer names, order numbers, account identifiers, and addresses from examples
- DPO pair construction — 132 preference pairs generated by sampling two responses per prompt from the SFT model (temperature 0.7 and 0.0), then ranked by the 120B judge on helpfulness and tone; used for the DPO alignment pass
Training Procedure
SFT Hyperparameters
| Hyperparameter | Value |
|---|
| Learning rate | 2e-4 |
| LR schedule | Cosine |
| Warmup ratio | 0.03 |
| Optimizer | paged_adamw_8bit |
| Gradient accumulation steps | 16 |
| Effective batch size | 16 |
| Max sequence length | 2,048 |
| Packing | True |
| NEFTune noise alpha | 5 |
| Epochs | 3 |
DPO Hyperparameters
| Hyperparameter | Value |
|---|
| Beta | 0.1 |
| Learning rate | 5e-6 |
| Epochs | 1 |
| Batch size | 1 |
| Gradient accumulation steps | 8 |
Infrastructure
| Property | Value |
|---|
| Hardware | NVIDIA Grace Blackwell GB10 (DGX Spark) |
| Unified memory | 128 GB |
| Frameworks | PyTorch, Hugging Face transformers, peft 0.19.1, trl, bitsandbytes |
Evaluation
Inference Performance
Measured against a live vLLM endpoint (NVFP4 + EAGLE-3, LoRA hot-loaded) on NVIDIA Grace Blackwell GB10. Target response length: 150 tokens.
| Metric | Value |
|---|
| Throughput — single user (mean) | 27.9 tok/s |
| Throughput — single user (peak) | 42.6 tok/s |
| Throughput — concurrent batch-8 (aggregate) | 128.4 tok/s |
| TTFT p50 | 127.9 ms |
| TTFT p95 | 136.8 ms |
| Total latency p50 (150-token response) | 5,460.4 ms |
| Total latency p95 (150-token response) | 10,346.8 ms |
Cost Analysis
Self-hosted electricity cost at $0.05/hr (Montreal hydro). Compute cost approaches $0 once hardware is amortized.
| Provider | Output cost ($/1M tokens) | Multiple vs self-hosted |
|---|
| Self-hosted (this adapter) | $0.4978 | baseline |
| GPT-4o | $10.00 | 20.1× more expensive |
| Claude Haiku 4.5 | $5.00 | 10.0× more expensive |
| GPT-4o-mini | $0.60 | 1.2× more expensive |
Academic Benchmarks
Evaluated via lm-evaluation-harness 0.4.x (local-completions model class) against the live vLLM NVFP4+EAGLE-3 endpoint. Tokenizer: nvidia/Llama-3.1-8B-Instruct-NVFP4. Limit: 50 samples per subtask. Date: 2026-05-01.
| Task | Metric | Score | Samples |
|---|
| MMLU-Pro (aggregate, 14 subjects) | exact_match | 42.7% | 700 |
| MMLU-Pro — Biology | exact_match | 70.0% | 50 |
| MMLU-Pro — Math | exact_match | 62.0% | 50 |
| MMLU-Pro — Economics | exact_match | 50.0% | 50 |
| MMLU-Pro — Other | exact_match | 54.0% | 50 |
| MMLU-Pro — Philosophy | exact_match | 48.0% | 50 |
| MMLU-Pro — Health | exact_match | 44.0% | 50 |
| MMLU-Pro — Computer Science | exact_match | 44.0% | 50 |
| MMLU-Pro — Psychology | exact_match | 40.0% | 50 |
| MMLU-Pro — Engineering | exact_match | 38.0% | 50 |
| MMLU-Pro — Business | exact_match | 42.0% | 50 |
| MMLU-Pro — Law | exact_match | 30.0% | 50 |
| MMLU-Pro — Chemistry | exact_match | 26.0% | 50 |
| MMLU-Pro — Physics | exact_match | 26.0% | 50 |
| MMLU-Pro — History | exact_match | 24.0% | 50 |
| HellaSwag | acc_norm | 76.0% | 50 |
| TruthfulQA MC1 | acc | 30.0% | 50 |
Domain Benchmarks
No standard off-shelf lm-evaluation-harness benchmark directly covers customer support dialogue quality.
LLM Judge (Pairwise Win Rate)
Pairwise comparison scored by a local gpt-oss-120b TRT-LLM judge. The judge receives a prompt plus two responses (finetune vs base model, order randomized) and picks the better one. Base model: meta-llama/Llama-3.1-8B-Instruct loaded in NF4 via bitsandbytes + PEFT. Evaluated on 50 held-out prompts from formatted_eval.jsonl. Date: 2026-05-01.
Note: This evaluation was run against the SFT checkpoint (adapter/), not the final DPO checkpoint (adapter_dpo/) that is published to Hugging Face. The DPO alignment pass is specifically designed to improve preference win rates; the DPO checkpoint is expected to score higher on pairwise preference evaluation, though it was not re-measured separately.
| Metric | Value |
|---|
| Prompts evaluated | 50 |
| Finetune wins | 16 (32%) |
| Base wins | 31 (62%) |
| Ties | 3 (6%) |
The SFT adapter wins on 32% of pairwise comparisons. This reflects the adapter's learned structure and tone conventions (intent acknowledgment, resolution confirmation, professional register) that the base model's general-purpose responses lack, while the base model retains an advantage on prompts where general helpfulness outweighs support-specific format.
Safety
Red-Team Evaluation
Evaluated against a 50-prompt adversarial suite drawn from JailbreakBench, AdvBench, PAIR, and the DAN archive. All tests conducted against the raw adapter endpoint without any external safety gateway.
| Metric | Value | Note |
|---|
| Adversarial block rate (raw adapter) | 0% | 45 attack prompts |
| Benign control pass rate | 100% | 5 benign controls |
The adapter inherits the safety alignment of the base Llama-3.1-8B-Instruct model. The 0% adversarial block rate at the raw adapter level is consistent with LoRA adapters trained without explicit red-team-targeted DPO. A 3-layer safety gateway (regex shields → Meta Prompt Guard 2 → Meta Llama Guard 3) is available via pylox deploy --with-safety. Customer-facing deployments should enable the gateway.
Limitations
- No product-specific knowledge: The adapter captures support dialogue style and structure, not any company's policies, products, or pricing. Product-specific context must be injected via system prompt or retrieval-augmented generation.
- Dataset scope: Trained on 378 examples from the Bitext dataset, which represents generic English-language B2C support conventions. May not generalize to B2B, enterprise SLA, or highly regulated industry support contexts.
- English only: Not evaluated on multilingual support scenarios.
- SFT win rate only: LLM judge win rate (32%) was measured on the SFT checkpoint; DPO checkpoint win rate was not separately measured.
- Sequence length: Max_seq_length=2,048. Long conversation histories must be truncated or summarized.
Bias, Fairness, and Ethical Considerations
This adapter produces responses based on patterns learned from Bitext customer support examples, which reflect generic English-language B2C support conventions. It may produce responses that assume Western business norms, return policies, or communication styles not applicable to the deployer's product or customer base. Outputs should be reviewed for brand fit and regulatory compliance before deployment. The adapter must not be used to handle sensitive customer data, financial disputes, or legal escalations without human oversight.
Quickstart
PEFT (direct adapter loading)
1from peft import PeftModel
2from transformers import AutoModelForCausalLM, AutoTokenizer
3import torch
4
5base_id = "meta-llama/Llama-3.1-8B-Instruct"
6adapter_id = "emiliogirard/cs-bitext-llama-3.1-8b-lora-dpo"
7
8tokenizer = AutoTokenizer.from_pretrained(base_id)
9model = AutoModelForCausalLM.from_pretrained(
10 base_id, torch_dtype=torch.bfloat16, device_map="auto"
11)
12model = PeftModel.from_pretrained(model, adapter_id)
13
14prompt = (
15 "You are a helpful customer support assistant.\n\n"
16 "Customer: I placed an order three days ago and I haven't received a shipping "
17 "confirmation yet. Order number is #84721. Can you help me?\n\n"
18 "Support Agent:"
19)
20inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
21out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
22print(tokenizer.decode(out[0], skip_special_tokens=True))
vLLM (OpenAI-compatible API)
1from openai import OpenAI
2
3client = OpenAI(base_url="http://localhost:8010/v1", api_key="none")
4response = client.chat.completions.create(
5 model="cs-bitext", # vLLM LoRA mount name
6 messages=[
7 {"role": "system", "content": "You are a helpful customer support assistant."},
8 {"role": "user", "content": "I was charged twice for my last order. How do I get a refund?"}
9 ],
10 max_tokens=256,
11)
12print(response.choices[0].message.content)
Citation
1@misc{girard_cs_bitext_2026,
2 author = {Girard, Emilio},
3 title = {CS Bitext -- Llama-3.1-8B LoRA (SFT + DPO)},
4 year = {2026},
5 publisher = {Hugging Face},
6 howpublished = {\url{https://huggingface.co/emiliogirard/cs-bitext-llama-3.1-8b-lora-dpo}}
7}
Built at
Pylox Forge — on-prem LLM fine-tuning and deployment on NVIDIA Grace Blackwell hardware.
