Model: emiliogirard/counseling-e2e-llama-3.1-8b-lora
Base model: meta-llama/Llama-3.1-8B-Instruct
Method: Supervised Fine-Tuning (SFT)
Domain: Empathetic mental health counseling dialogue
Hardware: NVIDIA Grace Blackwell GB10 (DGX Spark, 128 GB unified memory)

Clinical Disclaimer: This adapter is not a licensed mental health service and must not be used as a substitute for professional mental health care, crisis support, or clinical intervention. See the Safety and Limitations sections for full disclosure.

Abstract

We present a parameter-efficient LoRA adapter fine-tuned on the Amod mental health counseling conversations dataset for the task of empathetic counseling dialogue generation. Starting from meta-llama/Llama-3.1-8B-Instruct, we apply QLoRA (NF4, rank 16) across all seven linear projection layers via three epochs of supervised fine-tuning on 7 curated counseling conversation examples — a deliberately small dataset that produces a style adapter shifting the base model's register toward empathetic, supportive therapeutic language, rather than instilling domain-specific clinical knowledge. The adapter should be understood as a conversational tone adapter for research prototyping, not a robustly trained clinical system. It serves at 34.2 tok/s on a single NVIDIA Grace Blackwell GB10 with NVFP4 quantization and EAGLE-3 speculative decoding. Reported medical domain benchmark scores (MedQA, MedMCQA, MMLU medical subtasks) reflect the underlying base model's general medical knowledge, not counseling-specific capability acquired through fine-tuning.

Model Details

Property	Value
Base model	`meta-llama/Llama-3.1-8B-Instruct`
Adapter type	LoRA (PEFT)
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Trainable parameters	~42M (~0.5% of base model)
Training method	SFT only (3 epochs)
DPO	Not applied
Training quantization	NF4 (bitsandbytes QLoRA, bnb_4bit_compute_dtype=bfloat16, double quant)
Inference quantization	NVFP4 via vLLM
Speculative decoding	EAGLE-3 (`RedHatAI/Llama-3.1-8B-Instruct-speculator.eagle3`, k=5)
License	Llama 3.1 Community License
Release date	2026-05

Intended Use

Primary use cases

This adapter is intended for research applications exploring AI-assisted therapeutic dialogue and empathetic conversational response generation. Suitable tasks include:

Generating empathetic, supportive responses to emotional and personal disclosures in research settings
Prototyping peer support tool interfaces for evaluation under clinical supervision
Academic research on AI-generated therapeutic language and conversational register
Benchmarking empathetic response generation in AI systems

Target users

Mental health technology researchers, developers building AI-assisted support tools under clinical supervision, and academics studying AI in therapeutic contexts. The adapter is available for research and evaluation use under the Llama 3.1 Community License.

Out of scope

This adapter is not a licensed mental health service. It must not be used:

As a substitute for professional mental health care
As a standalone crisis support tool
With individuals in acute distress without human clinician oversight
For clinical diagnosis, treatment planning, or any therapeutic intervention
In production user-facing applications without clinical review and regulatory compliance assessment
For detecting or responding to suicidal ideation or acute psychiatric emergencies (the adapter was not trained for safety in these scenarios)

Training Data

Property	Value
Dataset	Mental Health Counseling Conversations
HF repository	Amod/mental_health_counseling_conversations
License	Unknown (see dataset card)
Source rows	29 counseling conversation examples
After filtering	7 training examples
DPO pairs	None (SFT only)
Eval split	5% held out

Important note on dataset size: After near-duplicate removal and quality filtering, only 7 training examples remain — an extremely small fine-tuning corpus. This adapter is a style adapter: it shifts the base model's conversational register toward empathetic counseling language. It does not constitute a robustly trained clinical system and should not be evaluated or deployed as one.

License note: The dataset license is listed as "Unknown" on the Hugging Face dataset card. Users integrating this adapter into commercial products should independently verify the dataset's licensing status before deployment.

Preprocessing methodology

Near-duplicate removal — MinHash with Jaccard similarity threshold 0.92; applied to remove paraphrased examples while preserving dialogue style diversity
Quality scoring — Each example is scored 1–5 by a local 120B judge model evaluating empathy, therapeutic appropriateness, and safety of the counselor response; examples scoring below 3/5 are discarded

Training Procedure

SFT Hyperparameters

Hyperparameter	Value
Learning rate	2e-4
LR schedule	Cosine
Warmup ratio	0.03
Optimizer	paged_adamw_8bit
Gradient accumulation steps	16
Effective batch size	16
Max sequence length	2,048
Packing	True
NEFTune noise alpha	5
Epochs	3

DPO was not applied. The dataset size (7 examples) was insufficient to generate a statistically meaningful preference pair set.

Infrastructure

Property	Value
Hardware	NVIDIA Grace Blackwell GB10 (DGX Spark)
Unified memory	128 GB
Frameworks	PyTorch, Hugging Face transformers, peft 0.19.1, trl, bitsandbytes

Evaluation

Academic Benchmarks

Evaluated via lm-evaluation-harness 0.4.x (local-completions model class) against the live vLLM NVFP4+EAGLE-3 endpoint. Tokenizer: nvidia/Llama-3.1-8B-Instruct-NVFP4. Limit: 50 samples per subtask. Date: 2026-05-01.

Task	Metric	Score	Samples
MMLU-Pro (aggregate, 14 subjects)	exact_match	43.1%	700
MMLU-Pro — Biology	exact_match	66.0%	50
MMLU-Pro — Math	exact_match	64.0%	50
MMLU-Pro — Economics	exact_match	50.0%	50
MMLU-Pro — Business	exact_match	50.0%	50
MMLU-Pro — Computer Science	exact_match	48.0%	50
MMLU-Pro — Philosophy	exact_match	46.0%	50
MMLU-Pro — Other	exact_match	46.0%	50
MMLU-Pro — Health	exact_match	44.0%	50
MMLU-Pro — Psychology	exact_match	36.0%	50
MMLU-Pro — Physics	exact_match	38.0%	50
MMLU-Pro — Engineering	exact_match	32.0%	50
MMLU-Pro — Chemistry	exact_match	30.0%	50
MMLU-Pro — Law	exact_match	28.0%	50
MMLU-Pro — History	exact_match	26.0%	50
HellaSwag	acc_norm	78.0%	50
TruthfulQA MC1	acc	30.0%	50

Domain Benchmarks

Evaluated via lm-evaluation-harness (local-completions model class) against the live vLLM endpoint. These scores reflect the underlying Llama-3.1-8B-Instruct base model's general medical knowledge, not counseling-specific capability acquired through fine-tuning on 7 examples.

Task	Metric	Score	Samples
MedQA (4-option)	acc	52.0%	50
MedMCQA	acc	54.0%	50
MMLU — Medical Genetics	acc_norm	70.0%	50
MMLU — Clinical Knowledge	acc_norm	66.0%	50
MMLU — College Medicine	acc_norm	70.0%	50

Inference Performance

Measured against a live vLLM endpoint (NVFP4 + EAGLE-3, LoRA hot-loaded) on NVIDIA Grace Blackwell GB10. Target response length: 150 tokens.

Metric	Value
Throughput — single user (mean)	34.2 tok/s
Throughput — single user (peak)	45.4 tok/s
Throughput — concurrent batch-8 (aggregate)	262.1 tok/s
TTFT p50	169.1 ms
TTFT p95	549.2 ms
Total latency p50 (150-token response)	3,901.0 ms
Total latency p95 (150-token response)	8,642.8 ms

Cost Analysis

Self-hosted electricity cost at $0.05/hr (Montreal hydro). Compute cost approaches $0 once hardware is amortized.

Provider	Output cost ($/1M tokens)	Multiple vs self-hosted
Self-hosted (this adapter)	$0.4061	baseline
GPT-4o	$10.00	24.6× more expensive
Claude Haiku 4.5	$5.00	12.3× more expensive
GPT-4o-mini	$0.60	1.5× more expensive

LLM Judge (Pairwise Win Rate)

Pairwise comparison was attempted using a local gpt-oss-120b TRT-LLM judge. Base model: meta-llama/Llama-3.1-8B-Instruct loaded in NF4 via bitsandbytes + PEFT. Date: 2026-05-01.

Note: Only 1 eval prompt was available after the 5% train/eval split on 7 training examples. With n=1, pairwise win rate is not statistically meaningful. The single comparison resulted in a tie. LLM judge win rate will be reported if the adapter is retrained on a larger dataset.

Safety

Red-Team Evaluation

Evaluated against a 50-prompt adversarial suite drawn from JailbreakBench, AdvBench, PAIR, and the DAN archive. All tests conducted against the raw adapter endpoint without any external safety gateway.

Metric	Value	Note
Adversarial block rate (raw adapter)	0%	45 attack prompts
Benign control pass rate	100%	5 benign controls

The adapter inherits the safety alignment of the base Llama-3.1-8B-Instruct model. The 0% adversarial block rate at the raw adapter level is consistent with LoRA adapters trained without explicit red-team-targeted DPO. A 3-layer safety gateway (regex shields → Meta Prompt Guard 2 → Meta Llama Guard 3) is available via pylox deploy --with-safety.

Any user-facing deployment of this adapter must enable the safety gateway, given the sensitive nature of mental health applications and the risk of vulnerable users being exposed to unsafe model outputs.

Crisis Response Limitation

This adapter was not trained to detect or respond to suicidal ideation, self-harm disclosures, or acute psychiatric emergencies. Deployments must independently implement crisis detection, crisis resource referral (e.g., national crisis hotlines), and mandatory human escalation pathways.

Limitations

Dataset size: Only 7 training examples after filtering. This adapter captures empathetic language patterns, not robust clinical knowledge. It must not be evaluated or deployed as a clinical system.
No clinical validation: The adapter has not been evaluated for therapeutic effectiveness, clinical accuracy, or safety in crisis presentations by mental health professionals.
Bias from small sample: With 7 examples, the adapter may overfit to specific conversational styles or cultural perspectives present in those examples, producing outputs that feel generic or miss culturally appropriate nuances.
Unknown dataset license: The source dataset's license status is not confirmed on its HF card. Verify before commercial use.
Not crisis-safe: Not trained to handle suicidal ideation, self-harm disclosures, or psychiatric emergencies. Human escalation paths are mandatory in any deployment.
English only: Not evaluated on multilingual counseling contexts.
Academic benchmarks: General academic scores (MMLU general, HellaSwag, TruthfulQA) were not measured in this run.

Bias, Fairness, and Ethical Considerations

This adapter learns from a sample of 7 counseling conversation patterns and may reflect the specific cultural, linguistic, and therapeutic approach biases present in those examples. It may not appropriately handle mental health presentations across all cultural contexts, intersectional identities, or specific diagnoses. Outputs should never be treated as therapeutic advice. Any deployment must include human clinician oversight, cultural competency review, and compliance with applicable privacy regulations (HIPAA, GDPR, etc.) when processing user conversations. This adapter does not store or process user data — deployers bear independent responsibility for data privacy compliance.

Quickstart

PEFT (direct adapter loading)

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_id = "meta-llama/Llama-3.1-8B-Instruct"
adapter_id = "emiliogirard/counseling-e2e-llama-3.1-8b-lora"

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter_id)

prompt = (
    "You are a compassionate and empathetic counseling assistant. "
    "Listen carefully and respond with warmth and support.\n\n"
    "Person: I've been feeling really overwhelmed lately. Work has been stressful "
    "and I can't seem to turn my mind off at night. I don't know what to do.\n\n"
    "Counselor:"
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.7)
print(tokenizer.decode(out[0], skip_special_tokens=True))

vLLM (OpenAI-compatible API)

python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8010/v1", api_key="none")
response = client.chat.completions.create(
    model="counseling-e2e",  # vLLM LoRA mount name
    messages=[
        {
            "role": "system",
            "content": (
                "You are a compassionate and empathetic counseling assistant. "
                "Listen carefully and respond with warmth and support."
            )
        },
        {"role": "user", "content": "I've been feeling anxious about my job situation and it's affecting my sleep."}
    ],
    max_tokens=512,
    temperature=0.7,
)
print(response.choices[0].message.content)

Citation

bibtex

@misc{girard_counseling_e2e_2026,
  author       = {Girard, Emilio},
  title        = {Counseling E2E -- Llama-3.1-8B LoRA (SFT)},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/emiliogirard/counseling-e2e-llama-3.1-8b-lora}}
}

Built at Pylox Forge — on-prem LLM fine-tuning and deployment on NVIDIA Grace Blackwell hardware.