# Download the model
wget https://huggingface.co/pavishanth-sujeevan/llama-3.2-3b-english-therapy-GGUF/resolve/main/model.gguf

# Run inference
./llama-cli -m model.gguf -p "I'm feeling anxious about my future" -n 200

With llama-cpp-python

python

from llama_cpp import Llama

llm = Llama(
    model_path="model.gguf",
    n_ctx=2048,
    n_threads=4,
    n_gpu_layers=35
)

output = llm(
    "User: I'm feeling stressed about work.\nTherapist:",
    max_tokens=200,
    temperature=0.7,
    top_p=0.9
)

print(output["choices"][0]["text"])

With Streamlit

python

import streamlit as st
from llama_cpp import Llama

@st.cache_resource
def load_model():
    return Llama(model_path="model.gguf", n_ctx=2048, n_gpu_layers=35)

llm = load_model()
user_input = st.text_input("How are you feeling?")

if user_input:
    response = llm(f"User: {user_input}\nTherapist:", max_tokens=200)
    st.write(response["choices"][0]["text"])

Intended Use

This model is designed for:

Mental health chatbots
Therapy conversation simulation
Empathetic response generation
Research in AI-assisted counseling

Limitations

Not a replacement for professional therapy
May produce incorrect or inappropriate responses
Requires human oversight in production use
Limited to English language

Training Details

Training Method: LoRA fine-tuning
Epochs: 3
Learning Rate: 2e-4
Batch Size: 1 (gradient accumulation: 8)

License

Apache 2.0 (inherits from LLaMA 3.2 base model)

E-motionAssistant/llama-3.2-3b-english-therapy-GGUF

LLaMA 3.2-3B English Therapy - GGUF Q4_K_M

Model Details

Performance Metrics

Usage

With llama.cpp

With llama-cpp-python

With Streamlit

Intended Use

Limitations

Training Details

License

E-motionAssistant/
llama-3.2-3b-english-therapy-GGUF