You can deploy this model and start earning money today!

SehwanMoon/
DepressLLM-llama3.3-70B

Name: DepressLLM-llama3.3-70B
Availability: InStock

Views

No views yet

Model Card for Model ID

DepressLLM was developed through domain‑adaptive fine‑tuning of the unsloth/llama-3.3-70b-instruct-bnb-4bit.
It predicts depression based on Experience of Happiness and Experience of Distress data.
We are currently preparing a paper entitled ‘A Domain‑Adapted Large Language Model Leveraging Real‑World Narrative Recordings for Interpretable Depression Detection.’
The GPT-4.1–fine-tuned model is available for immediate use at https://depressllm.streamlit.app/.

Developed by: Sehwan Moon
License: llama3.3
Finetuned from model: unsloth/llama-3.3-70b-instruct-bnb-4bit

The following is an example of model loading and inference.

python

from unsloth import FastLanguageModel
from peft import PeftModel
import torch, numpy as np

BASE = "unsloth/llama-3.3-70b-instruct-bnb-4bit"
ADAPTER = "SehwanMoon/DepressLLM-llama3.3-70B"  # HF adapter weights
max_seq_length = 7000

# Load base model + LoRA adapter
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = BASE,
    max_seq_length = max_seq_length,
    load_in_4bit = True,
)
model = PeftModel.from_pretrained(model, ADAPTER)
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
model.eval()

# Example: single transcript
sample = {
    "messages": [
        {"role": "user", "content": "A transcript of a participant talking about the topic of happiness and distress."},
    ]
}

# Prompt
prompt = (
    "You will be given a transcript of a participant talking about the topic of happiness and distress.\n"
    "Classify the transcript into one of the PHQ-9 scores (0–27).\n"
    f"Narrative:\n{sample['messages'][0]['content']}"
)

inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=max_seq_length)
inputs = {kk: vv.to(device) for kk, vv in inputs.items()}

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=1,           # 1-step next-token distribution
        do_sample=False,
        output_scores=True,
        return_dict_in_generate=True,
    )

# Next-token distribution
logits = torch.log_softmax(out.scores[0][0], dim=-1)
topk_logp, topk_idx = torch.topk(logits, k=50)

answer_list = []
for logp, idx in zip(topk_logp.tolist(), topk_idx.tolist()):
    token_str = tokenizer.decode([idx]).strip()
    prob_pct = float(np.round(np.exp(logp) * 100, 2))
    answer_list.append([token_str, prob_pct])

# Post-processing
def safe_convert_to_int(value):
    try:
        return int(value)
    except ValueError:
        return value

answer_list = [[safe_convert_to_int(item[0]), item[1]] for item in answer_list]

group_0_to_4, group_5_to_27 = 0, 0
for prediction, probability in answer_list:
    if isinstance(prediction, int):
        if 0 <= prediction <= 4:
            group_0_to_4 += probability
        elif 5 <= prediction <= 27:
            group_5_to_27 += probability

normalized = [(group_0_to_4 / (group_0_to_4 + group_5_to_27)) * 100,
              (group_5_to_27 / (group_0_to_4 + group_5_to_27)) * 100]

print("P(Depression) %:", normalized[1])

Prompt Instructions (as used in the paper)

This repository provides the prompt templates used in our paper for (i) score-based zero-shot classification, (ii) binary classification, and (iii) SToPS-based fine-tuning variants (with/without explanations and confidence). These prompts are designed for transcripts in which a participant describes both happy and distressing autobiographical memories.

Most users only need the Instruction for SToPS-Based Fine-Tuning with Explanation and Confidence prompt below.
Additional variants (binary, explanation, confidence) are provided for paper reproducibility and ablation settings.

Instruction for Score-Based Zero-Shot Classification

You will be given a transcript of a participant talking about happiness and distress.
Classify the transcript into one of the PHQ-9 scores (0–27).
Respond with only the score as an integer. Do not include any other text.

Instruction for Binary Classification

You will be given a transcript of a participant talking about happiness and distress.
Classify the transcript into one of the two classes: 0 (normal) or 1 (depression).

Instruction for SToPS-Based Fine-Tuning

You will be given a transcript of a participant talking about happiness and distress.
Classify the transcript into one of the PHQ-9 scores (0–27).

Instruction for SToPS-Based Fine-Tuning with Explanation

You will be given a transcript of a participant talking about happiness and distress.

Classify the transcript into one of the PHQ-9 scores (0–27).

Write a brief explanation for your prediction by referring to evidence from the transcript.

Highlight all significant words or phrases that influenced your decision, separated by commas.

Example output format:
<PHQ-9 score as integer>
Explanation: <Brief explanation, citing specific evidence from the transcript.>
Significant words/phrases: <phrase 1>, <phrase 2>, …

Instruction for SToPS-Based Fine-Tuning with Explanation and Confidence

You will be given a transcript of a participant talking about happiness and distress.

Classify the transcript into one of the PHQ-9 scores (0–27).

Write a brief explanation for your prediction by referring to evidence from the transcript.

Highlight all significant words or phrases that influenced your decision, separated by commas.

Provide a confidence score (as a percentage) indicating how certain you are of your prediction.

Example output format:
<PHQ-9 score as integer>
Explanation: <Brief explanation, citing specific evidence from the transcript.>
Significant words/phrases: <phrase 1>, <phrase 2>, …
Confidence score: <percentage>%

SehwanMoon/DepressLLM-llama3.3-70B