MindBridge-Llama is a LoRA-adapted conversational language model fine-tuned for supportive mental health dialogue. Built on top of khazarai/MentalChat-16K, it is designed to generate empathetic, supportive, and context-aware responses to mental health-related conversations.

Note: This repository contains only the LoRA adapter weights. The base model (khazarai/MentalChat-16K) must be downloaded separately.

Model Details

Model Name: MindBridge-Llama
Base Model: khazarai/MentalChat-16K
Architecture: LLaMA + LoRA (PEFT)
Framework: Hugging Face Transformers + PEFT
Fine-tuned By: Smukherjee

Intended Use

This model is intended for:

Mental health conversational AI
Supportive dialogue generation
Educational NLP projects
Mental health LLM research
Experimental chatbot development

This model is intended only for research, educational, and experimental purposes.

Training

MindBridge-Llama was fine-tuned using Low-Rank Adaptation (LoRA) on a curated collection of publicly available mental health conversational datasets.

The fine-tuning process included:

Tokenization using the MentalChat tokenizer
LoRA-based parameter-efficient fine-tuning with PEFT
Supervised training using Hugging Face Transformers
Evaluation on a held-out validation split

Compared to the base model, MindBridge-Llama is specialized for the MindBridge project and is intended to work alongside MindBridge-RoBERTa to provide more context-aware conversational responses.

Requirements

Install the required libraries before loading the model.

bash

pip uninstall -y torchao
pip install peft transformers accelerate sentencepiece bitsandbytes

Note: If using Kaggle, restart the notebook session after uninstalling torchao.

Usage

Load the base model first and then apply the LoRA adapter.

python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

model_name = "khazarai/MentalChat-16K"

tokenizer = AutoTokenizer.from_pretrained(model_name)

base_model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

model = PeftModel.from_pretrained(
    base_model,
    "S-mukherjee/MindBridge-Llama"
)

Inference Example

Use the tokenizer's chat template before generating a response.

python

messages = [
    {
        "role": "user",
        "content": "I feel like nobody understands me anymore."
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
    add_generation_prompt=True,
)

inputs = {k: v.to(model.device) for k, v in inputs.items()}

outputs = model.generate(
    **inputs,
    max_new_tokens=150,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    repetition_penalty=1.1,
    pad_token_id=tokenizer.eos_token_id,
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Example

User

I feel like nobody understands me anymore.

Assistant

It sounds really tough and lonely to you. I'm sorry you're feeling this way. I'm here to listen if you'd like to talk about what's been making you feel this way.

Limitations

This model is intended only for research, educational, and experimental applications.

It is not a medical device and should not be used to diagnose mental health conditions or replace qualified mental health professionals. Generated responses may occasionally be inaccurate, incomplete, or inappropriate for sensitive situations. Human oversight is recommended for any real-world deployment.

Citation

If you use this model in research or projects, please cite this repository.

Acknowledgements

This model is built upon khazarai/MentalChat-16K and fine-tuned using the Hugging Face Transformers and PEFT libraries.

Related Models

This conversational model is part of the MindBridge project.

For mental health text classification, please also check out:

🧠 MindBridge-RoBERTa
https://huggingface.co/S-mukherjee/MindBridge-RoBERTa

Together, these models form a two-stage AI-assisted mental health support pipeline:

MindBridge-RoBERTa analyzes user text and identifies the mental health category.
MindBridge-Llama generates an empathetic and context-aware conversational response based on the user's input and the identified mental health context.

Author

Smukherjee

Developed as part of the MindBridge project, which explores the use of Large Language Models and NLP for mental health support and conversational AI.

🤗 Hugging Face: https://huggingface.co/S-mukherjee
📌 Project: MindBridge – AI-powered Mental Health Assistant