You can deploy this model and start earning money today!

DemonC/
ZenBot

Views

No views yet

ZenBot: A Local LLM for Personalized Stress and Sleep Support.

This repository contains a pruned version of a specialized LLM, fine-tuned for chat and general text generation. The model has been optimized for size while retaining high performance.

Model Files

config.json - Model architecture configuration.
model.safetensors - Model weights in safe tensor format.
tokenizer.json - Tokenizer vocabulary.
tokenizer_config.json - Tokenizer configuration.
special_tokens_map.json - Mapping of special tokens.
chat_template.jinja - Template for chat prompts (optional).
generation_config.json - Default generation parameters.

Installation

Install the required packages:

pip install torch transformers safetensors

Usage

Loading the Model

python

from transformers import AutoModelForCausalLM, AutoTokenizer

# Replace with your Hugging Face repo path
repo_id = "DemonC/ZeBot"

tokenizer = AutoTokenizer.from_pretrained(repo_id, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(repo_id, device_map="auto", torch_dtype="auto")

# Example input
input_text = "Hello, how are you today?"
inputs = tokenizer(input_text, return_tensors="pt")

# Generate output
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Model Details

Architecture: LLaMA-style causal language model (LlamaForCausalLM)
- Hidden size: 2048
- Number of layers: 22
- Attention heads: 32
- MLP (feedforward) size: 5632
- Head dimension: 64
- Vocabulary size: 32,000 tokens
- Activation function: SiLU
- Max sequence length: 2048 tokens
- Precision: float16
- Multi-query attention: 4 key/value heads
Fine-tuning dataset: Specialized for chat and text generation tasks.
Optimizations: Model pruning applied for smaller footprint without significant loss in quality.
Intended Use: Research, personal projects, and local AI applications.

Notes

This model is privacy-preserving and can run locally on your machine.
Make sure to use a compatible PyTorch version to avoid runtime errors.
chat_template.jinja can be used to structure prompts if you are building a chat application.

Chat Usage Example

This example shows how to use chat_template.jinja to structure prompts and generate responses with the model.

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from jinja2 import Template
import torch

# Load the model and tokenizer
repo_id = "DemonC/ZenBot"  
tokenizer = AutoTokenizer.from_pretrained(repo_id, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(repo_id, device_map="auto", torch_dtype=torch.float16)

# Load and render the chat template
with open("chat_template.jinja", "r") as f:
    template_content = f.read()
template = Template(template_content)

# Example system prompt and user input
system_prompt = "You are a friendly assistant that helps with sleep and stress."
user_input = "I couldn't sleep well last night. Any tips?"

# Render the final prompt
final_prompt = template.render(system=system_prompt, user_input=user_input)
print("Prompt to model:\n", final_prompt)

# Tokenize and generate response
inputs = tokenizer(final_prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    pad_token_id=tokenizer.eos_token_id
)

# Decode and print the model output
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print("\nModel Response:\n", response)

Example Output: You are a friendly assistant that helps with sleep and stress. User: I couldn't sleep well last night. Any tips? Assistant: Try to avoid caffeine in the evening, keep your bedroom dark and quiet, and establish a consistent sleep routine. Deep breathing or light stretching before bed can also help you relax.