You can deploy this model and start earning money today!

Emperordzd/
Nexara-Nano-100M-Chat

Name: Nexara-Nano-100M-Chat
Rating: 2.5 (12 reviews)

roneneldan/TinyStories

mit

Views

No views yet

Nexara-Nano-100M-Chat

Nexara-Nano-100M-Chat is a tiny Decoder-Only Transformer pretrained and supervised fine-tuned (SFT) from scratch. It is designed solely for experimental and educational purposes.

[!WARNING] This model has only ~110,138,880 parameters and is trained on a small corpus. It is not guaranteed to perform well in production environments. Expect frequent hallucinations and limited factual knowledge.

Model Architecture

Type: Causal Decoder-Only Transformer (with Rotary Position Embeddings (RoPE), Pre-LN, Tied Embeddings, and GELU activation).
Parameters: ~110,138,880
Layers: 12
Attention Heads: 12
Embedding Dimension: 768
Context Length: 512 tokens
License: MIT License

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("your_username/Nexara-Nano-100M-Chat", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("your_username/Nexara-Nano-100M-Chat", trust_remote_code=True)

# Format SFT prompt template
prompt = "### System:\nYou are Nexara, a helpful and polite AI assistant.\n\n### User:\nWhat is your name?\n\n### Assistant:\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=64, temperature=0.1)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Emperordzd/Nexara-Nano-100M-Chat

Nexara-Nano-100M-Chat

Model Architecture

Usage

Emperordzd/
Nexara-Nano-100M-Chat