You can deploy this model and start earning money today!

Views

No views yet

TinyBrain-100M Instruct

Name: tinybrain-100m-instruct
Availability: InStock

A 103M parameter experimental chat/instruct model fine-tuned from TinyBrain-100M Base.

TinyBrain-100M Instruct is a small instruction-tuned causal language model fine-tuned from exnivo/tinybrain-100m-base using exnivo/tinybrain-instruct-sft-200k.

This is a very small instruct model. It can answer simple prompts, explain basic ideas, give short plans, and sometimes show uncertainty behavior, but it is not a reliable general assistant. It may hallucinate, repeat text, fail at math, produce broken completions, or misunderstand prompts.

TinyBrain-100M Instruct was fine-tuned with a simple User/Assistant style format and no system prompt.

Quick Start

python

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "exnivo/tinybrain-100m-instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "User: Explain photosynthesis in simple words.\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=80,
    do_sample=False,
    repetition_penalty=1.15,
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.eos_token_id,
)

text = tokenizer.decode(outputs[0], skip_special_tokens=True)

answer = text.split("Assistant:", 1)[-1].strip()
answer = answer.split("\nUser:", 1)[0].strip()

print(answer)

At a Glance

Item	Details
Model type	Instruction-tuned causal language model
Parameters	103,385,856
Approx. size	103.4M
Architecture	LLaMA-style causal transformer
Language	English
Context length	2048 tokens
Vocabulary size	24,000
Tokenizer	Custom TinyBrain tokenizer
Base model	`exnivo/tinybrain-100m-base`
SFT dataset	`exnivo/tinybrain-instruct-sft-200k`
Prompt style	`User:` / `Assistant:`
System prompt	Not used during fine-tuning

Model Details

Item	Value
Parameters	103.4M
Architecture	`llama` / `LlamaForCausalLM`
Vocabulary size	24,000
Context length	2048 tokens
Hidden size	768
Intermediate size	2048
Layers	12
Attention heads	12
Key/value heads	12
Activation	SiLU
RMS norm epsilon	`1e-05`
Tied embeddings	true
BOS token	`<
EOS token	`<
PAD token	`<
Base model	`exnivo/tinybrain-100m-base`
SFT dataset	`exnivo/tinybrain-instruct-sft-200k`

Prompt Format

TinyBrain-100M Instruct was fine-tuned without a system prompt.

Use this simple format:

text

User: Your message here
Assistant:

Example:

text

User: Explain photosynthesis in simple words.
Assistant:

For best results:

keep prompts short and direct
do not use a system prompt
use short generation lengths
prefer greedy or low-temperature generation
stop/cut the output if it starts a new User: turn

Recommended Generation Settings

For stable short answers:

python

outputs = model.generate(
    **inputs,
    max_new_tokens=80,
    do_sample=False,
    repetition_penalty=1.15,
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.eos_token_id,
)

For slightly more varied answers:

python

outputs = model.generate(
    **inputs,
    max_new_tokens=80,
    do_sample=True,
    temperature=0.5,
    top_p=0.85,
    repetition_penalty=1.15,
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.eos_token_id,
)

For a very small model like this, long generations often become repetitive or unstable. Short completions usually work better.

Example Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "exnivo/tinybrain-100m-instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

def ask(message, max_new_tokens=80):
    prompt = f"User: {message}\nAssistant:"
    inputs = tokenizer(prompt, return_tensors="pt")

    outputs = model.generate(
        **inputs,
        max_new_tokens=max_new_tokens,
        do_sample=False,
        repetition_penalty=1.15,
        no_repeat_ngram_size=3,
        pad_token_id=tokenizer.eos_token_id,
    )

    text = tokenizer.decode(outputs[0], skip_special_tokens=True)
    answer = text.split("Assistant:", 1)[-1].strip()
    answer = answer.split("\nUser:", 1)[0].strip()
    return answer

print(ask("Explain gravity in simple words."))

Training Data

TinyBrain-100M Instruct was fine-tuned on:

exnivo/tinybrain-instruct-sft-200k

The SFT dataset contains 196,668 rows of English instruction/chat examples focused on short, learnable assistant behavior.

Dataset categories include:

Category	Rows	Percent
`source_grounded_education_factual`	49,882	25.36%
`math_reasoning`	37,611	19.12%
`clean_conversation`	34,257	17.42%
`messy_idea_to_plan`	29,978	15.24%
`simplify_explain`	19,990	10.16%
`honesty_uncertainty`	14,957	7.61%
`simple_coding`	9,993	5.08%

The dataset was designed for small models and uses short assistant responses across education, basic math, planning, simplification, simple coding, clean conversation, and uncertainty behavior.

Relationship to TinyBrain

TinyBrain is a small LLM project focused on compact datasets, small base models, and instruction-tuned models.

Stage	Repository	Purpose
Pretraining corpus	`exnivo/tinybrain-pretrain-corpus-2b`	Base language model training data
Base model	`exnivo/tinybrain-100m-base`	Small causal LM trained from scratch
SFT dataset	`exnivo/tinybrain-instruct-sft-200k`	Instruction/chat fine-tuning data
Instruct model	`exnivo/tinybrain-100m-instruct`	Chat/instruct model fine-tuned from the base model

Pipeline:

text

TinyBrain Pretrain Corpus 2B
        ↓
TinyBrain-100M Base
        ↓
TinyBrain Instruct 200K
        ↓
TinyBrain-100M Instruct

Expected Behavior

TinyBrain-100M Instruct can sometimes handle:

simple explanations
short educational questions
basic planning
simple rewriting/simplification
simple coding prompts
some uncertainty/refusal-style prompts
casual assistant-like responses

Example prompt:

text

User: Explain photosynthesis in simple words.
Assistant:

Possible output style:

Photosynthesis is how plants make their own food using sunlight, water, and air. They turn these into sugar and oxygen.

Because this is a very small model, outputs may be unstable. It can repeat, drift, or produce broken text, especially with long generations or higher sampling temperatures.

Evaluation Notes

A lightweight local report confirmed the model architecture and parameter count:

Metric	Value
Total parameters	103,385,856
Trainable parameters	103,385,856
Approx. parameters	103.4M

Manual prompt tests showed that the model behaves better with the plain User: / Assistant: format than with custom chat-special-token formatting.

The model can produce useful short answers for some prompts, but it still performs poorly on reliable math, longer reasoning, and some instruction-following tasks.

This should be treated as an experimental small-model checkpoint, not a benchmark-grade assistant.

Intended Use

TinyBrain-100M Instruct is intended for:

small-model experiments
local lightweight assistant tests
instruction-tuning research
comparing base vs instruct behavior
educational model experiments
studying tiny LLM limitations
continued fine-tuning
dataset/model pipeline demos

This model is useful for exploring how much instruction-following behavior can be added to a small 100M-parameter model.

Not Intended For

Do not rely on this model for:

medical advice
legal advice
financial advice
emergency decisions
safety-critical systems
factual authority
current news or live information
advanced math
advanced coding
long-form reasoning
production assistant use without further training and evaluation

This is an experimental model and should not be used as a source of truth.

Strengths

TinyBrain-100M Instruct is useful because it is:

small
lightweight
easy to run locally
fine-tuned from a matching TinyBrain base model
trained on a public TinyBrain SFT dataset
designed for short assistant-style responses
useful for base-vs-instruct comparison
good for studying tiny model behavior

Limitations

TinyBrain-100M Instruct has major limitations.

The model may:

hallucinate facts
fail simple math
repeat words or phrases
produce broken text
drift off-topic
answer too briefly
misunderstand prompts
generate unreliable code
fail at longer reasoning
fail refusal or safety behavior
continue into fake new user turns

For best results, keep prompts short and use short generation lengths.

Known Weaknesses

Based on local testing, this model is especially weak at:

reliable arithmetic
robust coding
long answers
multi-step reasoning
clean formatting
high-temperature sampling
long context use

It may answer simple educational prompts better than math or code prompts.

Suggested Evaluation

Recommended checks:

short factual prompts
simple explanation prompts
basic math prompts
correction prompts
refusal/uncertainty prompts
repetition tests
prompt-format tests
base vs instruct comparison
SFT dataset overfitting checks
generation temperature sensitivity

Example prompts:

text

User: Explain gravity in simple words.
Assistant:

text

User: What is 17 + 25?
Assistant:

text

User: What will the weather be tomorrow in my city?
Assistant:

text

User: Give me 3 quick tips to keep my room tidy.
Assistant:

text

User: Write a simple Python function that reverses a string.
Assistant:

Training

TinyBrain-100M Instruct was fine-tuned from:

exnivo/tinybrain-100m-base

using:

exnivo/tinybrain-instruct-sft-200k

The base model was trained from scratch on exnivo/tinybrain-pretrain-corpus-2b.

Citation

If you use this model, you can cite it as:

bibtex

@misc{tinybrain_100m_instruct,
  title = {TinyBrain-100M Instruct},
  author = {exnivo},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/exnivo/tinybrain-100m-instruct}}
}

Related Repositories

Pretraining corpus: exnivo/tinybrain-pretrain-corpus-2b
Base model: exnivo/tinybrain-100m-base
SFT dataset: exnivo/tinybrain-instruct-sft-200k
Instruct model: exnivo/tinybrain-100m-instruct

License

This model is released under the Apache 2.0 license.

The SFT dataset and pretraining corpus are mixed-source datasets and may have their own licensing considerations. Users should review the dataset cards and upstream source metadata before commercial use.

Disclaimer

TinyBrain-100M Instruct is an experimental tiny instruction-tuned language model. It may produce incorrect, biased, unsafe, nonsensical, or misleading outputs.

Do not use this model for high-stakes decisions or as a reliable source of factual information.