Tara 1.1 1M SFT is a tiny educational assistant model packed for Hugging Face. It is a 1M-class GPT-style causal language model trained as a compact learning version in the Tara family, inspired by the larger aungkomyint/tara10m-sft-v1-2k project.

This release intentionally reduces the scale from the 10.4M Tara10M SFT model to an 865K-parameter GPT-2-compatible model so it can be trained, inspected, shipped, and loaded easily on modest hardware.

This is a school/learning project model. It is not a production assistant.

Model Details

Name: Tara 1.1 1M SFT
Architecture: GPT-2-style decoder-only causal LM
Hugging Face class: GPT2LMHeadModel
Parameters: 865,344
Layers: 4
Hidden size: 96
Attention heads: 4
Vocabulary: 4,096 BPE tokens
Context length: 256 tokens
Tensor type: float32
Format: safetensors
License: Apache-2.0

Relationship To Tara10M

The reference Tara10M model is a 10.4M-parameter Llama-style Burmese-English SFT model with a 16K SentencePiece vocabulary and 1,024-token context. Tara 1.1 is not the same architecture and is not a drop-in replacement for Tara10M.

Tara 1.1 is the smaller educational branch:

much smaller parameter count
GPT-2-compatible architecture
English-focused project/domain knowledge
tuned around short assistant-style answers
designed for quick local experiments and classroom demonstration

Training Summary

Training was staged locally:

Continued base-model improvement on WikiText.
Domain base continuation on local Tara project knowledge in data/base_school.txt.
SFT on local instruction/chat data in data/sft_assistant.txt.
Multi-turn repair pass on data/multiturn_fix.txt.
Conservative final SFT refresh from tara-1m-assistant-v2 to preserve stronger assistant behavior while adding domain knowledge.

The final shipped checkpoint is:

outputs/tara-1m-assistant-v5-packed

Evaluation

Lower is better.

Model	Eval file	Loss	Perplexity
tara-1m-assistant-v2	`data/sft_assistant.txt`	3.0114	20.3157
Tara 1.1 1M SFT	`data/sft_assistant.txt`	2.1528	8.6091
tara-1m-assistant-v2	`data/multiturn_fix.txt`	4.0452	57.1240
Tara 1.1 1M SFT	`data/multiturn_fix.txt`	3.2565	25.9596

Intended Use

Best test areas:

explaining Tara 1M
explaining tokenizers, base models, SFT, and safetensors
short school-project assistant answers
simple multi-turn educational chat

Prompt format:

text

User: What is SFT?
Assistant:

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "aungkomyint/tara1.1"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

prompt = "User: What is SFT?\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=80,
    do_sample=False,
    pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Local folder usage:

python

from transformers import pipeline

pipe = pipeline("text-generation", model="./tara1.1-hf")
print(pipe("User: What is Tara 1M?\nAssistant:", max_new_tokens=80)[0]["generated_text"])

Example Outputs

Prompt:

text

User: What is SFT?
Assistant:

Output:

SFT means supervised fine tuning. It trains the model on examples of instructions and good answers.

Prompt:

text

User: How do I improve the model?
Assistant:

Output:

Use cleaner data, train longer, test outputs, then add better examples for weak areas.

Limitations

This model is intentionally tiny and should be treated as an experiment.

Known limitations:

weak factual reliability outside the training domain
repetition on some prompts
short context window
not robust for long-form generation
not suitable for safety-critical, legal, medical, or financial use

Files

model.safetensors
config.json
generation_config.json
tokenizer.json
tokenizer_config.json
special_tokens_map.json
vocab.json
merges.txt
chat_template.jinja