Property	Value
Base model	IDK-1 (pre-trained, step 25k)
Parameters	106.24M
Architecture	LLaMA-style decoder-only transformer
Vocab size	40,002 (40k BPE + 2 special tokens)
Context length	512 tokens
Language	Indonesian (Bahasa Indonesia)
License	Apache 2.0

Architecture Config

dim       = 768
n_layers  = 12
n_heads   = 12
n_kv_heads = 4   (GQA)
ffn_dim   = 2048
RoPE theta = 500,000
logit_cap  = 30.0 (Gemma 2 style soft-capping)

Training

SFT Data

4,810 instruction pairs in ChatML format
Topics: factual Indonesian Q&A, summarization, ELI5 explanations, practical tips, conversations, count-following tasks
Format:

json

{"messages": [
  {"role": "user", "content": "..."},
  {"role": "assistant", "content": "..."}
]}

SFT Rounds

Round	Base	Data	LR	Epochs	Best Val
v1	IDK-1 step 25k	1,390 pairs	2e-5	3	3.0506
v2	IDK-1 step 25k	3,010 pairs	3e-5	5	2.1709
v3	sft_best v2	3,810 pairs	1e-5	3	2.0808
v4	sft_best v3	4,810 pairs	5e-6	3	1.3670

Training was done on Kaggle (T4 GPU) using PyTorch with loss masking on non-assistant tokens.

Special Tokens

<|im_start|>  → id 40000
<|im_end|>    → id 40001

Usage

python

import torch
from tokenizers import Tokenizer

# Load tokenizer
tokenizer = Tokenizer.from_file("tokenizer.json")
im_start = tokenizer.token_to_id("<|im_start|>")
im_end   = tokenizer.token_to_id("<|im_end|>")

def build_prompt(user_message):
    return f"<|im_start|>user\n{user_message}<|im_end|>\n<|im_start|>assistant\n"

# Load model (see IDK-1 repo for model definition)
# model = IDK1Model(IDK1Config())
# ckpt = torch.load("sft_best.pt", map_location="cpu")
# model.load_state_dict(ckpt["model"])

prompt = build_prompt("Jelaskan apa itu kecerdasan buatan dalam 3 poin.")

Limitations

Open-ended reasoning — complex topics may drift or produce incoherent output. Root cause: noisy CulturaX pre-training data + 100M param ceiling.
Knowledge cutoff — pre-trained on Wikipedia ID + CulturaX ID snapshots. No real-time knowledge.
Context length — max 512 tokens. Not suitable for long-document tasks.
Language — optimized for Indonesian. English or mixed-language prompts may degrade quality.
Not for production — this is a research/learning project. Do not use for medical, legal, or safety-critical applications.

What Works Well

✅ Count-following instructions ("Sebutkan 3 hal tentang...")
✅ Short factual Q&A in Indonesian
✅ Simple summarization
✅ Practical tips and how-to explanations
✅ Basic conversational responses

Project

IDK-1 was built as a learning + portfolio project to demonstrate training an Indonesian SLM from scratch on commodity hardware (Kaggle free tier).

GitHub: github.com/ripkiiii/IDK-1
Blog: deflated.xyz
Pre-trained base: idk-ai/IDK-1
SFT dataset: idk-ai/IDK-1-Instruct-Data

Citation

bibtex

@misc{idk1instruct2026,
  title  = {IDK-1-Instruct: Instruction-tuned Indonesian Small Language Model},
  author = {Muhammad Rifky Firmansyah Sujana},
  year   = {2026},
  url    = {https://huggingface.co/idk-ai/IDK-1-Instruct}
}