Views
No views yet
transformers)..safetensors + _meta.json), dibedakan lewat field backend: tensorflow sehingga tidak tertukar dengan checkpoint PyTorch.| Komponen | Nilai default |
|---|---|
| Tipe | Decoder-only transformer (pre-LN) |
| Layer / Head | 4 / 4 |
| Dimensi embedding | 128 |
| Konteks (block_size) | 128 token |
| Tokenizer | char (karakter) atau bpe (byte-pair encoding) |
| Attention | Multi-head causal (use_causal_mask) |
| Aktivasi | GELU (MLP) |
indigotf/
model.py build_gpt + generate (top-k, temperature)
common.py clean_text, meta save/load, build_tokenizer
bpe.py Tokenizer BPE level-byte
tokenizer.py Tokenizer karakter
train.py Training loop (split val, best-checkpoint, resume, cosine LR + warmup)
generate.py Inferensi dari checkpoint .safetensors
data/sample.txt Teks contoh Indonesia1pip install -r requirements.txt
2
3# Training
4python train.py --data data/sample.txt --steps 2000
5python train.py --data data/sample.txt --tokenizer bpe --vocab-size 512
6python train.py --init-from out/indigo_best.safetensors --steps 1000 # resume
7
8# Generate
9python generate.py --prompt "Indigo" --max-new 300
10python generate.py --ckpt out/indigo_best.safetensors --seed 42step yang akurat untuk resume), --init-from, tokenizer
char/bpe, cosine LR + warmup, serta pilihan --device.