metadata/run_config.json: training run configuration, if available
Training data
This model was trained on:
Ethosoft/nedo-turkish-65k-tokenized-60b
That dataset is a tokenized Turkish web-corpus snapshot containing approximately 60.95B uint16 tokens.
Training notes
This checkpoint corresponds to the stable 4xH200 base-pretraining run.
Recorded final evaluation:
step: 5000
val_loss: 1.5961
ppl: 4.93
A later 8xH200 continued-pretraining experiment exists in the project history, but the best logged evaluation checkpoint from that run was not saved as a file. Therefore this repository releases the stable saved base-pretraining checkpoint.
Recommended use
This checkpoint is best used for:
continued Turkish pretraining
supervised fine-tuning experiments
Turkish small language model research
tokenizer/model ablation studies
reproducibility of the NEDO Turkish SLM pipeline
Not intended as
a production assistant model
a safety-aligned chatbot
a Transformers-native checkpoint
a fully benchmarked general-purpose model
Known limitations
Not HF Transformers-compatible yet
No production KV-cache inference wrapper
Instruction-following is weak in the base model
Can repeat under open-ended prompting
Systematic benchmark evaluation is still incomplete
Citation and attribution
If you use this model, please attribute:
NEDO Turkish SLM project
NEDO Turkish 65K Tokenized Web Corpus
NEDO Turkish Tokenizer
FineWeb / FineWeb-style upstream data sources where applicable
Suggested attribution:
NEDOQwen 0.8B Base Pretrained.
Turkish decoder-only SLM trained with the NEDO Turkish 65K tokenizer
on the NEDO Turkish 65K Tokenized Web Corpus.
