Released as part of the NOESIS Professional Multilingual Dubbing Automation Platform (framework: DHCF-FNO — Deterministic Hybrid Control Framework for Frozen Neural Operators).
An experimental Kazakh continued-pretraining checkpoint based on inference-optimization/Kimi-K3-0.40B.
This is an early smoke-test checkpoint created to validate a complete Kazakh language-model pretraining pipeline on Kaggle.
Important limitation
The base checkpoint is a tiny architecture-development version of Kimi K3. Its weights were initialized from scratch and trained only on a toy copypasta dataset. It is not a distilled or compressed version of the trained 2.8T Kimi K3 model.
This checkpoint is therefore an educational research artifact, not a production language model.
Tokenization: native Kimi tokenizer accelerated with Gigatoken
Results
Metric
Before
After
Validation loss
16.7615
4.9539
Validation perplexity
19,030,254
141.72
Example generations
The model has started learning Kazakh Cyrillic, morphology, punctuation, and sentence structure, but its generations remain semantically incoherent and contain invented words. This is expected after only 1.64M training tokens.
For reproducible comparisons, use a fixed random seed and report the prompt, decoding parameters, Transformers version, and hardware. Because this is an intermediate checkpoint, outputs should be treated as experimental and inspected manually.
Intended use
This checkpoint is intended for studying Kazakh language adaptation, inspecting Kimi K3's small architecture-development configuration, testing dataset and pretraining pipelines, and comparing intermediate language-acquisition checkpoints.
It should not be used for factual, safety-critical, medical, legal, financial, or production tasks. The checkpoint has not been instruction-tuned or safety-aligned for general deployment.
License
This repository is released under the MIT License. Review the base model and dataset terms before redistribution or commercial use.
