PKaI Nano 1
PKaI Nano 1 is a 120M-class base language model from PowderKeg
Intelligence. It is a compact LLaMA-style decoder model trained from
scratch with a Mistral tokenizer and released as a PKaI-native artifact.
This repository contains PKaI-native weights and metadata, not a drop-in
transformers.AutoModelForCausalLM package.
Files
model.safetensors: PKaI base model weights.config.json: PKaI model architecture configuration.tokenizer.json: PKaI tokenizer metadata.tokenizer.model: SentencePiece tokenizer model from mistralai/Mistral-7B-v0.1.THIRD_PARTY_NOTICES.txt: tokenizer and training-data provenance notices.LICENSE: Apache License, Version 2.0.
Architecture
- Parameters: 112,680,448
- Vocabulary size: 32,000
- Context length: 512
- Layers: 14
- Attention heads: 12
- KV heads: 4
- Embedding size: 768
- Tied embeddings: yes
- QK normalization: yes
Training Data
Publicly disclosed training data sources include:
HuggingFaceFW/fineweb-edu, released under the Open Data Commons Attribution
License (ODC-By) v1.0 and subject to Common Crawl Terms of Use as noted by the
dataset card.HuggingFaceTB/cosmopedia, released under the Apache License, Version 2.0.
See THIRD_PARTY_NOTICES.txt for source URLs, citations, and attribution notes.
License
Copyright 2026 PowderKeg Intelligence LLC.
The PKaI Nano 1 model artifact is released under the Apache License, Version
2.0. The bundled tokenizer and training-data sources have their own provenance
and notices listed in THIRD_PARTY_NOTICES.txt.
Limitations
This is a small base model and has not been instruction-tuned or safety-tuned.
It may produce inaccurate, unsafe, biased, or otherwise unsuitable text. Users
are responsible for evaluating fitness, safety, and legal compliance for their
own use cases.
