paLLM TinyStories for Palm OS
paLLM is a TinyStories-class transformer language model packaged for local
inference on a Palm Tungsten E2. The model is embedded into a Palm OS
application as row-wise INT8 resources and generated entirely on-device after
installation.
This is a research/retro-computing artifact, not a general assistant model.
Generation on the tested Palm Tungsten E2 is roughly one token every seven
seconds, and output quality is intentionally limited by the tiny model size,
fixed-point inference, and Palm-safe approximations.
Contents
model/q8_chunk_*.bin: row-wise INT8 model payload split into Palm resource-sized chunks.model/q8_index.h: chunk index used by the Palm runtime.model/q8_resources.rcp: Palm resource declarations for the model chunks.model/palm_vocab.h: 512-token TinyStories tokenizer table compiled into the app.palm/: Palm OS C runtime and resources.source/Makefile: build entry point from the GitHub repository.
The embedded model payload is 282,584 bytes across 18 chunks. The original
project builds a .prc of about 292 KiB.
Model Lineage
This package is based on the tiny Llama/TinyStories ecosystem:
- Andrej Karpathy's
llama2.c
karpathy/tinyllamas, especially the TinyStories 260K family- The TinyStories dataset/paper by Ronen Eldan and Yuanzhi Li
The Palm build is not a direct llama2.c port. It adds a custom quantizer,
Palm resource packing, a fixed-point runtime, and decoder guardrails for a
constrained Palm OS environment.
Runtime
Target hardware:
- Palm Tungsten E2
- Palm OS Garnet 5.4.x
- 200 MHz Intel XScale-class CPU
- No hardware floating point unit
- Small dynamic heap and limited non-volatile storage
Runtime characteristics:
- Q12 fixed-point activations
- row-wise Q8 weights
- Palm resource chunk loading
- simplified sigmoid
- recency-based attention approximation rather than full softmax
- greedy decoding with small readability guardrails
Use
The full build flow lives in the GitHub project:
1git clone https://github.com/maddiedreese/paLLM.git
2cd paLLM
3make
The GitHub Actions workflow builds the Palm .prc using a Palm OS build
package, m68k-none-elf-gcc, PilRC, and Palm OS 5 SDK headers.
Limitations
- Output quality is fragile and prompt-dependent.
- The attention path is an approximation, not faithful transformer softmax.
- The decoder includes hand-written guardrails.
- This is tuned for a tiny TinyStories-style model, not assistant behavior.
- Build reproducibility depends on the Palm build package used by the GitHub workflow.
Citation and Links
