Opt.Gear-1M

[!Note] This repository contains model weights for Opt.Gear-1M, a Tiny Language Model (TLM) designed to run on Micro-Controller Units (MCUs), together with artifacts for embedded deployment.

Opt.Gear-1M is not a general-purpose chatbot. It is trained for a narrow embedded-control setting — mapping short natural-language commands to structured board-control intents — and is intended for deeply embedded systems such as sensors, controllers, small robots, and offline human-machine interfaces.

For general-purpose on-device text generation, see Opt.Gear-270M and Opt.Gear-1B.

20 tokens/s on a 400MHz STM32 Cortex-M7 — about 50ms per token.

Opt.Gear-1M Highlights

• MCU-first design: every component — vocabulary, context length, KV cache, activation function, memory allocation — is scaled around Flash/SRAM limits and the absence of large matrix accelerators, not just the parameter count.
• Fixed-size decoding state: the ConvKV-Gated Mixer keeps a convolution-kernel-sized state (kernel 4) instead of a sequence-length-dependent cache, keeping SRAM usage predictable during auto-regressive decoding.
• Simple, accelerator-free operators: ReLU6GLU feed-forward and static memory allocation make every operation cheap to implement on a Cortex-M7 CPU.
• Compact tokenizer: a custom 2,048-token byte-level BPE keeps the embedding tables and runtime memory footprint compatible with flash-limited deployment.
• Self-contained embedded C runtime: runs without ONNX or X-CUBE-AI dependencies; footprint is measured from the final ELF, not theoretical weight size.

Model Overview

• Type: Causal Language Model (hybrid attention + convolutional mixer)
• Training Stage: task-focused board-control training on stm32_cmd_synth (no separate web-scale pre-training phase)
• Architecture
  • Number of Parameters: ~1M
  • Hidden Dimension: 128
  • Number of Layers: 5
  • Hidden Layout: hybrid of GQA and ConvKV-Gated Mixer
  • Grouped-Query Attention:
    • Number of Attention Heads: 2 for Q and 1 for KV
    • Head Dimension: 64
    • QK-Normalization: QK-LN
  • ConvKV-Gated Mixer:
    • Causal depthwise 1D convolution on key/value streams
    • Convolution Kernel Size: 4 (widened receptive field for the very small hidden dimension)
  • Feed-Forward Network:
    • Type: ReLU6GLU — ReLU6(x) = min(max(0, x), 6); bounded activation range, simple to implement without large accelerators
    • Intermediate Dimension: 272
  • Rotary Position Embedding: global theta 1,000,000
  • Tokenizer: custom byte-level BPE, vocabulary 2,048
  • Word Embedding: untied (separate input embedding and LM head)
• Context Length: 512 (deployment build uses a 160-token sequence length with a 40-token generation limit)

Board-Control Post-Training

Measured Performance (STM32H747I-DISCO)

• 20 tokens/s means generating the 40-token maximum takes about 2 seconds of decode time — fast enough for short command responses and local status generation, with token-by-token output observable in real time.
• .text/.rodata (≈1.5MB) is not the pure weight size: it includes the embedded runtime code, quantized weights, 16-bit embedding/LM head, lookup tables, scales, and constants. For MCU models, the final ELF — not the theoretical parameter count — is the meaningful footprint measure.
• Static RAM (≈449KB) is dominated by statically allocated activation, cache (GQA KV cache + ConvKV fixed-size state), intermediate buffers, logit/sampling buffers, and the reserved heap/stack.

Deployment

• Summarizing sensor data into short natural language
• Local command response for small robots
• Offline interfaces for industrial controllers
• Structured status message generation
• Simple Q&A on devices with limited network connectivity
• Short, domain-specific embedded assistants

Best Practices

1. Stay in the trained domain: The model is post-trained for short English board-control commands with JSON-style outputs. Out-of-domain prompts (open-ended questions, long-form generation, non-English input) will not produce reliable results.
2. Respect the deployment limits: The model configuration targets a 512-token maximum context, and the reference deployment build uses 160-token sequences with a 40-token generation cap. Longer sequences increase the SRAM-resident cache and workspace.
3. Budget by ELF, not parameter count: When adapting the runtime or retraining the model, verify Flash/SRAM budgets with arm-none-eabi-size on the final ELF — runtime code, lookup tables, and scales share the Flash with the weights.
4. Predictability over coverage: On embedded systems, predictable memory use and stable interactive latency matter more than long-context benchmark performance. The fixed-size ConvKV state exists precisely to keep decoding state independent of context length.

Limitations

Citation