NaviKinase 1.0
A from-scratch, decoder-only protein language model for the
phosphotransferase superfamily (EC 2.7.-: protein kinases plus
sugar/lipid/nucleotide kinases), trained entirely locally on Apple Silicon
via
MLX — no cloud compute, no
fine-tuning of an existing model.
Model details
| |
|---|
| Parameters | 149,604,416 (149.6M) |
| Architecture | Decoder-only transformer, pre-norm RMSNorm, RoPE, tied embeddings |
| Layers | 18 |
| d_model | 832 |
| Attention heads | 13 (head_dim 64) |
| FFN dim | 3328 |
| Context length | 1024 tokens |
| Vocab size | 64 reserved (45 used) |
| Precision | fp32 |
| Framework | MLX |
Training data
- Scope: UniProtKB entries with EC 2.7.- (phosphotransferase superfamily),
length 30-2000 residues, non-fragment.
- Deduplicated at the UniRef90 cluster level (one representative sequence
per cluster) to prevent overfitting to near-identical bacterial strain
variants; train/val/test split at the cluster level to keep held-out
evaluation meaningful.
- 2,450,504 training sequences, 1,320,176,103 training tokens.
- Each example is packed as
<BOS> <EC-tag> residues <EOS>, where the
EC-tag conditions generation on one of 14 EC 2.7.- sub-subclasses (e.g.
<EC_2.7.11> for protein-serine/threonine kinases) or <EC_UNKNOWN> for
sequences without a recognized sub-subclass (~3.3% of the corpus).
Training
- AdamW (betas 0.9/0.95, weight decay 0.1), cosine LR schedule with warmup,
peak LR 3e-4.
- Effective batch size 64 sequences/step (batch 4 × grad-accum 16),
1024-token context.
- Ran the full 40,289-step ceiling (2 epochs) to completion — no early
stopping triggered (patience=5 eval intervals, val loss kept improving
every eval).
- Trained unattended over several days on a MacBook, with automatic
checkpointing, battery-aware pause/resume, and crash recovery via
launchd.
Evaluation
| Metric | Value |
|---|
| Best validation loss (step 40,000) | 1.3809 |
| Best validation perplexity | 3.98 |
| Held-out test loss | 1.4633 |
| Held-out test perplexity | 4.32 |
The test-set number is the one to cite — it was never touched during
training or checkpoint selection (validation was used for the latter, so it
carries a small optimistic bias).
Motif presence (qualitative, not a hard benchmark)
50 unconditioned samples were scanned for canonical kinase catalytic motifs
(glycine-rich P-loop, catalytic HRD, activation-loop DFG). These are
protein-kinase-specific motifs, so hit rates are naturally lower on
unconditioned samples than on samples conditioned on a kinase EC tag (e.g.
--ec-class 2.7.11):
| Motif | Hit rate (unconditioned, n=50) |
|---|
| P-loop (GxGxxG) | 28.0% |
| Catalytic HRD | 6.0% |
| Activation DFG | 12.0% |
Usage
This model speaks amino acid sequences (FASTA-style), not English — there is
no chat template or instruction-following. Three ways to use it:
Generate from scratch, or continue a real sequence:
1pip install mlx
2python generate.py --n 10 --max-len 400
3python generate.py --n 5 --ec-class 2.7.11 --temperature 0.8 # protein-Ser/Thr kinases
4python generate.py --n 3 --prefix MKTAYIAKQRQISFVKSHFSRQ # continue a real sequence
Score how well a real sequence fits what the model learned (teacher-forced
loss/perplexity over your sequence — a plausibility signal, not a
classifier or structural prediction):
1python score.py --sequence MKTAYIAKQRQISFVKSHFSRQ...
2python score.py --fasta my_protein.fasta --ec-class 2.7.11
3python score.py --sequence MKTAY... --per-position # flag atypical stretches
4python score.py --sequence MKTAY... --classify # EXPERIMENTAL, see script docstring
Plain-English front end (pattern matching over the tools above —
not a language model; NaviKinase itself has zero English understanding.
See chat.py's docstring for exactly which request phrasings it
recognizes, and what it does when a request doesn't match one). In the
interactive REPL, it asks a clarifying question rather than guessing
whenever the answer would actually change the output — an ambiguous
"kinase" with no subtype specified when generating from scratch, or
multiple UniProt name-search matches — but proceeds directly when a real
sequence (pasted or fetched) already anchors the result, since asking
there would just be friction. In single-shot --query mode there's no one
to ask, so it reports what it needs instead of guessing:
1python chat.py
2python chat.py --query "how plausible is this sequence: MKTAYIAKQRQISFVKSHFSRQ"
3python chat.py --query "generate a serine/threonine kinase"
4python chat.py --query "fetch protein P00519 and continue it" # live UniProt lookup
5python chat.py --query "search pubmed for kinase inhibitor resistance"
6python chat.py --query "research protein P00519" # multi-step, see below
Multi-step research pipeline (research.py) — "research protein X"
chains fetch → score → EXPERIMENTAL classify → motif scan → PubMed search
into one synthesized report, autonomously (you don't specify each step).
The step sequence is fixed, not dynamically planned — this is the one
useful compound goal for this model's narrow domain. Real example (human
ABL1, P00519): correctly fetched, scored as typical (perplexity 1.64), the
EXPERIMENTAL classifier's top pick (EC 2.7.10, tyrosine kinases) matches
ABL1's real annotation, and the motif scan found all three canonical
kinase motifs — a genuine, coherent result, though the classify step
remains an unvalidated proxy same as everywhere else in this repo.
Live lookups (lookup.py) — unlike everything else here, these hit
real, public APIs (UniProt, NCBI PubMed) and need internet access:
fetch_uniprot_by_accession(accession) — real protein sequence by
exact accession, unambiguous.
fetch_uniprot_candidates(query, n) — best-effort free-text name/keyword
search, returns up to n real candidates (accession, name, organism,
sequence) for disambiguation — verified against the live API to not
reliably rank the intended protein first for ambiguous multi-word
queries, which is why callers should show candidates rather than assume
the top hit.
fetch_pubmed_abstracts(query, n) — real citations/abstracts for a
keyword search. Informational only — displayed as plain text for you
to read; NaviKinase cannot process or reason about this text, it is never
passed into the model.
1from model import KinaseGPT, ModelConfig
2from tokenizer import BOS_ID, EOS_ID, ec_field_to_tag_id, decode
3import mlx.core as mx
4
5cfg = ModelConfig.from_json("config.json")
6model = KinaseGPT(cfg)
7model.load_weights("model.safetensors")
8model.eval()
Real 3D structure prediction (
structure.py /
fold.py) — a genuinely
different model, not NaviKinase. NaviKinase was trained only on 1D sequence
text and has no representation for 3D coordinates at all;
fold.py calls
ESMFold (Meta's public structure-prediction
API,
api.esmatlas.com) on a sequence — one NaviKinase generated, or a real
one fetched from UniProt — and returns what
that service predicts. Needs
internet access. Confirmed by live testing: this free service is real but
genuinely flaky (a request can 504 and succeed on retry seconds later,
independent of sequence length), so
fold_sequence() retries with backoff
by default — this is required for reliable use, not optional robustness:
1python fold.py --sequence MKTAYIAKQRQISFVKSHFSRQ... --out structure.pdb
2python fold.py --fasta my_protein.fasta --out structure.pdb
3python chat.py --query "fold this sequence: MKTAYIAKQRQISFVKSHFSRQ..."
Output includes a mean per-atom confidence score (0-1 scale, ESMFold's
reuse of the PDB B-factor column for a pLDDT-style estimate — higher is
more confident). The resulting PDB structure data is Meta Platforms, Inc.,
licensed CC-BY-4.0 (see the REMARK lines in any returned PDB file) —
separate from and unrelated to this repo's own CC BY-NC 4.0 license on the
NaviKinase model itself; only the model weights/code here are
non-commercial-restricted, not ESMFold's output.
Files
model.safetensors — model weights (fp32)
model.py — standalone MLX architecture definition
tokenizer.py — frozen tokenizer/vocab for this specific checkpoint
config.json — architecture hyperparameters
generate.py — generation / sequence-continuation script
score.py — plausibility scoring for a real sequence
motifs.py — regex catalytic-motif scan (P-loop, HRD, DFG)
chat.py — plain-English pattern-matching front end over all the tools here
nlu.py — shared request-parsing/clarification helpers (used by chat.py and research.py)
research.py — multi-step autonomous "research protein X" pipeline
lookup.py — live UniProt sequence fetch + PubMed citation search (needs internet)
structure.py — real 3D structure prediction via ESMFold (needs internet; a different model, not NaviKinase)
fold.py — standalone CLI for structure.py
Important: use the bundled tokenizer.py, not a shared/updated one
This repo ships its own frozen copy of the tokenizer. If you've seen a
later, broader version of this project (e.g. a follow-up model covering the
full EC 2.- superfamily), do not use that tokenizer with this
checkpoint — its EC-tag vocabulary was extended and <EC_UNKNOWN>'s token
ID shifted as a result. The tokenizer.py in this repo is the exact,
correct vocabulary this model was trained with (<EC_UNKNOWN> = token ID
44).
Responsible use / dual-use disclosure
This model generates protein sequences — real biomolecules, not text.
Anyone using generated output for wet-lab work bears full responsibility
for independently reviewing it before synthesis.
No built-in screening: this repo has no content moderation, no
toxin/hazard blocklist, and no automated biosecurity screening on
generated or scored sequences. It has not been reviewed by a biosecurity
expert. Every fresh generation from generate.py, chat.py, or the
webapp carries an automatic non-synthesis notice for this reason — that
notice is the extent of the safeguard, not a substitute for real review.
Why the realistic risk is limited (not eliminated): the model's entire
vocabulary and training scope is EC 2.7.- phosphotransferases (kinases) —
one of the most common, well-characterized enzyme families in biology,
present in nearly all organisms and not associated with toxicity or
pathogenicity. Its vocabulary is closed and EC-tag-conditioned, not
free-form English prompting, so there is no mechanism to prompt it into
generating a different, higher-risk protein class. That narrowness is a
real, structural constraint — but it was a consequence of data-availability
and model-size decisions (see Training data above), not an engineered
safety feature, so don't treat scope alone as a safety guarantee.
Before synthesizing any generated sequence:
- Treat it as an unvalidated hypothesis, not a confirmed real protein —
see Evaluation above for exactly what has and hasn't been measured.
- Route synthesis through a DNA synthesis provider that performs standard
biosecurity screening (most reputable providers already do this as a
matter of course).
- Independently review function and safety yourself; nothing in this repo
screens for toxicity, pathogenicity, or hazard.
Limitations
- Scope is EC 2.7.- specifically — it has no exposure to other transferase
classes, non-enzymatic proteins, or non-protein biomolecules.
- Character-level, single-sequence model: no structural, multiple-sequence-
alignment, or homology information is used, only raw sequence statistics.
- ~3.3% of training data has no recognized EC sub-subclass and was trained
under a shared
<EC_UNKNOWN> tag.
- Motif-presence checks are a cheap directional signal, not a validated
functional or structural correctness benchmark.
chat.py is keyword/regex pattern matching, not a language model — it
only recognizes the request styles listed in its docstring, and says so
explicitly rather than guessing when a request doesn't match. It does not
give NaviKinase any actual English understanding.
score.py --classify is an improvised proxy (comparing loss under each
EC tag) — the model was never trained as a classifier, so treat its
output as a rough signal, not a validated prediction.
lookup.py's UniProt name/keyword search (as opposed to exact accession
lookup) is best-effort free-text ranking, not guaranteed to return the
specific protein you meant for ambiguous or multi-word queries — always
check the accession it reports. fetch_pubmed_abstracts results are
shown to you verbatim and are never interpreted by the model. Both
require internet access, unlike the rest of this release, which is fully
offline.
research.py's pipeline runs a fixed sequence of steps (fetch, score,
classify, motif scan, literature search) — it is not a dynamic planner
that decides what to do based on reasoning about your request. The
classify step inherits the same "experimental, unvalidated" caveat as
score.py --classify above.
structure.py/fold.py call ESMFold, a separate third-party model — not
NaviKinase, and not covered by this repo's evaluation numbers above. It's a
free public service with no uptime guarantee and no length limit
enforced here; large sequences may be slow or fail independent of
NaviKinase itself. Requires internet access.
License
CC BY-NC 4.0 (Attribution-NonCommercial). Free to use, share, and adapt
for non-commercial purposes with attribution.
Commercial use — including
any use intended to generate revenue — requires prior written permission
from the author. See
LICENSE for full terms, or contact
jaydevkbhatt100@gmail.com for commercial licensing inquiries.