Views
No views yet
| Model | Params | Vocab | Context | Val Loss | Notes |
|---|---|---|---|---|---|
| MicroJulia | 4,992 | 27 chars | 64 | 2.43 | First proof-of-concept |
| JuliaGPT | 8,096 | 29 chars | 256 | 2.34 | Expanded context + vocab |
| JuliaGPT-v2 | ~10M | 38 chars | 256 | 2.91 | Scaled-up char-level |
| Parameter | Value |
|---|---|
| Architecture | 1-layer Transformer (pure Julia, scalar autograd) |
| Parameters | 8,096 |
| Embedding dim | 16 |
| Layers | 1 |
| Attention heads | 4 |
| Head dim | 4 |
| FFN hidden dim | 64 |
| Context length | 256 characters |
| Vocabulary | 29 characters (a-z, space, period, + BOS) |
.abcdefghijklmnopqrstuvwxyz + BOS| Value | |
|---|---|
| Dataset | Aristotle's Rhetoric + Euclid's Elements (8,461 chunks) |
| Best val loss | 2.34 |
| Framework | Pure Julia (scalar autograd, no Flux/Lux) |
| File | Description |
|---|---|
best_model.json | Original model weights + optimizer state (JSON format, scalar autograd) |
vocab.json | 38-character vocabulary array |
data/aristotle_rhetoric.txt | Training data |
.jld2 checkpoint files in this repo contain a different, larger model (384d/6L/38vocab). That model has been moved to JuliaGPT-v2. The original JuliaGPT is preserved in best_model.json.| Parameter | Value |
|---|---|
| vocab_size | 29 |
| context_length | 256 |