Views
No views yet
nn.Transformer or
nn.MultiheadAttention), trained to decrypt binary cipher text into English plaintext.| Axis | Setting |
|---|---|
| Positional encoding | sinusoidal |
| Attention | mha |
| Normalization | layernorm |
| Tokenization | byte |
d_model 256 | 4 encoder layers | 4 decoder layers
8 heads | d_ff 1024 | dropout 0.1 | pre-norm
AdamW lr 0.0003 | warmup 1000 | 15000 steps | batch 64 | seed 42
fp32 (no AMP)| Metric | Value |
|---|---|
| Bit-level accuracy | 0.98090 |
| Sequence accuracy | 0.6806 |
| Levenshtein distance | 1.21 |
| BLEU | N/A |
| ROUGE-L | N/A |
| Parameters | 10,389,504 |
| Peak GPU memory (MB) | 977.8 |
| Seconds per training step | 0.0862 |
torch.load(...)["model"] loads into build_model(get_config("c5"), ...).