Purple Squirrel R1 — Multichain LoRA Adapters
LoRA adapter weights for
Purple Squirrel R1 Multichain, fine-tuned on 58 conference sessions from
Wrapped Events covering cross-chain protocols, DeFi infrastructure, and Web3 technology.
Use these adapters to apply the multichain fine-tuning to the base model yourself, or continue training with your own data.
Adapter Details
| Property | Value |
|---|
| Base Model | DeepSeek-R1-Distill-Llama-8B (4-bit) |
| Method | LoRA (Low-Rank Adaptation) |
| Rank | 8 |
| Scale | 20.0 |
| Dropout | 0.0 |
| LoRA Layers | 4 |
| Trainable Params | 2.621M / 8,030M (0.033%) |
| Framework | MLX-LM 0.29.1 |
| Adapter Size | ~10 MB |
| Hardware | Apple M-series (16GB RAM) |
| Peak Memory | 6.184 GB |
Training Configuration
1framework: mlx-lm 0.29.1
2method: LoRA
3lora_layers: 4
4lora_rank: 8
5learning_rate: 1e-5
6batch_size: 1
7iterations: 200
8max_seq_length: 1024
9grad_checkpoint: true
10save_every: 100
11seed: 42
Training Curve
| Iteration | Train Loss | Val Loss | Improvement |
|---|
| 0 | — | 3.799 | baseline |
| 50 | 3.202 | 3.241 | -14.7% |
| 100 | 3.056 | 3.126 | -17.7% |
| 150 | 3.140 | 3.098 | -18.5% |
| 200 | 3.083 | 3.091 | -18.6% |
Files
├── adapters.safetensors # Final adapter weights (iteration 200)
├── adapter_config.json # Training config & hyperparameters
└── checkpoints/
├── 0000100_adapters.safetensors # Checkpoint at iteration 100
└── 0000200_adapters.safetensors # Checkpoint at iteration 200
Usage with MLX
1from mlx_lm import load, generate
2
3# Load base model with LoRA adapters
4model, tokenizer = load(
5 "mlx-community/DeepSeek-R1-Distill-Llama-8B-4bit",
6 adapter_path="purplesquirrelnetworks/purple-squirrel-r1-multichain-lora"
7)
8
9messages = [
10 {"role": "system", "content": "You are a multichain ecosystem expert."},
11 {"role": "user", "content": "How does Wormhole enable cross-chain messaging?"}
12]
13
14prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
15response = generate(model, tokenizer, prompt=prompt, max_tokens=500)
16print(response)
Continue Fine-Tuning
1mlx_lm.lora \
2 --model mlx-community/DeepSeek-R1-Distill-Llama-8B-4bit \
3 --resume-adapter-file purplesquirrelnetworks/purple-squirrel-r1-multichain-lora/adapters.safetensors \
4 --data /path/to/your/data \
5 --iters 100
Domain Knowledge
Protocols covered: Wormhole, LayerZero, ZetaChain, Compose Network, Aptos, Monad, NEAR, Polygon, Stacks, Aurora, Pyth, 1inch, Beefy, Relay, Pipe Network, DoubleZero, BitcoinOS.
Topics: cross-chain messaging, L1/L2 ecosystems, DeFi infrastructure, onchain AI agents, RWA tokenization, account abstraction, sustainable yield.
Related Resources
Citation
1@misc{purplesquirrel-r1-multichain-lora-2025,
2 title={Purple Squirrel R1 Multichain LoRA Adapters},
3 author={Karsten, Matthew},
4 year={2025},
5 publisher={Purple Squirrel Media},
6 howpublished={\url{https://huggingface.co/purplesquirrelnetworks/purple-squirrel-r1-multichain-lora}},
7 note={MLX LoRA adapters for DeepSeek-R1-Distill-Llama-8B, fine-tuned on Wrapped Events multichain conference data}
8}
License
MIT
Contact
