Views
No views yet
| Setting | Value |
|---|---|
| Base model | ByteDance/Ouro-2.6B-Thinking |
| Datasets | UltraChat-200k → NVARC-augmented-puzzles |
| Quantization | 4-bit NF4 double-quant (BitsAndBytes) |
| LoRA r / alpha | 16 / 32 |
| Bits (w / a / kv) | 4 / 4 / 4 |
| Rotations used | R3 (Q/K head_dim), R4 (down_proj input, block-Hadamard) |
| Max seq length | 1024 |
| Transformers | 4.54.1 |
| File | Description |
|---|---|
roste_adapter.pt | Trained lora_A + lora_B tensors from apply_roste |
roste_config.json | Hyperparameters needed to reconstruct the model |
tokenizer.* | Tokenizer saved from base model |
apply_roste implementation.
The weight-injection pattern is:1saved = torch.load("roste_adapter.pt", map_location="cpu")
2current = dict(model.named_parameters())
3with torch.no_grad():
4 for name, tensor in saved.items():
5 if name in current and current[name].shape == tensor.shape:
6 current[name].copy_(tensor.to(current[name].device, current[name].dtype))