Base Model

Base model: mistralai/Ministral-3-14B-Reasoning-2512
Architecture: Mistral3ForConditionalGeneration (language component fine-tuned with LoRA)
Max context length: 262,144 tokens (from text_config.max_position_embeddings)

Training Data (project-local corpus)

DAPT: train=408, validation=27, test=45
SFT: train=1620, validation=204, test=190
Sources: curated Commodore 64 manuals and technical documents from this project.

Training Recipe

Pipeline: DAPT + SFT (LoRA)
Precision: bf16
Max sequence length: 2048
Batch size per device: 1
Gradient accumulation: 16
Learning rate: 2e-05
Epochs: 3.0
Warmup steps: 100
Logging/save/eval steps: 50/500/250
Optimizer: adamw_torch, eval strategy: steps
Gradient checkpointing: False
LoRA: r=16, alpha=32, dropout=0.05, scope=language_qkvo
SFT options: assistant_only_loss=True, packing=False

Run summary:

DAPT checkpoint: checkpoint-78
SFT checkpoint: checkpoint-306
DAPT global steps: 78 / 78
SFT global steps: 306 / 306
Last logged SFT step: 300
Last logged SFT loss: 0.03266857624053955
Last logged SFT token accuracy: 0.9903446585685015
Card generated at (UTC): 2026-03-02T16:47:33.221202+00:00
Source git revision: 13fafe7

Reasoning Validation Snapshot

Validation status: FAIL
Source artifacts: results/reasoning_validation/14b/20260302_152057
Note: contract/format retention passed; failure is due to strict exact-token determinism (hash mismatch across repeated same-seed runs). | Metric | Value | | --- | ---: | | single_think_tag_rate | 1.0000 | | single_balanced_tag_rate | 1.0000 | | single_final_after_think_rate | 1.0000 | | multi_turn_retention_rate | 1.0000 | | format_contract_pass_rate | 1.0000 | | exact_hash_match_rate | 0.3403 | | semantic_similarity_avg | 0.9956 | | crash_or_timeout_rate | 0.0000 |

Usage (Transformers + PEFT)

python

import torch
from peft import PeftModel
from transformers import AutoTokenizer
from transformers.models.mistral3 import Mistral3ForConditionalGeneration

base_id = "mistralai/Ministral-3-14B-Reasoning-2512"
adapter_id = "ibitato/c64-ministral-3-14b-thinking-c64-reasoning-lora"

tokenizer = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)
base_model = Mistral3ForConditionalGeneration.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base_model, adapter_id)

prompt = "Explain the C64 SID chip in one concise paragraph."
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Limitations

This adapter is specialized for C64 technical assistance, not for broad benchmark optimization.
No additional safety fine-tuning was applied beyond the base model behavior.
Evaluation in this repo focuses on training diagnostics; downstream benchmarking should be done per target use case.