This model is a fine-tuned version of LiquidAI/LFM2.5-8B-A1B, adapted on the angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k dataset for English text-generation and reasoning-style responses.

The fine-tuning run used a custom Convergent Intelligence optimizer stack, CIxOpt, designed for heterogeneous routing across parameter types. The goal of this checkpoint is to test whether a Liquid Foundation Model backbone can be adapted efficiently through targeted sparse participation rather than broad full-model modification.

This is an experimental research checkpoint intended for continued evaluation, domain adaptation, and architecture/optimizer testing.

Base Model

Base: LiquidAI/LFM2.5-8B-A1B
Architecture family: Liquid Foundation Model / hybrid causal language model
Task: Causal language modeling
Language: English
License: Apache 2.0, inherited from the released model metadata unless otherwise restricted by upstream dependencies

Dataset

Fine-tuning data:

angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k

The dataset was processed into chat-style training examples using tokenizer-compatible chat formatting where available. Empty or malformed examples were filtered before tokenization.

Training Method

This model was trained using the Convergent Intelligence CIxOpt optimizer framework.

Optimizer Design

CIxOpt applies heterogeneous routing based on parameter type and tensor structure:

Lion-style sign momentum for large projection matrices
AdamW-style updates for sensitive normalization surfaces
Adamax-style handling available for embeddings or language-head-style parameters
fp32 optimizer state for bf16/fp16 model safety
Gradient centralization for eligible matrix-shaped gradients
Discrepancy-aware caution filtering for sign updates
Decoupled weight decay
Gradient clipping during training

Sparse Participation Strategy

The fine-tuning strategy was designed to avoid unnecessary modification of the full pretrained backbone. Instead, training focused on selected adaptation surfaces, especially upper-layer projection and normalization modules.

The intended training philosophy was:

text freeze most pretrained structure adapt upper reasoning / response-shaping layers preserve lower representational substrate route parameter groups by optimizer behavior

This makes the checkpoint useful for studying efficient adaptation of LFM-family models under constrained compute.

Intended Use

This model is intended for:

Research on Liquid Foundation Model fine-tuning
Optimizer experiments with CIxOpt
Reasoning-style text generation
Instruction-following experiments
Lightweight comparative evaluation against other small or sparse-adapted causal LMs
Continued fine-tuning or domain adaptation

Example use cases:

Analytical response generation
Reasoning trace compression
Technical explanation
Experimental agent backbones
Small-scale model behavior studies

Out-of-Scope Use

This model is not intended for high-stakes autonomous deployment without additional evaluation.

Do not use this model as the sole decision-maker for:

Medical diagnosis
Legal judgment
Financial decisions
Emergency response
Cyber offensive automation
Personnel screening
Surveillance or targeting decisions
Any setting requiring verified factual accuracy

Limitations

This is an experimental fine-tuned checkpoint. Known or expected limitations include:

May hallucinate facts, citations, dates, or source attributions
May inherit biases or artifacts from the base model and fine-tuning data
May overproduce reasoning-style explanations when shorter answers are preferred
May be sensitive to prompt formatting
Has not been fully benchmarked across safety, factuality, coding, mathematics, or instruction-following suites
Fine-tuning on reasoning-style data does not guarantee correct reasoning
Sparse or targeted adaptation may leave some capabilities close to the base model while changing others unevenly

Safety Notes

Users should independently validate important outputs. Generated content may be plausible but incorrect.

For deployment-facing use, additional steps are recommended:

Benchmark against known evaluation suites
Run toxicity and bias evaluation
Test refusal behavior
Evaluate hallucination rate
Compare against the base model
Add domain-specific guardrails
Use retrieval or verification for factual tasks

Example Usage

python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "reaperdoesntknow/LFM2.5-8B-A1B-Opus-Distil"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

prompt = "Explain why stable positional encoding matters for long-context language models."

inputs = tokenizer(
    prompt,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=1024,
        do_sample=True,
        temperature=0.7,
        top_p=0.95,
        repetition_penalty=1.05,
        pad_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

Training Configuration

Approximate training configuration used:

text base_model: LiquidAI/LFM2.5-8B-A1B dataset: angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k task: causal language modeling / instruction-style SFT optimizer: CIxOpt state_dtype: fp32 weight_decay: enabled gradient_clipping: enabled chat_template: tokenizer-compatible formatting padding: max_length loss_masking: padding tokens masked with -100

Exact loss curves, benchmark scores, and hardware details should be added after evaluation.

Evaluation

Formal benchmark results have not yet been added.

Recommended evaluation targets:

Perplexity on held-out validation data
MT-Bench-style instruction following
IFEval
GSM8K or similar lightweight reasoning checks
MMLU-style knowledge evaluation
TruthfulQA-style hallucination testing
Human preference comparison against the base model
Side-by-side testing against smaller LFM2.5 checkpoints

Citation

Base model:

bibtex @misc{liquidai_lfm25, title = {LFM2.5-8B-A1B}, author = {Liquid AI}, year = {2025}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/LiquidAI/LFM2.5-8B-A1B}} }

Fine-tuning dataset:

bibtex @misc{angrygiraffe_reasoning_dataset, title = {claude-opus-4.6-4.7-reasoning-8.7k}, author = {angrygiraffe}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/datasets/angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k}} }

Author / Maintainer

Fine-tuning and optimizer experimentation by: Convergent Intelligence LLC | Research & Development Divisions Research and development focus: AI systems, intelligence analysis, mathematical frameworks, optimizer design, and efficient model adaptation.

Disclaimer

This model is provided for research and experimentation. It should not be treated as a verified expert system. Outputs require human review, especially in factual, technical, legal, medical, financial, operational, or safety-critical contexts.