You can deploy this model and start earning money today!

MarxistLeninist/
AGILLM-4.3-clean-20260802

Name: AGILLM-4.3-clean-20260802
Availability: InStock

Views

No views yet

AGILLM 4.3

Adaptive General Intelligence Large Language Model v4.3 — a 1.22B parameter language model with novel DiffusionBlock + Mixture-of-Experts architecture.

Architecture

Component	Details
Parameters	1.22B (1,221,580,802)
Architecture	Transformer + DiffusionBlock + MoE
DiffusionBlock	14 blocks, loss-balanced schedule, transformer router
MoE	2 experts, top-1, shared expert (mult=2)
Attention	Sublinear (window=128, stride=128, sinks=4)
Context	1536 tokens
Optimizer	AdamW 8-bit with cosine LR decay
Training Objective	Stochastic (AR 45% / SAT 25% / NAT 30%)

Training Status

🟢 Actively training on RTX 3090 Ti (fedC recovery run)

Progress: ~33% through 67.2B token run (~22.1B tokens seen)
Throughput: ~70k tok/s
Current step: ~897,000 (recovery from step 727,857)
LR Schedule: Cosine decay (min_mult=0.10)
Data: fineweb, fineweb-edu, wikipedia, c4, openwebtext, falcon-refinedweb, proof-pile-2, smollm-corpus + quality anchors

Sample Generations (Step 894,129)

⚠️ Model is at 33% of training. Outputs are expectedly noisy/incoherent at this stage.

Prompt	Output (64 tokens, CPU inference)
`The capital of France is`	Mz disc Topunion was resume abstra trop spacerrel Class WelshWangomas...
`def fibonacci(n):`	Similarly form,meditortextttexpectTotal that.
`A polite greeting is`	Fraction stringColor.Pathsrolledplay chips...frogrendvements numerals...
`2 + 2 =`	200 ofPMeurSUMzvements BandTotalcovering Delta quadrillixy Lagrange zoo...
`The theory of relativity states that`	$$ European Numbers? QuantuplBlocksond. Total Cenidirectionalteness...
`Once upon a time`	Mont?/no.redbit spectrometer Kidyunpless Cravements did Matlabitat...
`To make tea, first`	quickly spodge quadrilexpect string-work... MatlabRemember that...
`The meaning of life is`	Mz)Bvements and to handmadeuckedhel857Paths notchrelationships...

Quality probe results (short-form, auto-infer):

Promoted checkpoint (step 364,414): 3/4 probes passed ("Paris" ✅, "hello" ✅, code ✅, math ❌)
Raw baseline (step 363,424): 0/4 probes passed
Improvement trend confirmed; long-form coherence still developing

Full sample data: samples/agillm43_samples_step894129_20260706.json

Recovery Checkpoint Mirror

This repository mirrors recovery checkpoints from the fedC training run:

checkpoints/recovery_fedC/artifacts/full/ — promoted full checkpoints
checkpoints/recovery_fedC/artifacts/delta/ — recovery backup deltas
latest_full.json, latest_delta.json — quality-gated uploader state

Quality Gate

Perplexity/val CE: ELIMINATED (owner directive — model is undertrained, loss metrics caused 4+ regression-to-pin restarts)
Promotion: Relative quality vs raw baseline (auto-infer probes)
Resume: Newest delta unconditional (no rollback)

Latest generation sample (fedC recovery)

Captured 2026-07-06, ~30 min after cosine-LR decay went live on the fedC recovery run. Raw greedy generations from the auto-infer quality smoke, baseline pin (step 363424) vs current checkpoint (step 894634). Perplexity is intentionally not used as a quality signal; these are the actual sampled tokens.

Prompt	Baseline `363424` (0/4)	Current `894634` (3/4)
`The capital of France is`	`, and "his bear toxiculeave`	`Paris.` ✅
`Python code to print hi:`	(word-salad)	`print "hello" ... def greet` ✅
`A polite greeting is`	(word-salad)	`Hello"` ✅
`2 + 2 =`	`and, - - t))), where`	`x. Python) print "hello"` ❌ (still no arithmetic)

Result: 3/4 probes now hit, up from 0/4 at the pin. The model is still early/undertrained (fragmentary, punctuation-noisy, code/prose bleed) but is emitting on-target tokens — the first qualitative jump after the constant-LR → cosine-decay fix. Checkpoint sha256 first64m: d9669293ccd9ce3376dfa0e21439e070d3b9ee21bd773da8bd9b9c690365242b.

MarxistLeninist/AGILLM-4.3-clean-20260802