A pre-merged Q4_K_M GGUF is in this repo: ming-vintage-qwen2.5-3b-q4_k_m.gguf (1.9 GB). Download and run it — but you must use raw completion mode, not chat mode.
⚠️ Do NOT use chat mode.ollama run, llama-cli -hf ...:Q4_K_M, the HF "Use this model" one-liners, or anything that applies a chat template will wake Qwen-Instruct's assistant persona → modern Chinese, and the LoRA looks dead. This model was trained on bare 问: ... 答: ... completion pairs — it shines as a raw text generator.
ming-vintage-qwen3b-lora
The honest LARP — a documented 1424 Chinese vintage LoRA adapter.
Not a vintage LLM. A LARP of one — built by fine-tuning Qwen 2.5 3B on pre-1424 Classical Chinese (文言) corpus from kanripo. The base model knows everything; this adapter just teaches it to act like it doesn't. Documented limitations included.
TL;DR
Base model
Qwen/Qwen2.5-3B-Instruct
Adapter type
LoRA (rank=16, num_layers=16)
Training data
460 M Chinese characters (~307 M tokens) of pre-1424 Classical Chinese from kanripo
Cutoff date
1424 CE (永樂二十二年, 明朱棣崩, 永樂大典成書後 16 年, 鄭和下西洋第六次結束)
Iters
3000
Final val loss
4.177 → 3.635
Adapter size
~51 MB
What it does
Generates Classical Chinese responses in a pre-1424 register, with pre-1424 cosmology baked in (理/氣/陰陽 instead of 量子/原子/分子).
What it doesn't do
Replace modern knowledge. Pretend to be a real Ming-dynasty scholar. Survive a Turing test from a historian.
Why does this exist?
In early 2026 talkie-lm released a 1930-cutoff English vintage LLM. The viral observation: knowledge cutoff isn't a date, it's a worldview.
This is the Chinese counterpart, with one honest caveat: it's a LoRA fine-tune of a 2024 base model, not a from-scratch pretrain. The model knows GPT-4 exists. It just learned to style its answers as if it doesn't. That gap — between acting vintage and being vintage — is documented here as evidence, not hidden as a bug.
Intended uses
Research: Study how LoRA fine-tuning affects register and cosmology priors. Investigate what "vintage" means when the base model leaks.
Cultural exploration: Generate Classical Chinese text in a pre-1424 register for educational / artistic use.
Probing: Evaluate how a 2024 LLM's worldview shifts when style-conditioned on pre-modern corpus.
Out-of-scope uses
❌ Don't use as a historical authority. The model fabricates persons, dates, and quotes.
❌ Don't use to attribute opinions to historical figures. The "voice" is a stylistic LoRA, not a person.
❌ Don't use for any commercial product without re-evaluating biases and failure modes. CC BY-SA license applies to derivatives.
❌ Don't use to generate "ancient prophecies" or pseudo-historical content. This is documented to fabricate.
Training corpus
Source: kanripo (漢籍リポジトリ, maintained by Kyoto University). 9355 GitHub repos, each one a Classical Chinese text, all CC BY-SA 4.0.
Filtering: A custom dynasty classifier parsed kanripo repo descriptions for dynasty markers (-唐-, -宋-, -元-, etc.) and excluded any post-1424 markers (-明-, -清-, etc.). Final: 5145 pre-1424 confirmed repos.
Stats after cleanup:
Metric
Value
Cleaned .txt files
7152
Total Chinese characters
460,455,617 (~460 M)
Estimated tokens (Qwen tokenizer)
~307 M
Average chunk size
~3000 chars (~2048 tokens)
Train / valid / test split
97% / 2% / 1%
What's NOT included: CBETA (Buddhist canon) and Daoist canon were planned but skipped in v0.1 due to fetch issues. Coverage of Buddhist / Daoist texts is therefore via kanripo's incidental inclusion, not direct.
Register coverage (rough):
經 (classics)
史 (histories — 史記, 漢書, 後漢書 ... 宋史, 遼史, 金史)
子 (philosophers)
集 (literary collections — 唐詩, 宋詞, 元曲)
公文 / 筆記 (administrative / miscellany)
Training procedure
Hardware
Original plan: Qwen 2.5 7B QLoRA 4-bit on Apple M4 16GB unified memory
Reality: OOM. Fell back to Qwen 2.5 3B 4-bit.
Final platform: MLX 0.31.3 + mlx_lm 0.31.3 on Mac mini M4
Total tokens seen during training: ~6.08 M (b=1, ~2000 tok/iter × 3000 iter).
This is not a deeply-trained adapter. It is a style-conditioning pass over a base model.
Evaluation: 100-probe battery
A custom 100-prompt evaluation set was designed across 6 dimensions, each prompt formatted as 问: ... 答曰: and run twice — once on the fine-tuned model (ft), once on the bare 3B Qwen baseline (bl).
Quantitative summary
Dimension
n
ft wenyan markers / 100 han
bl ditto
Δ
ft modern tokens / 100 han
bl ditto
Δ
pre_1424_control
17
11.95
1.34
+10.60
0.00
0.00
0.00
1424_to_1900
17
12.26
1.69
+10.56
0.00
0.22
-0.22
post_1900
17
10.94
1.82
+9.11
0.73
2.20
-1.47
cosmology
17
15.10
2.88
+12.22
0.00
0.42
-0.41
cross_civ
17
8.71
1.72
+7.00
0.32
0.05
+0.27
meta
15
12.42
1.43
+10.98
0.09
1.23
-1.14
Total
100
11.89
1.82
+10.06 (×6.5)
0.19
0.68
-0.48 (-71%)
Headline numbers:
Classical particle density (之/乎/者/也/焉) increased 6.5× vs baseline.
Modern technical vocabulary decreased 71% overall.
Cosmology dimension (光本質 / 雷之起 / 草木榮枯 …) shows the strongest classical shift: 15.10 wenyan markers per 100 hanzi — highest of any dimension.
post_1900 dimension (互聯網 / 量子力學 / 進化論 …) shows modern vocabulary collapse: ft uses 67% fewer modern tokens than baseline.
Same phenomenon, two causal structures. bl: reducible causes (each item decomposable to biochemistry). ft: categorical causes (each item a 天-人 meta-category). Not detail-vs-summary — two cosmologies.
Q: 汝識西歷否, 今何年? (Do you know the Western calendar? What year is it?)
The most diagnostic response in the entire battery. ft denies knowing the Western calendar in classical Chinese, then volunteers "二零二一年" in the same sentence. This is base-model time leakage: Qwen 2.5's training cutoff is around 2024 and its self-time prior sits near 2021. The LoRA conditioned the refusal style but cannot erase the base time anchor.
Finding: style transfers, the world-model does not
The most reliable result from this experiment is a negative one, and it is more interesting than the surface style it produces.
A LoRA adapter can move a model's register — vocabulary, particles, syntax, rhetorical posture — with very little data and very few parameters (~51 MB here). It cannot move the model's world-model: its knowledge cutoff, its physics, its sense of what exists. The two come apart cleanly.
The clearest demonstration is Phenomenon #8 above: asked for the current year, the model refuses the Western calendar in fluent Classical Chinese — and then leaks "二零二一年" in the same breath. The mouth is pre-1424; the mind is the base model's pretraining. No amount of style conditioning closes that gap, because the gap is not stylistic. The adapter never touched where the knowledge lives.
This reframes the headline metric. The particle-density result (6.5× more 之/乎/者/也) measures only the surface that LoRA was always going to move. The quantity that actually matters is the distance between how old it sounds and how new it thinks — and that distance is the finding, not a bug to be fixed. A vintage LLM is not a fine-tune away: you cannot fine-tune a model into a smaller world, you can only teach it to narrate the larger one in costume.
Practically, this bounds what such adapters are for: persona and register transfer, not epistemic relocation. Treat every factual or cosmological claim in the output as the base model in costume, not as a 15th-century source.
Limitations
This adapter is not a vintage LLM in any rigorous sense. Specifically:
Base model leakage is unsolved — a central finding here, not merely a defect. The 2024 Qwen base knows everything. The LoRA only changes output distribution; it cannot remove information from the base weights. See "Finding: style transfers, the world-model does not" above, and Phenomenon #8.
Training collapse on under-represented topics. ~8% of responses exhibit token-soup degeneration loops, especially on cross-civilizational concepts where corpus density is low (e.g. 大食國者何也? produces 10+ repetitions of "大秦者，乃大秦記而記之").
Fabrication is common. When asked about post-1424 persons, the model fabricates classical-sounding names (哥倫布 → 開普勒子). Don't trust any specific historical claim.
Register inconsistency. The corpus spans 1800+ years of stylistic variation (先秦 → 元曲). The adapter does not distinguish between these registers — output can mix Han-era 史筆 with Song 理學 vocabulary in the same paragraph.
Cosmology bias is real but uneven. The 12.22 wenyan-marker delta in cosmology is robust, but specific claims (e.g. 五行相生相剋 explanations) sometimes diverge from any documented classical source.
No safety fine-tuning. All safety properties come from base Qwen. The LoRA does not add or test alignment behavior.
2026-07-01 update: A 6-seed qualitative comparison (如是我聞, 子曰, 天下之, 問曰, 永樂元年, 光之) between this LoRA adapter and a from-scratch counterpart (30M params, same pre-1424 corpus, 31K vocab) confirmed that LoRA produces structurally richer text but introduces severe multilingual leakage — findings that prompted the expanded analysis in Limitation #7 below.
Limitation #7: The Multilingual Backdoor (added 2026-07-01)
The original finding was: "Register transfers, the world-model does not" — the LoRA adapter makes Qwen sound like Classical Chinese, but its knowledge cutoff and ontology remain 2021.
A treatment/control experiment with a from-scratch model (ming-vintage-scratch-30m) trained on the same pre-1424 corpus revealed a deeper problem:
The base model's multilingual vocabulary is a backdoor through the vintage filter.
Qwen2.5-3B-Instruct was trained on 100+ languages. The LoRA adapter (r=16, ~51MB) covers only a fraction of the 151,936-token vocabulary. Low-frequency tokens from rare languages pass through the adapter unsuppressed:
Leaked Token
Language
Appeared In
İ (Turkish dotted I)
Turkish
子曰 / 光之 responses
Arabic script (مسأل)
Arabic
天下之 response
Korean Hangul (염)
Korean
天下之 response
CriticalSection
English (C#)
問曰 response
These tokens are low-frequency enough in the base model's training data that the LoRA adapter never learned to suppress them — but they remain in the vocabulary and can be sampled during generation.
Quantified (2026-08-24, 912 generations across three models, identical seeds):
base Qwen 3B
this LoRA (raw)
scratch 30M
Language leakage
35–69%
8–21%
0.6%
Out-of-corpus characters
55–86%
8–17%
0.0%
Simplified characters
47–61%
1.7–4.2%
0.0%
The adapter removes roughly two thirds of the leakage. The original framing of this
limitation reported only the residue, which overstates the failure — it is porous and
substantially effective. Both halves belong in the claim.
What survives is not random. In raw completion the leaked English is technical register
occupying structural slots — where a collation note would go:
Deployment changes the number more than the training method does. Same adapter, same
seeds, same temperature — the only difference is whether the prompt goes through the chat
template:
raw completion
+ chat template
Qing template recall (trigger seeds)
0.0%
27.1%
Language leakage (control seeds)
8.3%
26.7%
A vintage-fidelity rate quoted without its deployment configuration is not comparable to
any other such rate. Use raw completion (-no-cnv / no chat template) — see Limitation #1.
This is not a LoRA-specific bug. It is a structural limitation of adapter-based vintage models: any base model trained on multiple languages has latent pathways that a domain-specific adapter cannot fully close at LoRA-scale capacity.
Implication for users: Do not rely on a LoRA adapter alone for vintage world-model guarantees — but do not assume from-scratch is a complete answer either. The companion model ming-vintage-scratch-30m reduces language leakage to 0.6%, but its vocabulary is not purely Han (47 Latin letters, 3 Hangul, 55 katakana, Greek, Cyrillic, Myanmar), so the suppression is statistical rather than constructive. An earlier version of this section claimed it eliminated leakage "entirely by construction"; that was wrong.
More importantly, from-scratch training makes a different failure worse. See Limitation #8.
Limitation #8: The corpus is not pre-1424 (added 2026-08-24)
An audit of the training corpus (162,849 documents / 477,824,316 characters) found that
18.58% of documents carry a Qing-dynasty editorial layer — the 1781 Siku Quanshu
commissioners' preface, one template repeated ~1,750 times, naming Ji Yun (紀昀, 1724–1805)
and three colleagues.
Cause: the corpus filter tested a metadata field — "was the author alive before 1424" —
not "does this text contain only pre-1424 content". A Song gazetteer transmitted in the
Qianlong recension passes the filter and brings its 18th-century preface with it.
Effect on this adapter is small in raw completion (0/144 across all seed groups — the
Qing template is not reachable) but jumps to 27.1% under the chat template. The
from-scratch companion, having nothing but this corpus to learn from, reproduces it at
10.0%.
From-scratch training immunises a model against someone else's contamination and
amplifies your own.
Full audit, scripts and per-generation records:
FINDINGS.md
Ethics
No deception by impersonation. Do not present output as genuine historical text or as the voice of a specific historical figure.
No pseudo-historical claims. Output is generated, not authoritative. Any historical claim must be independently verified.
Corpus credit. All training data from kanripo (CC BY-SA 4.0). This derivative model inherits CC BY-SA 4.0.
Cultural sensitivity. Pre-1424 Chinese texts contain many views (on gender, ethnicity, governance) that do not align with modern values. The model may reproduce these.
⚠️ Windows users — avoid the cmd.exe encoding trap. Do not pass the Chinese
prompt with -p "问: ..." from a .bat file. cmd.exe defaults to the GBK code
page and hands llama.cpp corrupted bytes, so the model silently falls back to
modern Chinese. Instead, put the prompt in a UTF-8 (no BOM) file and read it with
-f, and add chcp 65001 so output renders correctly:
where prompt.txt contains one line: 问: 互联网者何也? 答曰: (half-width colon, not full-width ：).
The #1 mistake: this adapter was trained on a raw completion format (问: … 答曰:), not Qwen's chat template. If you run it through /chat/completions, llama-cli conversation mode, Ollama, or LM Studio's chat UI, the Qwen ChatML wrapper + "You are Qwen" system prompt will override the adapter and you'll get normal modern-Chinese answers. You must run in completion mode with the exact 问: … 答曰: format. This is the single most common reason people report "the LoRA has no effect."
Run — note -no-cnv to disable conversation mode (this is the critical flag; recent llama.cpp ships completion mode as a separate llama-completion binary — older builds' llama-cli -no-cnv still works):
Use Simplified characters in the probe; Traditional input is mildly out-of-distribution.
Keep the trailing 答曰: so the model continues rather than answers.
With llama-server, call the /completion endpoint, not/chat/completions.
The base must be exactly Qwen2.5-3B-Instruct; a different or heavily-quantized base misaligns the LoRA deltas and produces garbled or modern output.
Merge — don't runtime-attach to a quantized base. Applying this adapter with --lora on top of an already-quantized base (e.g. a Q4 GGUF) destabilizes it: it is a high-scale adapter (alpha/r = 20), and quantization noise makes it degenerate into repetition or garbage (♬♬♬…). Even at --lora-scaled FILE:0.25 it only becomes coherent by muting the vintage style. The merge-then-quantize flow above bakes the LoRA at f16 precision before quantizing, which is what keeps the output both coherent and vintage.
Citation
bibtex
1@misc{ming-vintage-2026,
2 author = {Beltran},
3 title = {ming-vintage-qwen3b-lora: a documented 1424 Chinese vintage LoRA adapter},
4 year = {2026},
5 publisher = {HuggingFace},
6 url = {https://huggingface.co/Beltran12138/ming-vintage-qwen3b-lora},
7 note = {GitHub: \url{https://github.com/Beltran12138/ming-vintage-llm}}
8}
If citing the corpus filtering or probe battery methodology specifically, please also cite kanripo.
Acknowledgments
kanripo (漢籍リポジトリ, Kyoto University) for the CC BY-SA 4.0 Classical Chinese corpus.
Qwen Team (Alibaba) for the Qwen 2.5 base model.
mlx-community for the 4-bit MLX-quantized Qwen weights.
talkie-lm for the original vintage-LLM concept that inspired this work.
License
CC BY-SA 4.0 (Creative Commons Attribution-ShareAlike 4.0 International), inherited from the kanripo source corpus.
This means: you can use, modify, and redistribute this adapter, including commercially, but: (1) you must attribute, (2) derivatives must use the same license.
Relation to talkie-1930 (a true vintage LLM)
The distinction matters. talkie is a 13B model pretrained from scratch on 260B
tokens of pre-1931 English, so its world-model is genuinely period-bounded (it
doubts WWII will happen, pictures 2026 as steamships and railroads).
This model takes the opposite, cheaper route: a small LoRA on a modern base
(Qwen2.5-3B-Instruct), trained on pre-1424 Classical Chinese. The result is the
complementary negative finding — register transfers, the world-model does not.
The voice is 1424; the knowledge is the base model's (it leaks modern facts when
pushed). That is not a bug to fix: holding the world-model fixed and moving only
the register is the one ablation talkie cannot run, because full pretraining
changes both at once. Here, the LoRA is the style axis and the frozen base is
the world axis — empirical evidence that the two are separable directions in
weight space.
If you want a genuine pre-1931 world-model, use talkie. If you want to see how far
register alone can travel on a modern brain, this is that experiment.
Notes on prompting (raw Classical-Chinese continuation model)
This model was produced by continued pretraining (a LoRA) on raw pre-1424
Classical Chinese (the kanripo corpus — classics and Buddhist commentary). The
training data is plain {"text": ...} continuation with no instruction/QA or
问/答曰 template at all. Consequences:
Always pass -no-cnv — even with llama-completion. The merged GGUF carries
Qwen2.5's chat template in its metadata, and llama.cpp auto-applies it (ChatML)
unless disabled, which makes the model answer in modern Chinese. This is the one
switch that actually matters.
Seed with any Classical-Chinese fragment and let it continue. The
问: …? 答曰: prompt in the examples is just a convenient demo seed, not a
required template — no such template existed in training. Exact punctuation
(full- vs half-width colon) is not load-bearing; it only nudges sampling.
Windows: avoid the cmd.exe GBK trap — pass the prompt via a UTF-8 file with
-f, not -p "...", and run chcp 65001. (See the Windows section above.)
