Property	Value
Base Model	google/mt5-small (300M params)
Training Method	Claude → mT5-small distillation
Languages	English, Korean, Japanese, Chinese
Training Data	v1: ~9,700 + v2: ~5,900 Korean examples
Format	ONNX (int8 quantized)
License	Apache 2.0

Based on the Dense X Retrieval (Propositionizer) approach, extended to multilingual.

Usage

Transformers.js (Browser / Node.js)

javascript

import { pipeline } from '@huggingface/transformers';

const decomposer = await pipeline(
  'text2text-generation',
  'liliplanet/propositionizer-mt5-small'
);

const result = await decomposer(
  'Title: Meeting. Section: . Content: The deadline is Friday and the rate was reduced.',
  { max_new_tokens: 256, repetition_penalty: 2.0 }
);
console.log(JSON.parse(result[0].generated_text));
// ["The deadline is Friday.", "The rate was reduced."]

Python (Transformers)

python

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("liliplanet/propositionizer-mt5-small")
model = AutoModelForSeq2SeqLM.from_pretrained("liliplanet/propositionizer-mt5-small")

input_text = "Title: 회의. Section: . Content: 김 대리가 시급을 낮추고 마감은 금요일이다."
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256, repetition_penalty=2.0, no_repeat_ngram_size=3, num_beams=4)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Input Format

Follow the Propositionizer format:

Title: {title}. Section: {section}. Content: {content}

Training

v1

Source texts: CNN/DailyMail, XSum, Wikipedia (EN/KO/JA/ZH), KLUE, XLSum
Labeling: Claude Haiku 4.5 atomic fact decomposition
Data: EN 4,879 / KO 2,860 / JA 983 / ZH 975 (~9,700 total)
Training: 5 epochs, Adafactor, lr=1e-3, batch_size=16

v2 (current)

Focus: Korean quality improvement
Additional data: ~5,900 Korean complex sentences (XLSum KO, KLUE NLI/RE/STS, KorQuAD, NSMC)
Improved prompt: language drift prevention, proper noun preservation
Training: continued training from v1, 3 epochs, lr=5e-4
Generation config: repetition_penalty=2.0, no_repeat_ngram_size=3

v2 improvements over v1

Korean language drift (한국어→영어 전환) resolved
Repetition loop eliminated
Proper noun preservation improved (CEO, Q2 등)

Comparison with Original Propositionizer

	Original (Flan-T5-Large)	This Model (mT5-Small)
Parameters	780M	300M
Languages	English only	EN, KO, JA, ZH
Teacher	GPT-4	Claude
Training Data	English only	Multilingual

Known Limitations

Small model (300M) has limited capacity for complex decompositions
May hallucinate facts not present in the source text, especially with uncommon proper nouns
Best suited for short-to-medium length paragraphs (< 500 chars)
Korean complex sentences with many IT terms may produce errors

Citation

bibtex

@article{chen2023densex,
  title={Dense X Retrieval: What Retrieval Granularity Should We Use?},
  author={Chen, Tong and Wang, Hongwei and Chen, Sihao and Yu, Wenhao and Ma, Kaixin and Zhao, Xinran and Zhang, Hongming and Yu, Dong},
  journal={arXiv preprint arXiv:2312.06648},
  year={2023}
}

Part of MemRosetta

This model is a component of the MemRosetta project for multilingual memory and knowledge extraction.