You can deploy this model and start earning money today!

yachagye/
korean-classical-chinese-punctuation

Name: korean-classical-chinese-punctuation
Availability: InStock

Views

No views yet

Korean Classical Chinese Punctuation Prediction Model

한국 고전한문 구두점 예측 모델

License: CC BY-NC-SA 4.0 · GitHub · DOI

Deep learning model for automatic punctuation prediction in Korean classical Chinese texts. Achieving F1 score of 0.9110 (v2.5) on 7 punctuation marks.

딥러닝 기반 한국 고전 한문 텍스트 자동 구두점 예측 모델입니다. 7종의 구두점을 F1 Score 0.9110 (v2.5)의 정확도로 예측합니다.

Model Versions

Version	Base Model	F1 Score	Note
v2.5	SikuRoBERTa (SIKU-BERT/sikuroberta)	0.9110	Latest Recommended (Korean Cultural Studies 111)
v2.0	SikuRoBERTa (SIKU-BERT/sikuroberta)	0.9110	Reference Model (Korean Cultural Studies 111)
v1	Chinese-RoBERTa (hfl/chinese-roberta-wwm-ext)	0.9050	Korean Journal of History 100

🔄 v2.0 → v2.5 Update

Version 2.5 is an incremental update of the v2.0 release, focusing on training stability and reproducibility. The model architecture, task definition, and data scheme remain identical to v2.0.

Name: Korean Classical Chinese Punctuation Prediction Model v2.5 Base Model: SIKU-BERT/sikuroberta Task: Multi-label Token Classification Labels: 7 punctuation marks (, 。 · ? ! 《》)

v2.5 Updates:

Removal of PatternAwareLoss
Adoption of standard BCEWithLogitsLoss for multi-label classification
Masked-mean loss computation based on token attention masks
Simplified training policy prioritizing stability and reproducibility

v2.0 (PatternAwareLoss) and v2.5 (standard BCE) showed equivalent performance, with no significant difference observed apart from a slight recall advantage of v2.0 on the closing bracket (》). Given sufficient training data and domain-specific pretraining, the structural pattern of consecutive punctuation marks converges probabilistically during training even without an explicit loss-function rule. Accordingly, v2.5 — the simpler training structure without additional assumptions — was adopted as the final model. (Korean Cultural Studies, vol. 111)

Validation - F1: 0.9110 (v2.5) Per punctuation:

?: F1=0.9485, P=0.9435, R=0.9535
,: F1=0.9182, P=0.9181, R=0.9183
。: F1=0.8880, P=0.9103, R=0.8668
·: F1=0.8866, P=0.9133, R=0.8615
《: F1=0.7805, P=0.8178, R=0.7465
》: F1=0.7735, P=0.8108, R=0.7395
!: F1=0.7015, P=0.8437, R=0.6003

Model Description

Architecture: Custom Multi-label Token Classification

BERT Encoder
Dropout Layer (rate: 0.1)
Linear Classification Head (768 → 7)
7 independent binary classifiers (one per punctuation type)
Sigmoid activation with threshold 0.5

Important Note: This model uses a custom architecture, not the standard AutoModelForTokenClassification. For full inference capabilities, use the custom PunctuationPredictor class from the GitHub repository.

Task: Multi-label Token Classification

Each character token can have 0 or more punctuation marks following it
7 punctuation types: , (comma), 。 (period), · (middle dot), ? (question mark), ! (exclamation mark), 《 (left title bracket), 》 (right title bracket)

Training Data: ~3.4 million training samples from Korean classical Chinese historical texts

Source documents: ~420 million characters
8 document genres: chronicles, registers, diaries, literary collections, laws, geography, biographies, etc.

Performance

Overall Performance

Version	F1 Score	Precision	Recall
v2.5	0.9110	-	-
v2.0	0.9110	0.9117	0.9103
v1	0.9050	0.9057	0.9043

Per-Punctuation Performance

Punctuation	v1 F1	v2.0 F1	v2.5 F1	v1 Precision	v2.0 Precision	v2.5 Precision	v1 Recall	v2.0 Recall	v2.5 Recall
?	0.9436	0.9486	0.9485	0.9419	0.9437	0.9435	0.9454	0.9536	0.9535
,	0.9127	0.9182	0.9182	0.9130	0.9175	0.9181	0.9124	0.9190	0.9183
。	0.8818	0.8879	0.8880	0.9054	0.9109	0.9103	0.8594	0.8661	0.8668
·	0.8759	0.8864	0.8866	0.9157	0.9134	0.9133	0.8394	0.8609	0.8615
《	0.7367	0.7817	0.7805	0.8155	0.8204	0.8178	0.6717	0.7465	0.7465
》	0.7311	0.7764	0.7735	0.8024	0.8126	0.8108	0.6713	0.7432	0.7395
!	0.6369	0.7007	0.7015	0.8114	0.8399	0.8437	0.5241	0.6011	0.6003

Domain-specific Performance (v1.0 → v2.5)

Domain	v1.0 F1	v2.5 F1	Δ F1	Data Size (chars)
Geography (지리지)	0.9116	0.9385	+0.0269	501,942
Chronicles (연대기)	0.9162	0.9199	+0.0037	30,682,976
Registers (등록)	0.9114	0.9257	+0.0143	1,896,232
Biographies (전기)	0.8606	0.8953	+0.0347	591,983
Laws (법령)	0.8485	0.8799	+0.0314	907,893
Literary Collections (문집)	0.8354	0.8693	+0.0339	1,885,268
Diaries (일기)	0.8229	0.8515	+0.0286	544,768

Chronicles, already above 0.91 in v1.0, show only a marginal gain, whereas the relatively lower-performing types (Diaries, Literary Collections, Laws, Biographies) show larger gains. See Korean Cultural Studies, vol. 111, for detailed analysis.

External Validation (Unseen Data, v1)

Dataset	F1 Score	Data Size (chars)	Source
한국문집총간 (Korean Literary Collections)	0.8784	166,763,095	ITKC Database
일성록 (Ilseongrok)	0.9065	6,743,710	Kyujanggak Institute for Korean Studies

How to Use

⚠️ Important: Custom Model Architecture

This model uses a custom architecture and cannot be loaded with standard Hugging Face methods. You must use the specialized inference code from the GitHub repository.

Installation

bash

# Clone repository
git clone https://github.com/yachagye/korean-classical-chinese-punctuation
cd korean-classical-chinese-punctuation

# Install dependencies
pip install torch transformers pytorch-lightning

Quick Start (Requires GitHub Repository Code)

python

from 구두점7_추론모델 import PunctuationPredictor

# Load model (requires .ckpt checkpoint file)
predictor = PunctuationPredictor(
    checkpoint_path="path/to/checkpoint.ckpt",
    device="cuda"  # or "cpu"
)

# Predict punctuation
text = "太祖康獻大王姓李諱成桂字君晉古諱旦號松軒"
result = predictor.predict(text)
print(result)
# Output: 太祖康獻大王, 姓李, 諱成桂, 字君晉。古諱旦, 號松軒。

Processing Files

TXT files:

python 구두점7_지정_txt.py --checkpoint checkpoint.ckpt --input your_file.txt

CSV files:

python 구두점7_지정_csv.py --checkpoint checkpoint.ckpt --input your_file.csv

GUI Application (Windows)

Standalone Windows executables are available on Google Drive - no Python installation required.

v2.5 (Recommended): 한문구두점추론_v2.5.zip
v2.0: 한문구두점추론_v2.0.zip
v1.0: 한문구두점추론_v1.0.zip

Training Details

Training Hyperparameters

Parameter	Value
GPU	L40S 48GB
Batch Size (per GPU)	80
Gradient Accumulation Steps	2
Effective Batch Size	160
Learning Rate	2e-5
Learning Rate Schedule	Linear warmup + decay
Warmup Ratio	0.1
Training Epochs	3
Mixed Precision	bf16-mixed
Optimizer	AdamW
Weight Decay	0.01
Gradient Clipping	1.0
Dropout Rate	0.1
Max Sequence Length	512 tokens
Framework	PyTorch Lightning

Training Data Processing

Sliding Window Approach:

Window size: 512 tokens
Overlap: 50 tokens

Data Split:

Training: ~90% (approx. 3.4M samples)
Validation: ~10%
Random seed: 42

Data Format (JSONL):

json

{
  "c": "太祖康獻大王姓李諱成桂字君晉",
  "l": [[],[],[],[],[1],[],[],[],[],[],[0,1],...],
  "n": 15
}

Training Data Sources

Primary Sources:

National Institute of Korean History Database (국사편찬위원회)
Korean Classics Database (한국고전종합DB)
Jangseogak Digital Archive (디지털장서각)

Preprocessing Pipeline (6 stages):

Remove non-Chinese text (Korean Hangul, Japanese Kana, numbers)
Standardize punctuation marks (26 types → unified format)
Keep only Chinese characters and punctuation
Reduce to 7 primary punctuation types
Remove lines without punctuation
Remove duplicate consecutive punctuation

Limitations

Paired Brackets: Book title markers (《》) show lower performance (F1 ~0.78)
Rare Punctuation: Exclamation mark (!) shows lowest recall (0.6003) due to scarcity
Context Window: 512-token limit (longer texts processed via sliding window)
Domain Bias: Trained primarily on official records

Intended Uses

✅ Recommended Uses

Digital humanities research
Historical document digitization
Academic study of classical Chinese texts
Educational tools
Public digital archives
Cultural heritage preservation

❌ Not Recommended

Commercial products without separate licensing
Critical scholarly editions without expert review
Real-time applications requiring 100% accuracy

Disclaimer: Predictions are not perfect. For important scholarly publications, always verify with domain experts.

Citation

If you use this model in your research, please cite:

APA Style:

Yang, J. (2025). Development and Application of a Deep Learning–Based Model 
for Automated Punctuation Inference in Korean Classical Chinese. 
The Korean Journal of History (Yoksahak Yongu), 100, 267-297. 
https://doi.org/10.37924/JSSW.100.9

Yang, J. (2026). Conventions of Interpretation: Improving a Punctuation 
Prediction Model for Korean Classical Chinese. Korean Cultural Studies, 
111, 7-29. https://doi.org/10.17948/kcs.2026..111.7

BibTeX:

bibtex

@article{yang2025punctuation,
  title={Development and Application of a Deep Learning--Based Model for Automated Punctuation Inference in Korean Classical Chinese},
  author={Yang, Junghyun},
  journal={The Korean Journal of History (Yoksahak Yongu)},
  volume={100},
  pages={267--297},
  year={2025},
  publisher={Honam Historical Society},
  doi={10.37924/JSSW.100.9}
}

@article{yang2026interpretation,
  title={Conventions of Interpretation: Improving a Punctuation Prediction Model for Korean Classical Chinese},
  author={Yang, Junghyun},
  journal={Korean Cultural Studies},
  volume={111},
  pages={7--29},
  year={2026},
  publisher={Research Institute of Korean Studies, Korea University},
  doi={10.17948/kcs.2026..111.7}
}

Paper Information:

v1.0 (Prior Study)
- Journal: 역사학연구 (The Korean Journal of History)
- Volume: 100 / Pages: 267-297
- Publication Date: November 30, 2025
- Publisher: 호남사학회 (Honam Historical Society)
- DOI: 10.37924/JSSW.100.9
v2.0–v2.5 (Improvement Study)
- Journal: 민족문화연구 (Korean Cultural Studies)
- Volume: 111 / Pages: 7-29
- Publication Year: 2026
- Publisher: 고려대학교 민족문화연구원 (Research Institute of Korean Studies, Korea University)
- DOI: 10.17948/kcs.2026..111.7

License

CC BY-NC-SA 4.0 (Creative Commons Attribution-NonCommercial-ShareAlike 4.0)

✅ Permitted:

Academic research and publication
Educational use
Non-commercial projects
Modification and redistribution (with same license)

❌ Restricted:

Commercial use without separate agreement
Proprietary derivatives

📧 Commercial Licensing: yachagye@naver.com

Full license: https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode

Contact

Developer: Junghyun Yang (양정현)
Email: yachagye@naver.com
Commercial Inquiries: yachagye@naver.com

yachagye/korean-classical-chinese-punctuation