⚠️ Do not use this model as evidence of AI authorship.
Munche-768-AI-Detector was built for dataset triage and proof-of-concept research. It cannot establish plagiarism, copyright infringement, misconduct, or whether a person or an AI wrote a text. Its output can be wrong.
Munche-768-AI-Detector
Baragi AI
Munche-768-AI-Detector classifies Korean genre-fiction passages as human, uncertain, or llm. It starts from Munche-768, then jointly tunes LoRA weights in the top four Transformer layers and a 769-parameter linear classifier.
The sealed test combines 277 human passages and 256 LLM passages from the independent-generation and content-preserving rewrite evaluations.
Metric
Result
AUROC
98.59%
Binary accuracy
94.00%
Balanced accuracy
93.91%
Human recall
96.03%
LLM recall
91.80%
Human false-positive rate
3.97%
Overall binary confusion matrix across independent generation and content-preserving rewrites
Independent-generation test
The sealed test contains 87 passages from human-written novels and 66 passages written directly by 11 language-model families. Content-preserving rewrites are excluded from this evaluation.
Metric
Result
Binary accuracy
100.00%
Balanced accuracy
100.00%
Human recall
100.00%
LLM recall
100.00%
Human false-positive rate
0.00%
Binary confusion matrix for independently written human and LLM fiction
Three-way decision
The thresholds were selected on validation data only. The classification score is not a calibrated probability that a passage was written by AI. Results below cover the full sealed test.
Score
Output
≤ 0.5000
Human
0.5000 < score < 0.8697
Uncertain
≥ 0.8697
LLM
Metric
Result
Coverage
93.62%
Accuracy among classified passages
95.19%
Human classified as LLM
1.08%
Human classified as uncertain
2.89%
LLM classified as human
8.20%
LLM classified as uncertain
10.16%
Within the independent-generation subset, all 87 human passages received a human decision. Of the 66 LLM passages, 64 received an LLM decision and two were uncertain.
Human, uncertain, and LLM outcomes for the independent-generation test
Content-preserving rewrite test
This test contains 190 human passages and 190 LLM rewrites that preserve the source content. It is harder than distinguishing independently written human and LLM fiction.
Metric
Binary decision
Three-way decision
AUROC
97.47%
N/A
Balanced accuracy
91.58%
N/A
Human recall
94.21%
N/A
LLM recall
88.95%
N/A
Human classified as LLM
5.79%
1.58%
LLM classified as human
11.05%
11.05%
Coverage
N/A
91.58%
Accuracy among classified passages
N/A
93.10%
Performance comparison between independent generation and content-preserving rewrites
Input length
Use passages between 384 and 2,048 EmbeddingGemma tokens. Inputs shorter than 384 tokens are not supported as stable operating inputs. Inputs longer than 2,048 tokens must be divided into separate windows before classification.
The training and evaluation corpora covered short rewrite passages near 400 tokens and independent fiction passages near the 2,048-token model limit. Document-level aggregation across multiple windows has not been calibrated.
Usage
Access to the gated EmbeddingGemma base model is required.
The training set combines human-written Korean genre fiction, independently generated LLM fiction, and content-preserving LLM rewrites. GPT-5.6 Sol Medium contributes 24 training passages and six validation passages. No Sol Medium passage was added to test.
The detector was initialized from Munche-768. Only LoRA weights in Transformer layers 20-23 and the linear classifier were updated. The selected checkpoint is step 275. A preservation loss limited movement away from the original Munche-768 embedding during tuning.
Raw human fiction is not distributed with this repository.
Limitations
The model was trained and evaluated on Korean genre fiction.
Generalization to language-model families absent from training remains unknown.
The model cannot identify text jointly written or substantially edited by humans and AI.
License
Munche-768-AI-Detector is derived from google/embeddinggemma-300m and Munche-768. Use is subject to the Gemma license and the access terms of the gated base model.
