unknown samples are excluded from training. Ambiguous or low-confidence predictions should be handled by the application using a confidence threshold.

Model Details

Model Description

This model is a fine-tuned KLUE-RoBERTa based text classification model for CaptureMate, an iOS screenshot organization and action recommendation app.

The model receives OCR text extracted from screenshots and predicts the most relevant screenshot category.

Developed by: CaptureMate
Model type: Text classification
Language(s): Korean, English, mixed OCR text
License: Not specified
Finetuned from model: KLUE-RoBERTa base
Number of labels: 4

Labels

ID	Label	Description
0	schedule	Schedule, reservation, ticket, event, or date-related screenshots
1	shopping	Shopping, product, price, payment, or commerce-related screenshots
2	place	Place, map, restaurant, store, travel, or location-related screenshots
3	memo	Text, article, note, content, or general information screenshots

Uses

Direct Use

Use this model to classify OCR-extracted screenshot text into CaptureMate categories.

Downstream Use

This model can be used inside a screenshot processing pipeline:

txt

Screenshot
-> OCR text extraction
-> Text preprocessing
-> Category classification
-> Category-specific action recommendation

Out-of-Scope Use

This model is not designed to:

Classify original images directly
Perform OCR
Extract structured fields such as dates, prices, addresses, or product names
Predict an unknown category directly
Classify content outside the CaptureMate screenshot domain

Bias, Risks, and Limitations

The model is trained on a small CaptureMate-specific screenshot OCR dataset. It may not generalize well to unrelated domains or OCR text from different user behavior patterns.

Known limitations:

The model always predicts one of the four labels.
There is no direct unknown output.
Very short or noisy OCR text can lead to unstable predictions.
Ambiguous cases such as ticket screenshots, food blog screenshots, or shopping-like memo content may be confused.

Recommendations

Use a confidence threshold at inference time.

Example:

txt

if confidence < 0.6:
    category = "unknown"
else:
    category = predicted_label

How to Get Started with the Model

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "YOUR_USERNAME/capturemate-category-classifier-v2-4class"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "2025.4.7 티켓 수령 공연 예약 정보"

inputs = tokenizer(
    text,
    return_tensors="pt",
    truncation=True,
    max_length=256
)

with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.softmax(outputs.logits, dim=-1)
    pred_id = probs.argmax(dim=-1).item()
    confidence = probs[0][pred_id].item()

label = model.config.id2label[pred_id]

print(label, confidence)

Training Details

Training Data

The model was trained on OCR text extracted from screenshot images collected for the CaptureMate project.

Dataset split:

Split	Samples
Train	340
Validation	73
Test	74

unknown samples were removed before creating the final training, validation, and test splits.

Training Procedure

Preprocessing

OCR text was extracted from screenshots.
Empty text samples were removed.
Labels outside the four target categories were excluded.
Text was tokenized with the KLUE-RoBERTa tokenizer.
Maximum sequence length: 256 tokens.

Training Hyperparameters

Hyperparameter	Value
Learning rate	1e-5
Epochs	8
Train batch size	16
Eval batch size	16
Warmup ratio	0.1
Weight decay	0.01
Best model metric	macro F1
Seed	42

Evaluation

Testing Data, Factors & Metrics

Testing Data

The test set contains 74 OCR text samples across four categories.

Metrics

The model was evaluated using:

Accuracy
Macro F1
Macro precision
Macro recall

Macro metrics are important because the dataset is not perfectly balanced across categories.

Results

Metric	Validation	Test
Accuracy	98.63%	94.59%
Macro F1	98.89%	94.55%
Macro Precision	99.07%	93.91%
Macro Recall	98.75%	95.47%

Test Set Per-Class Results

Class	Precision	Recall	F1-score	Support
schedule	90.00%	100.00%	94.74%	9
shopping	96.15%	96.15%	96.15%	26
place	94.74%	100.00%	97.30%	18
memo	94.74%	85.71%	90.00%	21

Confusion Matrix

Rows are true labels, columns are predicted labels.

True \ Pred	schedule	shopping	place	memo
schedule	9	0	0	0
shopping	0	25	0	1
place	0	0	18	0
memo	1	1	1	18

Technical Specifications

Model Architecture and Objective

Architecture: KLUE-RoBERTa sequence classification
Objective: Single-label multi-class classification
Input: OCR text
Output: One of four CaptureMate categories

Software

transformers
datasets
torch
scikit-learn

Model Card Authors

CaptureMate

Model Card Contact

CaptureMate project maintainer

hur03/capturemate-category-classifier-v2-4class

CaptureMate Category Classifier v2 4-Class