yeseul0-0/

docred-stage3-atlop-fine-tuning-v5

Evaluation Results (Stage 3 — GAIN-lite, v5)

Dataset Statistics

• Documents: 998
• Predicted Triples: 10,671
• Gold Triples: 12,275
• Correct Predictions: 6,929

Main Metrics

• Micro F1: 60.39%
  • Precision: 64.93%
  • Recall: 56.45%
• Ign F1: 58.17%
  • Precision: 63.42%
  • Recall: 53.72%
• Evidence F1: 42.45%
  • Precision: 40.91%
  • Recall: 44.12%

Detailed Analysis

• Intra-sentence F1: 67.29%
  • Precision: 72.02%
  • Recall: 63.14%
• Inter-sentence F1: 52.06%
  • Precision: 56.28%
  • Recall: 48.42%

Interpretation

• The Micro F1 (60.39%) indicates consistent overall performance.
• The Ign F1 (58.17%) suggests reasonable generalization beyond memorized relations.
• The Evidence F1 (42.45%) shows that evidence prediction is functional but still remains a challenging component.

• Intra F1 (67.29%)
• Inter F1 (52.06%)

• The model is strong at within-sentence relation extraction
• But cross-sentence (multi-hop) reasoning still has room for improvement

Key Insight

Output Files

• Predictions: results/stage3/v5/predictions.json
• Submission File: results/stage3/v5/result.json