yeseul0-0/

docred-stage3-atlop-fine-tuning-v6

Evaluation Results (Stage 3 — GAIN-lite)

Dataset Statistics

• Documents: 998
• Predicted Triples: 10,484
• Gold Triples: 12,275
• Correct Predictions: 6,877

Main Metrics

• Micro F1: 60.43%
  • Precision: 65.60%
  • Recall: 56.02%
• Ign F1: 58.22%
  • Precision: 64.06%
  • Recall: 53.36%
• Evidence F1: 42.28%
  • Precision: 40.93%
  • Recall: 43.71%

Detailed Analysis

• Intra-sentence F1: 67.53%
  • Precision: 72.19%
  • Recall: 63.44%
• Inter-sentence F1: 51.67%
  • Precision: 57.17%
  • Recall: 47.13%

Interpretation

• The Micro F1 (60.43%) indicates stable overall performance in multi-label relation extraction.
• The Ign F1 (58.22%) confirms that the model generalizes reasonably well beyond memorized relational facts.
• The Evidence F1 (42.28%) shows that evidence prediction remains a challenging task, though it provides meaningful interpretability signals.

• Intra-sentence F1 (67.53%)
• Inter-sentence F1 (51.67%)

• The model performs significantly better on local (within-sentence) relations
• While cross-sentence (multi-hop) reasoning is improved, it still remains the main bottleneck

Key Insight (Stage 2 → Stage 3)

• ✔ Improved modeling of entity interactions
• ✔ Better handling of cross-sentence dependencies
• ❗ Still limited performance on long-range reasoning (Inter F1 gap)

Output Files

• Predictions: results/stage3/v6/predictions.json
• Submission File: results/stage3/v6/result.json

Notes

• This model is evaluated under the DocRED standard evaluation protocol
• Results may vary depending on threshold tuning and preprocessing pipeline
• Inter-sentence performance highlights the importance of graph-based reasoning modules