EDAM_all-MiniLM-L6-v2_additive_gcn_h512_o64_cosine_e128_early (Early Fusion Architecture)

Improved Architecture

• Pipeline: Transformer → TokenOntologyFusion → Pooling
• Base Model: all-MiniLM-L6-v2
• Fusion Method: additive
• Ontology Concepts: 3,511
• Model Dimension: 384

Key Improvements

1. Early Fusion: Ontology knowledge influences token representations before pooling
2. Better Concept Matching: Combined text + structural embeddings for richer concept representation
3. Adaptive Fusion: Multiple fusion methods (cross-attention, additive, gated residual)

Usage

Performance Notes

• This model should perform better than base models on domain-specific tasks
• The early fusion architecture allows better integration of structural knowledge
• Model size is preserved while adding ontological understanding