This is a fine-tuned cross-encoder model specifically optimized for Scientific Fact Verification tasks, trained as part of the PathfinderRAG research project.

Model Details

Model Type: Cross-Encoder for Regression (continuous similarity scores)
Base Model: cross-encoder/ms-marco-MiniLM-L-6-v2
Training Dataset: SciFact (Scientific claim verification against research papers)
Task: Scientific Fact Verification
Library: sentence-transformers
License: MIT

Performance Metrics

Final Regression Metrics

Metric	Value	Description
MSE	0.100697	Mean Squared Error (lower is better)
MAE	0.183992	Mean Absolute Error (lower is better)
RMSE	0.317328	Root Mean Squared Error (lower is better)
R² Score	0.401894	Coefficient of determination (higher is better)
Pearson Correlation	0.758721	Linear correlation (higher is better)
Spearman Correlation	0.709262	Rank correlation (higher is better)

Training Details

Training Duration: 32 minutes
Epochs: 10
Early Stopping: No
Best Correlation Score: 0.689202
Final MSE: 0.100697

Training Configuration

Batch Size: 12
Learning Rate: 1.5e-05
Max Epochs: 10
Weight Decay: 0.02
Warmup Steps: 200

Usage

This model can be used with the sentence-transformers library for computing semantic similarity scores between query-document pairs.

Installation

pip install sentence-transformers

Basic Usage

python

from sentence_transformers import CrossEncoder

# Load the model
model = CrossEncoder('OloriBern/trailrag-cross-encoder-scifact-enhanced')

# Example usage
pairs = [
    ['What is artificial intelligence?', 'AI is a field of computer science focused on creating intelligent machines.'],
    ['What is artificial intelligence?', 'Paris is the capital of France.']
]

# Get similarity scores (continuous values, not binary)
scores = model.predict(pairs)
print(scores)  # Higher scores indicate better semantic match

Advanced Usage in PathfinderRAG

python

from sentence_transformers import CrossEncoder

# Initialize for PathfinderRAG exploration
cross_encoder = CrossEncoder('OloriBern/trailrag-cross-encoder-scifact-enhanced')

def score_query_document_pair(query: str, document: str) -> float:
    """Score a query-document pair for relevance."""
    score = cross_encoder.predict([[query, document]])[0]
    return float(score)

# Use in document exploration
query = "Your research query"
documents = ["Document 1 text", "Document 2 text", ...]

# Score all pairs
scores = cross_encoder.predict([[query, doc] for doc in documents])
ranked_docs = sorted(zip(documents, scores), key=lambda x: x[1], reverse=True)

Training Process

This model was trained using regression metrics (not classification) to predict continuous similarity scores in the range [0, 1]. The training process focused on:

Data Quality: Used authentic SciFact examples with careful contamination filtering
Regression Approach: Avoided binary classification, maintaining continuous label distribution
Correlation Optimization: Maximized Spearman correlation for effective ranking
Scientific Rigor: All metrics derived from real training runs without simulation

Why Regression Over Classification?

Cross-encoders for information retrieval should predict continuous similarity scores, not binary classifications. This approach:

Preserves fine-grained similarity distinctions
Enables better ranking and document selection
Provides more informative scores for downstream applications
Aligns with the mathematical foundation of information retrieval

Dataset

SciFact: Scientific claim verification against research papers

Task Type: Scientific Fact Verification
Training Examples: 1,000 high-quality pairs
Validation Split: 20% (200 examples)
Quality Threshold: ≥0.70 (authentic TrailRAG metrics)
Contamination: Zero overlap between splits

Limitations

Optimized specifically for scientific fact verification tasks
Performance may vary on out-of-domain data
Requires sentence-transformers library for inference
CPU-based training (GPU optimization available for future versions)

Citation

bibtex

@misc{trailrag-cross-encoder-scifact,
  title = {TrailRAG Cross-Encoder: SciFact Enhanced},
  author = {PathfinderRAG Team},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/OloriBern/trailrag-cross-encoder-scifact-enhanced}
}

Model Card Contact

For questions about this model, please open an issue in the PathfinderRAG repository or contact the development team.

This model card was automatically generated using the TrailRAG model card generator with authentic training metrics.