A fine-tuned SentenceTransformers bi-encoder model for semantic similarity and information retrieval. This model is specifically trained for finding relevant presentation templates based on user queries, descriptions, and metadata (industries, categories, tags) as part of the Field-Adaptive Dense Retrieval framework for structured documents.

Developed by: Mudasir Syed (mudasir13cs)

Model type: SentenceTransformer (Bi-encoder)

Language(s) (NLP): English

License: Apache 2.0

Finetuned from model: sentence-transformers/all-MiniLM-L6-v2

Paper: Field-Adaptive Dense Retrieval of Structured Documents

Model Sources

Repository: https://github.com/mudasir13cs/hybrid-search
Paper: https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE12352544
Base Model: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2

Uses

Direct Use

This model is designed for semantic search and information retrieval tasks, specifically for finding relevant presentation templates based on natural language queries. It implements field-adaptive dense retrieval for structured documents.

Downstream Use

Presentation template recommendation systems
Content discovery platforms
Semantic search engines
Information retrieval systems
Field-adaptive dense retrieval applications
Structured document search and ranking

Out-of-Scope Use

Text generation
Question answering
Machine translation
Any task not related to semantic similarity or document retrieval

Bias, Risks, and Limitations

The model is trained on presentation template data and may not generalize well to other domains
Performance may vary based on the quality and diversity of training data
The model inherits biases present in the base model and training data
Model outputs are optimized for presentation template domain

How to Get Started with the Model

python

from sentence_transformers import SentenceTransformer
import torch

# Load the model
model = SentenceTransformer("mudasir13cs/Field-adaptive-bi-encoder")

# Encode text for similarity search
queries = ["business presentation template", "marketing slides for startups"]
embeddings = model.encode(queries)

# Compute similarity
from sentence_transformers import util
cosine_scores = util.cos_sim(embeddings[0], embeddings[1])
print(f"Similarity: {cosine_scores.item():.4f}")

# For retrieval tasks
documents = [
    "Professional business strategy presentation template",
    "Modern marketing presentation for tech startups",
    "Financial report template for quarterly reviews"
]

# Encode queries and documents
query_embeddings = model.encode(queries)
doc_embeddings = model.encode(documents)

# Find most similar documents
similarities = util.cos_sim(query_embeddings, doc_embeddings)
print(f"Top matches: {similarities}")

Training Details

Training Data

Dataset: Presentation template dataset with descriptions and queries
Size: Custom dataset of presentation templates with metadata
Source: Curated presentation template collection from structured documents
Domain: Presentation templates with field-adaptive metadata

Training Procedure

Architecture: SentenceTransformer (all-MiniLM-L6-v2) with contrastive learning
Base Model: sentence-transformers/all-MiniLM-L6-v2
Loss Function: Triplet loss with hard negative mining / Multiple Negatives Ranking Loss
Optimizer: AdamW
Learning Rate: 2e-5
Batch Size: 16
Epochs: 3

Training Hyperparameters

Training regime: Supervised learning with contrastive loss
Hardware: GPU (NVIDIA)
Training time: ~2 hours
Max Sequence Length: 512 tokens

Evaluation

Testing Data, Factors & Metrics

Testing Data: Validation split from presentation template dataset
Factors: Query-description similarity, template relevance, field-adaptive retrieval performance
Metrics:
- MAP@K (Mean Average Precision at K)
- MRR@K (Mean Reciprocal Rank at K)
- NDCG@K (Normalized Discounted Cumulative Gain at K)
- Cosine similarity scores
- Recall@K

Results

MAP@10: ~0.85
MRR@10: ~0.90
NDCG@10: ~0.88
Performance: Optimized for presentation template retrieval in structured document search
Domain: High performance on field-adaptive dense retrieval tasks

Environmental Impact

Hardware Type: NVIDIA GPU
Hours used: ~2 hours
Cloud Provider: Local/Cloud
Carbon Emitted: Minimal (efficient fine-tuning)

Technical Specifications

Model Architecture and Objective

Base Architecture: Transformer-based bi-encoder (all-MiniLM-L6-v2)
Objective: Learn semantic representations for field-adaptive dense retrieval
Input: Text sequences (queries, descriptions, and metadata)
Output: 384-dimensional dense embeddings
Pooling: Mean pooling strategy

Compute Infrastructure

Hardware: NVIDIA GPU
Software: PyTorch, SentenceTransformers, Transformers

Citation

Paper:

bibtex

@article{field_adaptive_dense_retrieval,
  title={Field-Adaptive Dense Retrieval of Structured Documents},
  author={Mudasir Syed},
  journal={DBPIA},
  year={2024},
  url={https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE12352544}
}

Model:

bibtex

@misc{field_adaptive_bi_encoder,
  title={Field-adaptive Bi-encoder for Presentation Template Search},
  author={Mudasir Syed},
  year={2024},
  howpublished={Hugging Face},
  url={https://huggingface.co/mudasir13cs/Field-adaptive-bi-encoder}
}

APA: Syed, M. (2024). Field-adaptive Bi-encoder for Presentation Template Search. Hugging Face. https://huggingface.co/mudasir13cs/Field-adaptive-bi-encoder

Model Card Authors

Mudasir Syed (mudasir13cs)

Model Card Contact

GitHub: https://github.com/mudasir13cs
Hugging Face: https://huggingface.co/mudasir13cs
LinkedIn: https://pk.linkedin.com/in/mudasir-sayed

Framework versions

SentenceTransformers: 2.2.2+
Transformers: 4.35.0+
PyTorch: 2.0.0+