This is an OnT (Ontology Transformer) model trained on the ANATOMY dataset, based on sentence-transformers/all-MiniLM-L12-v2. OnT is a language model-based framework for ontology embeddings, enabling effective representation of concepts as points in hyperbolic space and axioms as hierarchical relationships between concepts.

Model Details

Model Description

Model Type: Ontology Transformer (OnT)
Base model: sentence-transformers/all-MiniLM-L12-v2
Training Dataset: ANATOMY
Maximum Sequence Length: 256 tokens
Output Dimensionality: 384 dimensions
Embedding Space: Hyperbolic Space
Key Features:
- Hyperbolic embeddings for ontology concept encoding
- Modeling of hierarchical relationships between concepts
- Support for role embeddings as rotations over hyperbolic spaces
- Concept rotation, transition, and existential quantifier representation

Model Sources

Repository: OnT on GitHub
Paper: Language Models as Ontology Encoders

Available Versions

This model is available in 4 versions (Git branches) to suit different use cases:

Branch	Training Type	Role Embedding	Use Case
`main` (default)	Prediction Dataset	✅ With role embedding	Default version: training on prediction dataset, support role embedding
`role-free`	Prediction Dataset	❌ Without role embedding	Training on prediction dataset, without role embedding
`inference-default`	Inference Dataset	✅ With role embedding	Training on inference dataset, with role support
`inference-role-free`	Inference Dataset	❌ Without role embedding	Training on inference dataset, without role embeddings

How to use different versions:

python

from OnT import OntologyTransformer

# Default version (main branch - OnTr with role embedding)
ont = OntologyTransformer.from_pretrained("Hui97/OnT-MiniLM-L12-anatomy")

# Role-free version (without role embedding)
ont = OntologyTransformer.from_pretrained("Hui97/OnT-MiniLM-L12-anatomy", revision="role-free")

# Inference version with role embedding
ont = OntologyTransformer.from_pretrained("Hui97/OnT-MiniLM-L12-anatomy", revision="inference-default")

# Inference version without role embedding
ont = OntologyTransformer.from_pretrained("Hui97/OnT-MiniLM-L12-anatomy", revision="inference-role-free")

Full Model Architecture

OntologyTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Installation

First, install the required dependencies:

pip install sentence-transformers==3.4.0.dev0

You also need to install HierarchyTransformers following the instructions in their repository.

Direct Usage

Load the model and use it for ontology concept encoding:

python

import torch
from OnT import OntologyTransformer

# Load the OnT model
path = "Hui97/OnT-MiniLM-L12-anatomy"
ont = OntologyTransformer.from_pretrained(path)

# Entity names to be encoded
entity_names = [
    'alveolar atrium',
    'organ part',
    'superior recess of lesser sac',
]

# Get the entity embeddings in hyperbolic space
entity_embeddings = ont.encode_concept(entity_names)
print(entity_embeddings.shape)
# [3, 384]

# Role sentences to be encoded
role_sentences = [
    "application attribute",
    "attribute",
    "chemical modifier"
]

# Get the role embeddings (rotations and scalings)
role_rotations, role_scalings = ont.encode_roles(role_sentences)

Citation

BibTeX

If you use this model, please cite:

bibtex

@article{yang2025language,
  title={Language Models as Ontology Encoders},
  author={Yang, Hui and Chen, Jiaoyan and He, Yuan and Gao, Yongsheng and Horrocks, Ian},
  journal={arXiv preprint arXiv:2507.14334},
  year={2025}
}