You can deploy this model and start earning money today!

Lexsi/
Orion-BiX

Name: Orion-BiX
Availability: InStock

2512.00181

mit

Views

No views yet

Orion-BiX Logo

Discord

GitHub

TabTune GitHub

Orion-BiX: Bi-Axial Meta-Learning Model for Tabular In-Context Learning

Orion-BiX is an advanced tabular foundation model that combines Bi-Axial Attention with Meta-Learning capabilities for few-shot tabular classification. The model extends the TabICL architecture with alternating attention patterns and episode-based training.

The model is part of Orion, a family of tabular foundation models with various enhancements.

Key Innovations

Bi-Axial Attention: Alternating attention patterns (Standard → Grouped → Hierarchical → Relational) that capture multi-scale feature interactions within tabular data
Meta-Learning with k-NN Support Selection: Episode-based training with intelligent support set selection using similarity metrics
Three-Component Architecture: Column embedding (Set Transformer), Bi-Axial row interaction, and In-Context Learning prediction

Architecture Overview

Input → tf_col (Set Transformer) → Bi-Axial Attention → tf_icl (ICL) → Output

Component Details:

tf_col (Column Embedder): Set Transformer for statistical distribution learning across features
Bi-Axial Attention: Replaces standard RowInteraction with alternating attention patterns:
- Standard Cross-Feature Attention
- Grouped Feature Attention
- Hierarchical Feature Attention
- Relational Feature Attention
- CLS Token Aggregation
tf_icl (ICL Predictor): In-context learning module for few-shot prediction

Usage

Direct (OrionBiX Python package)

python

from orion_bix.sklearn import OrionBiXClassifier

# Initialize and use
clf = OrionBiXClassifier()
clf.fit(X_train, y_train)
predictions = clf.predict(X_test)

This code will automatically download the pre-trained model from Hugging Face and use a GPU if available.

Via TabTune (unified TFM library)

OrionBix can be used either directly via its own Python package or through TabTune, which provides a unified interface over several tabular foundation models.

python

from tabtune import TabularPipeline

pipeline = TabularPipeline(
    model_name="OrionBix",          # use OrionBix through TabTune
    tuning_strategy="inference",    # zero-shot / in-context mode
    tuning_params={"device": "cuda"}  # or "cpu"
)

pipeline.fit(X_train, y_train)
predictions = pipeline.predict(X_test)

When used through TabTune, the OrionBiX weights are automatically downloaded from this Hugging Face repository on first use, and TabTune handles model-aware preprocessing for you.

Installation

Via TabTune (recommended if you want multiple tabular FMs)

pip install tabtune

This installs TabTune and its built-in OrionBiX integration; no separate orion-bix install is required.

From the OrionBiX source

Option 1: From the local clone

bash

cd orion-bix
pip install -e .

Option 2: From the Git Remote

pip install git+https://github.com/Lexsi-Labs/Orion-BiX.git

Citation

If you use Orion-BiX in your research, please cite our paper::

bibtex

@misc{bouadi2025orionbix,
      title={Orion-Bix: Bi-Axial Attention for Tabular In-Context Learning}, 
      author={Mohamed Bouadi and Pratinav Seth and Aditya Tanna and Vinay Kumar Sankarapu},
      year={2025},
      eprint={2512.00181},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2512.00181}, 
}

Lexsi/Orion-BiX