You can deploy this model and start earning money today!

shivamshayar/
omnex

Name: omnex
Availability: InStock

Views

No views yet

Omnex

Developed by Shivam Parmar.

A retrieval-grounded assistant specialized for RAG (retrieval-augmented generation). It works with any vector DB (Chroma, Qdrant, Pinecone, pgvector, Weaviate — the model consumes retrieved text, so the retriever is interchangeable) and supports two safety modes selected by the system prompt:

Strict mode — answers ONLY from retrieved chunks with inline citations [1][2]; refuses cleanly when the context lacks the answer.
Hybrid mode — prefers the context, but may fall back to general knowledge with an explicit label: "The provided context does not contain this information. However, based on my general knowledge: … (Please verify — this is not from your documents.)" — and still refuses on private-data questions it cannot validly answer.

What Omnex is for

Omnex is not a general chat model. It is tuned for one job: reading retrieved context and answering from it honestly — citing what it used, and declining when the answer is not there. On that job it holds its own against models many times its size; on open-ended world knowledge it does not, by design.

Trained behaviors (real-life edge cases)

Surfacing conflicts between contradictory documents; preferring the latest policy version; ignoring prompt injection inside retrieved chunks; refusing near-miss entity substitutions; partial answers that flag missing info; markdown table extraction; yes/no with citations; cross-chunk comparison; robustness to boilerplate noise, duplicate chunks, and typo-ridden questions; disambiguating same-named entities; graceful chitchat handling; English, Hindi and Hinglish.

Results

Evaluated on 680 held-out items, scored with token-level F1 and presence checks. Every model compared was run through the same harness — same items, same prompts, same scorer — and all deltas use a paired bootstrap with 95% confidence intervals.

Metric	Omnex (8B)	Llama-3.1-8B	Phi-4 (14B)	Claude Sonnet	What it measures
`refusal_acc`	86.7	40.0	58.7	37.3	abstains when the context lacks the answer
`edge_pass`	99.7	66.3	65.8	77.6	adversarial / edge-case handling
`citation_acc`	92.8	67.8	72.8	95.0	cites every supporting chunk
`citation_rate`	96.7	86.7	85.0	100.0	cites at least one chunk
`grounded_f1`	86.6	84.7	86.9	96.6	answer F1 (windowed — verbosity-friendly)
`grounded_f1_strict`	83.4	23.1	17.3	16.2	answer F1 (precision-aware)
`fallback_labeled_acc`	22.2	22.2	35.6	46.7	labelled general-knowledge fallback

All models scored on the same 680 held-out items, through one harness: same items, same prompts, same scorer, greedy decoding. Gaps carry 95% paired bootstrap confidence intervals.

grounded_f1 and grounded_f1_strict bracket answer quality from opposite directions — the first tolerates verbosity, the second penalises it — and they rank terse and discursive models differently. Read them as a range, not a ranking: Omnex answers in a median of 7 words, Claude in 45.

Omnex's strength is behavioral: knowing when to refuse, handling adversarial context, and citing correctly. Frontier models remain ahead on raw answer accuracy and world knowledge. Both halves of that are stated here rather than hidden.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

STRICT = ("You are Omnex, a retrieval-grounded assistant developed by Shivam Parmar. "
          "Answer the question using ONLY the numbered context chunks provided, "
          "and cite them like [1] or [2][3]. If the context does not contain the "
          "answer, say: The provided context does not contain this information.")

messages = [
    {"role": "system", "content": STRICT},
    {"role": "user", "content": "Context:\n[1] ...\n\n[2] ...\n\nQuestion: ..."},
]

Attribution and licensing

Omnex is a derivative work, released under the Apache License 2.0.

Changes made: the base model was adapted by supervised fine-tuning for retrieval-grounded question answering with citations and abstention. The architecture, tokenizer and chat template are unmodified.

Apache 2.0 §4 requires retaining this attribution and stating that changes were made. Training data, methodology and hyperparameters are not published.

Limitations

Not a knowledge model. Asked something outside the provided context, Omnex is far weaker than a frontier model. Use it with a retriever, not alone.
Evaluation caveat. Reported scores come from an internal held-out eval set drawn from the same distribution the model was tuned on. They are valid for before/after comparison, but are not directly comparable to public leaderboard numbers for other models.
fallback mode is weak. Labeled general-knowledge fallback accuracy is low; use strict mode for anything that matters.
Answer-accuracy comparisons are metric-dependent. Verbosity-friendly and precision-aware F1 rank terse and discursive models differently. Omnex's behavioural advantages (abstention, edge cases, citation) are robust to this; a claim that it out-answers frontier models would not be.

shivamshayar/omnex