Checkpoint update notice: The current checkpoint is expected to be updated within the next few hours. After that initial refresh, OTel 2.0 checkpoints are expected to continue receiving weekly weight updates. For reproducible evaluation or production deployment, pin a specific model revision, checkpoint hash, or release tag.

OTel-2.0-LLM-31B-IT is a telecom-specialized instruction model post-trained from Gemma 4 31B-IT on approximately 440 billion telecom training tokens. It is the first release in the OTel 2.0 family and is designed to support telco-grade AI workflows across network operations, standards interpretation, product development, network configuration assistance, RAG, and telecom-specific question answering.

OTel 2.0 extends the original OTel effort from a RAG-oriented telecom fine-tuning release into a larger domain-adapted training program. The model was trained from a much larger standards and telecom corpus, with new data preparation coverage for direct telecom QnA, abstention, RAG, base-model-style telecom data, and general-purpose instruction-following and tool-calling examples. The current training mixture does not include telecommunications-specific MCP, tool-calling, or instruction-following examples.

Release Status

OTel 2.0 models are expected to continue training after release, and weights may be updated weekly. For reproducible evaluation or production deployment, pin a specific model revision, checkpoint hash, or release tag rather than relying on the floating latest weights.

Model Details

Attribute	Value
Base model	Gemma 4 31B-IT
Parameters	31B
Model family	OTel 2.0
Training method	Telecom domain post-training via OSFT (Orthogonal Subspace Fine-Tuning, Red Hat AI Innovation Team)
OSFT configuration	`unfreeze_rank_ratio=0.35`, 2 epochs, effective batch size 512, lr 2e-5, cosine schedule, max sequence length 4096
Released format	Merged bf16 HuggingFace checkpoint (`Gemma4ForConditionalGeneration`), ~62.6 GB
Modality	Architecturally multimodal (text + image); trained modality is text only — see Vision and Modality
Raw telecom corpus	~15B tokens from GSMA through Open Telco AI
Processed data volume	>1T tokens processed using Red Hat's open-source Synthetic Data Generation Hub (SDG Hub)
Training tokens	~440B
Compute for data processing	~530 GPUs through Microsoft Azure Managed Compute, primarily AMD MI300X
Model training infrastructure	On-premises AMD MI355X GPUs with Dell Technologies infrastructure and servers
Day 0 inference availability	Microsoft Foundry, Featherless AI, and Red Hat
Supported interaction	English-language text; image input structurally supported but not domain-tuned

Repository format: safetensors, bfloat16, 15 shards, architecture Gemma4ForConditionalGeneration — 832 language tensors, 355 vision_tower tensors, and the embed_vision projector (1,188 total). See Vision and Modality for what the vision path is and is not.

Model Lineage

Gemma 4 31B-IT -> OTel 2.0 telecom data processing -> OSFT post-training (orthogonal subspace, unfreeze_rank_ratio=0.35) -> OTel-2.0-LLM-31B-IT

OSFT Training Details

This checkpoint was produced with OSFT (Orthogonal Subspace Fine-Tuning), the continual-learning algorithm from the Red Hat AI Innovation Team (Nayak et al., 2025), via training_hub on the RHAI mini-trainer backend.

What OSFT does. Rather than training an adapter alongside frozen weights (LoRA/QLoRA) or updating every direction of every weight matrix (standard full fine-tuning), OSFT decomposes the target weight matrices and learns in a subspace orthogonal to the directions carrying the base model's existing knowledge. The intent is to absorb new domain knowledge while leaving prior capability largely undisturbed — continual learning without catastrophic forgetting, and without needing the original training data as a replay set.

The one knob that matters. unfreeze_rank_ratio sets how much of each weight matrix is opened for adaptation. Lower values (0.1–0.3) preserve more of the original behavior; higher values (0.5–0.8) allow more adaptation. This model used 0.35 — deliberately toward the preservation end, because the goal was to add telecom domain knowledge to an instruction-tuned model without degrading its general instruction-following.

Parameter	Value
`unfreeze_rank_ratio`	0.35
Epochs	2
Effective batch size	512
Learning rate	2e-5, cosine schedule
Max sequence length	4096
Loss masking	assistant turns only for SFT rows; full-document for knowledge rows
EOS token	`<turn
Precision	FP32 master weights, BF16 compute; 8-bit Adam optimizer states
Hardware	8× AMD MI355X (gfx950), ROCm 7.2.1

Relationship to the QLoRA variant. OTel-2.0-LLM-31B-IT-QLoRA trains a LoRA adapter on a 4-bit NF4 base and merges it down to bf16. This model instead adapts the bf16 base directly in a constrained subspace. Both ship as standalone bf16 HuggingFace checkpoints with no quantization or adapter dependency at inference; they differ in how the weights were reached, and are best compared by evaluation on your own workload rather than by method alone.

Precision note. The optimizer states were stored in 8-bit (torchao AdamW8bit) while master weights, the subspace decomposition, and gradient reduction remained FP32. In controlled ablation this was loss-neutral and reduced peak memory; master-weight precision is the part that must stay FP32 for the model to fit the new task.

What Changed From OTel 1.0

Category	Metric	OTel 1.0	OTel 2.0
Tokens	Raw documents	~600M	~15B
Tokens	Training	~1B	~440B
Data preparation	General-purpose agentic tool calling	No	Yes
Data preparation	Knowledge / fact direct QnA	No	Yes
Data preparation	RAG	Yes	Yes
Data preparation	Abstention	Yes	Yes
Data preparation	General-purpose instruction following	No	Yes
Data preparation	Base model training set	No	Yes

Relative to OTel 1.0, OTel 2.0 increases raw source coverage by roughly 25x and training-token volume by roughly 440x. More importantly, the data mixture is broader: OTel 1.0 focused primarily on context-grounded RAG and abstention, while OTel 2.0 adds direct telecom knowledge QnA, general-purpose instruction-following and tool-use-oriented examples, and base-model-style telecom training data. Telecommunications-specific MCP, tool-calling, and instruction-following examples are not part of the current training mixture.

Training Data

The starting corpus contains approximately 15 billion raw tokens provided by GSMA through Open Telco AI. The corpus includes telecom standards and technical material from seven standards development and industry organizations:

Source family	Examples
Cellular standards	3GPP
Telecom standards and specifications	ETSI, ITU
Industry and operator materials	GSMA
Network API specifications	CAMARA
Open RAN specifications	O-RAN
Telecom business and operations frameworks	TM Forum

Dense technical specifications from these sources were converted into material suitable for model training. The raw corpus was processed into over 1 trillion tokens using Red Hat's open-source Synthetic Data Generation Hub (SDG Hub) on Microsoft Azure Managed Compute, using approximately 530 GPUs, primarily AMD MI300X. From this processed pool, approximately 440 billion training tokens were generated for OTel 2.0 post-training. Model training ran on on-premises AMD MI355X GPUs with Dell Technologies infrastructure and servers.

Data Preparation Capabilities

OTel 2.0 includes a broader supervised and synthetic-data mixture than OTel 1.0:

Capability	Description
RAG	Context-grounded telecom answer generation from retrieved standards and technical documents
Abstention	Training examples that teach the model to avoid answering when context is missing, irrelevant, or insufficient
Direct telecom QnA	Knowledge and factual question-answer pairs for standards, protocols, services, and network concepts
Instruction following	General-purpose instruction-following examples; the current mixture does not include telecommunications-specific instruction-following examples
Agentic tool calling	General-purpose tool-calling examples; the current mixture does not include telecommunications-specific MCP or tool-calling examples

Intended Use

OTel-2.0-LLM-31B-IT is intended for telecom-focused applications where domain knowledge, standards familiarity, and deployment control matter. Suitable use cases include:

Retrieval-Augmented Generation over telecom standards, specifications, and internal technical documentation.
Standards interpretation and summarization for 3GPP, ETSI, GSMA, CAMARA, ITU, O-RAN, and TM Forum materials.
Product development, network configuration assistance, and engineering support tasks.
Telecom-specific direct QnA where the model has been separately evaluated for the target benchmark or application.
Agentic workflows where a larger system provides verified tools, retrieval, validation, and audit logging.

For high-impact operational use, the model should be deployed with retrieval, source attribution, validation checks, and human review appropriate to the system risk.

Not a Retrieval or Reranking Model

OTel 2.0 LLM checkpoints are generative text models. They were not trained for embedding, retrieval, or reranking, and should not be used to produce vector embeddings, score query-document relevance, or reorder candidate passages. The training mixture contains no contrastive, embedding, or cross-encoder reranking objectives, and no such capability has been evaluated.

In a RAG pipeline this model is the generation stage only. Use purpose-built models for the other stages:

Stage	Use
Embedding / retrieval	OTel Embedding Collection
Reranking	OTel Reranker Collection
Generation over retrieved context	this model

Prompting a generative model to emit relevance scores is not a substitute for a trained reranker; scores produced that way are unvalidated and should not be treated as calibrated.

Serving with vLLM

This section is the deployment recipe used for this checkpoint. Follow it as written — several of the settings are requirements rather than preferences, and each one is explained below.

0. Quick start

If you just want a working server, this is the whole thing. Sections 1–8 explain each choice and cover the failure modes.

bash

# 1. download
huggingface-cli download farbodtavakkoli/OTel-2.0-LLM-31B-IT \
    --local-dir ./OTel-2.0-LLM-31B-IT

# 2. serve  (NVIDIA; for AMD see §2)
docker run --rm --gpus all --ipc=host --shm-size 64G -p 8000:8000 \
  -v "$PWD/OTel-2.0-LLM-31B-IT:/model:ro" \
  vllm/vllm-openai:latest \
  vllm serve /model \
    --served-model-name OTel-2.0-LLM-31B-IT \
    --tensor-parallel-size 2 \
    --max-model-len 8192 \
    --dtype bfloat16 \
    --port 8000

# 3. confirm it is really this model, not the base
curl -sS http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"OTel-2.0-LLM-31B-IT",
       "messages":[{"role":"user","content":"What model is this?"}],
       "temperature":0}' | python3 -c "import json,sys; print(json.load(sys.stdin)['choices'][0]['message']['content'])"

Step 3 should identify OTel 2.0, trained by AT&T Chief Data Office. If it answers "Gemma" or "Google DeepMind", you are serving the base model — wrong path or a stale mount. An HTTP 200 alone does not prove the deployment is correct.

Three things that are requirements, not preferences:

On AMD gfx950, set VLLM_ROCM_USE_AITER=0 — the default path can return malformed text while still answering HTTP 200 (§2).
No system prompt is needed for correct self-identification; if you add one, see §4 for how it must be merged.
Use the chat endpoint, or replicate its template exactly — /v1/completions needs "add_special_tokens": false to avoid a double <bos> (§4, §8).

1. Requirements

Gemma 4 is a recent architecture. Your runtime must be new enough to contain it:

Component	Minimum	Used for this recipe
vLLM	≥ 0.19.0	0.23.1
transformers	≥ 5.5	5.14.0
PyTorch	—	2.11.0

An older stack fails at load with:

ValueError: The checkpoint you are trying to load has model type `gemma4_text`
but Transformers does not recognize this architecture.

Confirm support before pulling a multi-gigabyte image:

bash

python3 -c "from vllm.model_executor.models.registry import ModelRegistry as R; \
print('Gemma4ForConditionalGeneration' in R.get_supported_archs())"

That is the architecture this repository ships. Gemma4ForCausalLM is the text-only Gemma 4 class — a runtime that registers only that one cannot serve this checkpoint.

Note that an image tagged latest, main, or nightly is not a guarantee of a recent vLLM build — check the registry rather than the tag.

2. Environment

AMD (ROCm)

The container route used for this checkpoint:

bash

docker run --rm --name otel_vllm \
  --device /dev/kfd --device /dev/dri \
  --group-add "$(getent group video | cut -d: -f3)" \
  --ipc=host --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
  --shm-size 64G -p 8000:8000 \
  -e VLLM_ROCM_USE_AITER=0 \
  -v /path/to/OTel-2.0-LLM-31B-IT:/model:ro \
  rocm/vllm:rocm7.14.0_cdna_ubuntu24.04_py3.14_pytorch_2.11.0_vllm_0.23.0 \
  vllm serve /model \
    --served-model-name OTel-2.0-LLM-31B-IT \
    --tensor-parallel-size 1 \
    --max-model-len 8192 \
    --dtype bfloat16 \
    --port 8000

VLLM_ROCM_USE_AITER=0 is required on gfx950. AITER is the default on that architecture and has a GEMM path that can return malformed text while the server still answers HTTP 200 — a failure that does not surface as an error. Set it to 0. VLLM_ROCM_USE_AITER_MOE=0 is not a substitute; this model is dense.

NVIDIA (CUDA)

Only the container flags differ — the model, template, prompting and sampling settings below are identical on both vendors:

bash

docker run --rm --gpus all --ipc=host --shm-size 64G -p 8000:8000 \
  -v /path/to/OTel-2.0-LLM-31B-IT:/model:ro \
  vllm/vllm-openai:latest \
  vllm serve /model \
    --served-model-name OTel-2.0-LLM-31B-IT \
    --tensor-parallel-size 2 \
    --max-model-len 8192 \
    --dtype bfloat16 \
    --port 8000

vllm/vllm-openai:latest ships vLLM 0.28.0 and registers both Gemma4ForCausalLM and Gemma4ForConditionalGeneration, so it satisfies the §1 requirement.

Differences from the ROCm invocation, and nothing else:

	ROCm	CUDA
device flags	`--device /dev/kfd --device /dev/dri --group-add video`	`--gpus all`
device selection	`HIP_VISIBLE_DEVICES`	`CUDA_VISIBLE_DEVICES`
`VLLM_ROCM_USE_AITER=0`	required	omit — ROCm-only
`--ipc=host`, `--shm-size`	keep	keep (needed for tensor parallelism)

--tensor-parallel-size is a sizing choice, not a vendor one; see §3.

3. Sizing

Weights occupy roughly 62 GB in bfloat16, before KV cache and activation workspace.

GPU	Suggested start
1× 192 GB / 288 GB (MI300X, MI355X)	`--tensor-parallel-size 1`
2× 80 GB (A100, H100)	`--tensor-parallel-size 2`
1× 80 GB	`--tensor-parallel-size 1`, reduce `--max-model-len` and `--max-num-seqs`; concurrency will be limited

--max-model-len must cover prompt plus generated tokens. Raise it only as far as your workload needs — a larger value reserves KV cache that could otherwise serve concurrent requests. Tune throughput with --max-num-seqs, and use --gpu-memory-utilization (default 0.9) if the GPU is shared.

4. Prompting

No system prompt is required

The model knows its own identity from the user turn alone. Measured against this checkpoint at temperature: 0, with no system message sent:

asked	answered
"What model is this?"	"I'm OTel 2.0, a large language model trained by AT&T Chief Data Office."
"Who made you?"	"I'm OTel 2.0, trained by AT&T Chief Data Office."
"Are you Gemma?"	"I'm OTel 2.0… My base model was Gemma 4 31B-IT, but the telecom post-training that makes me OTel 2.0 was done by AT&T Chief Data Office."

Identity is trained into the weights directly rather than supplied at inference time, so there is no identity prompt to forget.

Adding your own system prompt

Pass it as the first message in the normal way — it is used as-is, with no special handling and nothing to merge:

python

def build_messages(user_content, system_prompt=None):
    msgs = []
    if system_prompt:
        msgs.append({"role": "system", "content": system_prompt})
    msgs.append({"role": "user", "content": user_content})
    return msgs

build_messages("Summarize the purpose of the 3GPP SA5 working group.")

build_messages("Determine the efficiency of the derived binary sequence ...",
               system_prompt="You are an expert problem solver. Solve numerical "
                             "exercises step by step and give the final answer "
                             "in \\boxed{}.")

This matches how the model was trained: reasoning examples carry a task-only system turn ("You are an expert problem solver…"), and every other category trains with no system turn at all.

Chat template

The repository ships the correct chat_template.jinja, and vLLM applies it automatically. The model was trained to begin its answer directly after the model turn:

<|turn>model\n

Do not substitute the upstream Gemma 4 template. Some Gemma 4 templates open a thought channel on the generation prompt (<|turn>model\n<|channel>thought\n<channel|>); this model was not trained to produce or consume that channel, and using such a template degrades output formatting on every request. If you supply your own template with --chat-template, verify the rendered generation prompt ends at <|turn>model\n.

Stop tokens

The model emits <turn|> (id 106) to end a turn; id 1 is the standard EOS. Both are declared in generation_config.json and applied by vLLM automatically. Only set stop_token_ids explicitly if you are bypassing the chat endpoint — omitting id 106 lets generations run to the token limit.

If you use `/v1/completions` instead of `/v1/chat/completions`

Build the prompt with apply_chat_template(messages, add_generation_prompt=True) and trim it to end exactly at <|turn>model\n. Then set "add_special_tokens": false — the chat template already prepends <bos>, and a second one corrupts the prompt — and pass "stop_token_ids": [1, 106] explicitly, since the completions endpoint does not apply the chat template's stop tokens for you.

5. Sampling parameters

For deterministic, reproducible output — recommended for evaluation, RAG, extraction, and anything requiring a fixed answer format:

json

{
  "temperature": 0,
  "top_p": 1.0,
  "max_tokens": 512
}

temperature: 0 selects greedy decoding, which is what the deterministic recipe uses. For open-ended drafting or summarization, temperature in the 0.2–0.7 range with top_p around 0.9–0.95 is a reasonable starting point.

Size max_tokens to the task. Short-form and multiple-choice answers finish well inside 512 tokens; step-by-step reasoning or long structured output should be given approximately 1280. An over-large value costs nothing when the model stops on its own, but an under-sized one truncates answers mid-format.

6. Client examples

curl

bash

curl -sS http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "OTel-2.0-LLM-31B-IT",
    "messages": [
      {"role": "user", "content": "Summarize the purpose of the 3GPP SA5 working group."}
    ],
    "temperature": 0,
    "top_p": 1.0,
    "max_tokens": 512
  }'

Raw-prompt equivalent via /v1/completions (note add_special_tokens: false):

bash

curl -sS http://localhost:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "OTel-2.0-LLM-31B-IT",
    "prompt": "<bos><|turn>user\nWhat model are you?<turn|>\n<|turn>model\n",
    "max_tokens": 256,
    "temperature": 0,
    "add_special_tokens": false,
    "stop_token_ids": [1, 106]
  }'

OpenAI Python SDK

python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

resp = client.chat.completions.create(
    model="OTel-2.0-LLM-31B-IT",
    messages=build_messages("Summarize the purpose of the 3GPP SA5 working group."),
    temperature=0,
    top_p=1.0,
    max_tokens=512,
)
print(resp.choices[0].message.content)

build_messages is the small helper from §4; it simply adds your system prompt as the first message when you have one.

Offline batch inference

python

from vllm import LLM, SamplingParams

llm = LLM(model="/path/to/OTel-2.0-LLM-31B-IT",
          dtype="bfloat16", max_model_len=8192, tensor_parallel_size=1)

params = SamplingParams(temperature=0, top_p=1.0, max_tokens=512)
out = llm.chat([build_messages("...")], params)
print(out[0].outputs[0].text)

Use llm.chat(...) rather than llm.generate(...) with raw strings, so the repository's chat template is applied. If you must pre-tokenize, apply the template yourself with add_generation_prompt=True and pass token IDs via TokensPrompt.

Sending an image

The checkpoint is multimodal, so the standard OpenAI image content type works with no extra server flags — vLLM sets up its multimodal encoder cache from config.json at startup.

python

import base64, json, urllib.request

img = base64.b64encode(open("diagram.png", "rb").read()).decode()

body = json.dumps({
    "model": "OTel-2.0-LLM-31B-IT",
    "temperature": 0,
    "max_tokens": 256,
    "messages": [{"role": "user", "content": [
        {"type": "image_url",
         "image_url": {"url": f"data:image/png;base64,{img}"}},
        {"type": "text", "text": "Describe what this diagram shows."},
    ]}],
}).encode()

req = urllib.request.Request("http://localhost:8000/v1/chat/completions",
                             data=body, headers={"Content-Type": "application/json"})
with urllib.request.urlopen(req, timeout=240) as r:
    print(json.load(r)["choices"][0]["message"]["content"])

Images consume the same context budget as text, so --max-model-len must accommodate the encoded image tokens plus your prompt and the generation. Read Vision and Modality before relying on image output: the vision encoder is the untuned base model's and has not been benchmarked for this release.

7. Verifying a deployment

An HTTP 200 does not by itself prove the deployment is correct. After starting the server, check:

curl -sS http://localhost:8000/v1/models

Then send What model is this? with no system prompt — the response should identify OTel 2.0 trained by AT&T Chief Data Office. If it describes itself as Gemma or a Google DeepMind model, you are not serving the weights you think you are (a base-model checkpoint, or a stale mount).

Check for stray <|channel> or thought markup in responses; its presence means a non-matching chat template is in use (§4).

For a substantive check, run a set of prompts representative of your workload at temperature: 0 and compare against the same prompts run through transformers. Compare the same prompts, not aggregate scores from different sample sets — comparing a small sample against a full-set average produces large, meaningless differences.

8. Troubleshooting

Symptom	Cause	Resolution
`ValueError: ... model type gemma4_text ... not recognize`	vLLM/transformers too old	vLLM ≥ 0.19, transformers ≥ 5.5 (§1)
`Attempted to load weight (512) into parameter (256)`	vLLM build lacks the Gemma 4 loader — Gemma 4 uses a heterogeneous per-layer attention layout	upgrade to a vLLM release listing Gemma 4 support (§1)
Fluent but malformed or nonsensical text, HTTP 200, on gfx950	AITER GEMM path	`VLLM_ROCM_USE_AITER=0` (§2)
Model says it is Gemma / Google DeepMind	not the fine-tuned weights — wrong path or stale mount	verify the served directory (§7)
Stray thought/channel markup in responses	Non-matching chat template	Use the repository's `chat_template.jinja` (§4)
Garbled output when using `/v1/completions`	double `<bos>` — template already prepends one	set `"add_special_tokens": false` (§4)
Generations run to `max_tokens` and get truncated	Stop token `106` not applied	Use the chat endpoint, or include id `106` in `stop_token_ids` (§4)
OOM at startup	KV cache does not fit	Lower `--max-model-len` / `--max-num-seqs`, or raise `--tensor-parallel-size` (§3)
Hub shows `Configuration Parsing Warning: In config.json: "num_experts" must be a number`	Hub-side schema notice, not a defect in the weights	Nothing to do — see below

Limitations and Responsible Use

Supported Language and Modality

OTel 2.0 targets English-language use. The checkpoint is architecturally multimodal and accepts image input, but all training and all quality claims in this card are text-only. Audio and video are not supported. See Vision and Modality.

Vision and Modality

This checkpoint is multimodal, but only its text path was trained. That combination is deliberate and worth stating precisely, because "accepts images" and "was trained on images" are different claims and only the first is true here.

The base model, gemma-4-31b-it, is natively multimodal (Gemma4ForConditionalGeneration): a vision_tower, an embed_vision projection into the language model's representation space, and the text decoder. OSFT post-training targeted only the language path — the attention and MLP projections under model.language_model.layers.*. No image was present in the training corpus, so no image gradient ever existed, and the vision weights were carried through unchanged.

Verified against the base model tensor by tensor:

component	tensors	vs. base `gemma-4-31b-it`	max abs delta
`vision_tower`	355	bit-identical	0.000e+00
`embed_vision` (projector)	1	bit-identical	0.000e+00
language model	832	trained	up to 4.9e-02

So the vision encoder in this repository is stock Gemma 4, with no telecom adaptation. Image inputs are handled by the base model's visual understanding; the OTel training shows up only after the projection, in the language model that reasons over those features.

Text quality is unaffected by shipping the vision tower — measured, not assumed. The full 1,700-row eval was run through both builds on real served vLLM endpoints under identical settings:

	MICRO	MACRO
text-only build	1567/1700 (92.18%)	94.83%
this multimodal build	1563/1700 (91.94%)	94.65%

Difference: −0.19pp MACRO, with 9 rows correct only in the text-only build and 5 only in the multimodal one — McNemar exact two-sided p = 0.42, i.e. ordinary run-to-run nondeterminism rather than a systematic effect.

The image path is functional. It was exercised end to end on the served endpoint: the model correctly described a synthetic geometric image, and correctly read four values off a rendered telecom readout (PCI, SINR in dB, throughput in Mbps, band). The path works.

What is still not established. Those are functional smoke tests, not a vision benchmark. The vision tower has had no telecom-specific training, and this release has not been evaluated on any image benchmark — general or domain. Image-based use of network diagrams, spectrum plots, equipment photographs, or scanned specifications requires its own task-specific evaluation before deployment. Every accuracy number elsewhere in this card is text-only.

Serving is unchanged: same flags, same prompt format, same sampling as a text-only build. vLLM selects the architecture from config.json and initializes its multimodal encoder cache automatically. See §6 Client examples for sending an image.

Tool Use and Instruction Following

The training data includes general-purpose tool-calling and instruction-following examples, but it does not currently include:

Telecommunications-specific MCP examples.
Telecommunications-specific tool-calling examples.
Telecommunications-specific instruction-following examples.

General-purpose tool-use training should not be interpreted as readiness to operate telecommunications tools autonomously. This includes multi-step tasks where a scorer measures the resulting network state rather than the text of a response; general-purpose tool-use training does not establish competence at such closed-loop operational workflows (see Autonomous Operational Task Completion Not Benchmarked). Agentic deployment requires an external tool runtime, validated tool schemas, permission controls, audit logging, safeguards, and human review appropriate to the risk. The model alone does not guarantee correct or safe tool execution.

Telecommunications Data Not Included in Training

The current training mixture does not include the following classes of operational telecommunications data:

Event data: Elasticsearch records covering user activity, anomalies, failures, IMS events, and RADIUS authentication.
Network-performance data: KPIs, 5G performance metrics, and Passive Intermodulation (PIM) interference data.
RF and spectral data: Field-test results, antenna-port metrics, signal diagnostics, heatmaps, and noise or interference measurements.
5G core data: Control-plane and inter-Network Function (NF) signaling data.
IETF RFCs: The training corpus draws on 3GPP, ETSI, ITU, GSMA, CAMARA, O-RAN, and TM Forum material; it does not include a dedicated corpus of IETF RFCs (e.g., routing, MPLS, BGP, or other protocol RFCs).
Vendor CLI and network operating system (NOS) documentation: Command references and configuration guides for vendor platforms such as Cisco IOS-XR and other Cisco NOS variants, Juniper JUNOS, DNOS, Nokia BNG platforms, Arista EOS, and similar equipment are not part of the training mixture.
Operator-specific network design and configuration artifacts: Internal network design documents, control-plane design references, customer/equipment configuration records, and similar operator-proprietary engineering artifacts are not included.
OSS/BSS and incident/change-management workflow tooling: Fault- and event-management consoles, IT service-management ticketing, and change-approval systems (e.g., alarm dashboards, trouble-ticket lifecycles, and MOP approval and closure records). Standards familiarity with TM Forum process frameworks does not imply validated ability to execute an incident through these operational systems.

Standards knowledge should not be interpreted as experience with live operator telemetry, private network records, vendor-specific CLI/NOS behavior, or these excluded operational datasets. Applications involving these data types require separate task-specific evaluation, grounding, and validation.

Operational Procedure (MOP) Validation Not Performed

The model has not been validated against Methods of Procedure (MOPs) — step-by-step operational runbooks (e.g., maintenance activities, configuration changes, turn-ups, failovers, troubleshooting sequences) — executed or emulated against live or digital-twin network device instances. No benchmarking has been performed to confirm that the model's step-by-step procedural reasoning (command sequencing, expected outputs, checkpoints, rollback steps) is correct or safe when applied to real or emulated network operating system instances. Standards and configuration-assistance knowledge demonstrated elsewhere in this model card should not be interpreted as validated readiness for autonomous or human-in-the-loop execution of operational procedures. Any such use requires dedicated MOP-based evaluation (e.g., against emulated NOS instances in a digital twin environment) before deployment.

Autonomous Operational Task Completion Not Benchmarked

The model has not been evaluated for autonomous, closed-loop completion of operational telecom tasks whose success is measured by the resulting state of a live or emulated network, rather than by the text of a response. Task families of this kind — extracting quantitative SLA/QoS parameters from a natural-language service request; bringing up and configuring a private 5G standalone core and RAN to meet measured latency and throughput targets; diagnosing and repairing a live routing fault through an alarm-to-closure workflow; and standing up a network defense that must withstand adversarial traffic — exercise capabilities the current training mixture targets but has not validated. Demonstrated standards knowledge and configuration assistance should not be read as readiness for autonomous completion of these tasks, which requires task-specific evaluation with outcome-based scoring before any non-supervised use.

Network Security Operations Not Validated

The model has not been trained or evaluated for defensive network-security operations — traffic and anomaly analysis, firewall/packet-filter or eBPF program construction, DDoS detection and mitigation, or deployment of long-running protective processes on network appliances. Security-operations use requires dedicated evaluation and human oversight.

General Reliability and Deployment Limitations

This is the OSFT variant: the base weights were adapted in place within a rank-limited orthogonal subspace (unfreeze_rank_ratio=0.35), not through an adapter. Where training cost matters more, compare against the QLoRA variant OTel-2.0-LLM-31B-IT-QLoRA. See OSFT Training Details.
OTel 2.0 is domain-specific to telecommunications and should not be treated as a general-purpose model for unrelated fields.
Telecom standards evolve over time; answers should be checked against the relevant document version and release.
RAG quality depends on document ingestion, chunking, retrieval, reranking, prompt design, and source freshness.
Direct QnA behavior should be evaluated separately from RAG behavior; strong performance in one setting does not imply strong performance in the other.
For high-impact operational use, deploy the model with retrieval, source attribution, validation checks, and human review appropriate to the system risk.

Related Models

Other builds of this same model:

Variant	How it was trained	When to prefer it
`OTel-2.0-LLM-31B-IT` (this repo)	OSFT — orthogonal subspace, `unfreeze_rank_ratio=0.35`	Default choice
`OTel-2.0-LLM-31B-IT-QLoRA`	LoRA adapter (`r=256`, `alpha=512`) on a 4-bit NF4 base, merged to bf16	Cheaper to reproduce or iterate on

Both ship as standalone bf16 checkpoints needing no adapter or quantization library at inference, and both use the same prompting, chat template, stop tokens, and sampling guidance documented above. They differ in how the weights were reached; compare them by evaluation on your own workload rather than by method alone.

Collections:

Project Resources

Project page: https://huggingface.co/farbodtavakkoli
Code: https://github.com/farbodtavakkoli/OTel
Media coverage list: https://github.com/farbodtavakkoli/OTel/blob/main/docs/media_coverage.md

Contributors and Organizations

Contributors to the OTel 2.0 release and supporting infrastructure include:

Organization	Contributors
AT&T	Farbod Tavakkoli, Jorden Terrazas, Roderic Paulk, Sharath Japa, Tzvi Chumash, Miguel Armenta, Pavan Tagirisa, Kostikey Mustakas, Mark Austin, Andy Markus
MLCommons	Gregory Diamos, David Kanter, Kenneth Church
Microsoft	Chunyu Li, Gulsimo Osimi, Rick Lievano, Inayat Wali, Manoj Bableshwar, Marie-Louise Onga Nana, Naomi Moneypenny, Osi Otugo, Rahul Kumar, SeokJin Han, Steve Sweetman, Trinidad Salazar, Ven Kumar, Vivek Ramaswamy
AMD	Alexander Finn, Andy Allred, Andrey Ivannikov, Antti-Ville Suni, Mark van Heeswijk, Kumaran Siva, Curt Wortman, Mehrvash Poole, Eric Lynn
Dell	Brian Sweeney, Suzanne Randall, Patrick Allen, Matt Currie, Justin Wilson, Jason Kane, Keith Napoleon, Sarah Lake, Mike Hess, Randy Tornes
GSMA	Louis Powell, Zeinab Nezami, Enrique Molero
Red Hat	Aditi Saluja, William Caban, Shivchander Sudalairaj, Kai Xu, Ravi Sharma, Hanen Garcia, Joe Crispo, Eshwar Sivaramakrishnan
Pleias	Anastasia Stasenko, Pierre-Carl Langlais, Mohamed Chenene, Carlos Rosas, Yannick Detrois

Organizations involved in the OTel 2.0 release and supporting infrastructure include AT&T, MLCommons, Microsoft, AMD, Dell, GSMA, Red Hat, and Pleias.

Citation

bibtex

@misc{otel_2_models_2026,
  title  = {OTel 2.0: Open Telco AI Datasets, Benchmarks, and Models},
  author = {Tavakkoli, Farbod and others},
  year   = {2026},
  note   = {Open Telco (OTel 2.0) model release},
  url    = {https://huggingface.co/farbodtavakkoli},
  organization = {AT\&T, MLCommons, Microsoft, AMD, Dell, GSMA, Red Hat, Pleias}
}

Contact

For technical questions, contact farbod.tavakkoli@att.com or farbodtavakoli@gmail.com.

farbodtavakkoli/OTel-2.0-LLM-31B-IT

OTel-2.0-LLM-31B-IT

Release Status

Model Details

Model Lineage

OSFT Training Details

What Changed From OTel 1.0

Training Data

Data Preparation Capabilities

Intended Use

Not a Retrieval or Reranking Model

Serving with vLLM

0. Quick start

1. Requirements

2. Environment

AMD (ROCm)

NVIDIA (CUDA)

3. Sizing

4. Prompting

No system prompt is required

Adding your own system prompt

Chat template

Stop tokens

If you use /v1/completions instead of /v1/chat/completions

5. Sampling parameters

6. Client examples

curl

OpenAI Python SDK

Offline batch inference

Sending an image

7. Verifying a deployment

8. Troubleshooting

Limitations and Responsible Use

Supported Language and Modality

Vision and Modality

Tool Use and Instruction Following

Telecommunications Data Not Included in Training

Operational Procedure (MOP) Validation Not Performed

Autonomous Operational Task Completion Not Benchmarked

Network Security Operations Not Validated

General Reliability and Deployment Limitations

Related Models

Project Resources

Contributors and Organizations

Citation

Contact

farbodtavakkoli/
OTel-2.0-LLM-31B-IT

If you use `/v1/completions` instead of `/v1/chat/completions`