You can deploy this model and start earning money today!

damoncrockett/
gpt2-with-attentions-onnx

Name: gpt2-with-attentions-onnx
Availability: InStock

openai-community/gpt2

quantized

mit

Views

No views yet

GPT-2 with attention outputs (ONNX / Transformers.js)

This repository provides an ONNX export of openai-community/gpt2 that exposes per-layer attention tensors for use with Transformers.js, especially in browser-based visualization and analysis workflows.

Standard browser-ready GPT-2 exports typically expose logits but not attention outputs. This export was created specifically to make full prompt-level attention maps available alongside ordinary causal language-model outputs.

Repo ID: damoncrockett/gpt2-with-attentions-onnx

What is included

This repo contains:

the original GPT-2 tokenizer and config files
an ONNX model under onnx/model.onnx
standard causal language-model logits
attention outputs for all 12 transformer blocks

The exported ONNX graph exposes:

logits
attentions.0
attentions.1
...
attentions.11

Each attention output has shape:

[batch_size, num_heads, sequence_length, sequence_length]

For GPT-2 small, that means:

12 layers
12 attention heads per layer

Intended use

This model is intended for:

browser-based attention visualization
token-by-token analysis of input sequences
educational demos
interpretability experiments
applications built with Transformers.js that need both logits and attention maps

Typical use cases include:

showing per-token logprobs over an input sentence
rendering causal attention heatmaps
comparing attention heads across layers
inspecting how a decoder-only model distributes attention over a prompt

Example use in Transformers.js

javascript

import { AutoTokenizer, AutoModelForCausalLM } from "@huggingface/transformers";

const MODEL_ID = "damoncrockett/gpt2-with-attentions-onnx";

const tokenizer = await AutoTokenizer.from_pretrained(MODEL_ID);
const model = await AutoModelForCausalLM.from_pretrained(MODEL_ID, {
  device: "webgpu",
});

const enc = await tokenizer("The capital of France is Paris.", {
  add_special_tokens: true,
  return_tensor: true,
});

const out = await model.forward({
  ...enc,
  output_attentions: true,
});

console.log(Object.keys(out));

Notes on output format

Depending on the exact Transformers.js build, attention outputs may need to be normalized before use. In practice, some setups expose attention tensors in a top-level array-like structure, while others expose one tensor per layer under distinct keys.

For visualization, the important assumption is that each layer yields an attention tensor with dimensions:

[1, H, T, T]

for a single input sequence, where:

H = number of heads
T = number of tokens in the input

Because this is a decoder-only model, the attention pattern is causal: tokens attend only to themselves and earlier positions.

Export details

This model was exported from openai-community/gpt2 to ONNX with attention outputs explicitly enabled.

Key export choices:

architecture: GPT-2 causal language model
full-sequence export
output_attentions=True
use_cache=False
eager attention implementation during export

These choices were made to preserve full prompt-level attention matrices suitable for visualization, rather than to maximize autoregressive decoding speed.

Limitations

This repo is intended primarily for analysis and visualization, not for maximum-throughput generation.

Because the export preserves full attention outputs:

it may be larger or slower than minimal browser GPT-2 exports
it is better suited to prompt inspection than to highly optimized decoding
attention maps should not be treated as a complete explanation of model behavior

As with other GPT-2 models, tokenization is subword-based, so displayed token strings may include leading-space markers or partial word fragments.

Base model

Base model:

openai-community/gpt2

This repository is a derived export of the base model for ONNX / Transformers.js compatibility with attention outputs enabled.

License

This repository inherits the licensing conditions of the base model. Please consult the upstream model card and license information for openai-community/gpt2.

Citation

If you use this export, please cite the original GPT-2 work and reference the upstream base model.