This model was trained on EVM (Ethereum Virtual Machine) bytecode data from multiple blockchain networks. It learns meaningful representations of smart contract bytecode, transaction calldata, and contract creation code.

Intended Uses

Bytecode analysis: Understanding smart contract structure and patterns
Smart contract embeddings: Generating vector representations for similarity search
On-chain data analysis: Encoding transaction calldata for downstream tasks
Security research: Analyzing bytecode patterns for vulnerability detection

Training Data

The model was trained on the following data:

Chains: Ethereum, optimism, base, unichain, bera, bsc
Data types: Contract calldata, creation bytecode, deployed bytecode
Training date: 2026-01-01

Training Procedure

Architecture: DeBERTaV2
Objective: Masked Language Modeling (MLM)
Vocab size: 32,768 (byte-level BPE)
Max length: 512 tokens
Masking probability: 30%

Model Configuration

json

{
  "hidden_size": 256,
  "num_hidden_layers": 6,
  "num_attention_heads": 4,
  "vocab_size": 32768
}

How to Use

python

from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("evm-alpha/semantic-evm-mlm-chkp1000")
model = AutoModel.from_pretrained("evm-alpha/semantic-evm-mlm-chkp1000")

# This tokenizer uses byte-level BPE (GPT-2 style).
# You must convert raw bytes to the GPT-2 unicode format:

def bytes_to_unicode_map():
    """GPT-2 byte-to-unicode mapping."""
    bs = list(range(ord("!"), ord("~") + 1)) + list(range(ord("¡"), ord("¬") + 1)) + list(range(ord("®"), ord("ÿ") + 1))
    cs = bs[:]
    n = 0
    for b in range(256):
        if b not in bs:
            bs.append(b)
            cs.append(256 + n)
            n += 1
    return dict(zip(bs, [chr(c) for c in cs]))

BYTE_MAP = bytes_to_unicode_map()

def encode_evm_bytes(data: bytes) -> str:
    """Convert raw EVM bytes to tokenizer input string."""
    return "".join(BYTE_MAP[b] for b in data)

# Example: encode raw EVM bytecode
bytecode_hex = "608060405234801561001057600080fd5b50"
raw_bytes = bytes.fromhex(bytecode_hex)
tokenizer_input = encode_evm_bytes(raw_bytes)

inputs = tokenizer(tokenizer_input, return_tensors="pt")
outputs = model(**inputs)

# Get embeddings (mean pooling)
embeddings = outputs.last_hidden_state  # [batch, seq_len, hidden_size]
mask = inputs["attention_mask"].unsqueeze(-1)
pooled = (embeddings * mask).sum(1) / mask.sum(1)

Limitations

Trained on specific EVM chains; may not generalize to all EVM variants
Max context: 512 tokens (~256 bytes of bytecode)
Optimized for EVM bytecode patterns; not intended for natural language

License

This model is released under the GNU Affero General Public License v3.0.

Citation

bibtex

@misc{evm-semanticbytecode,
  author = {evm-alpha},
  title = {EVM SemanticBytecode},
  year = {2026},
  url = {https://huggingface.co/evm-alpha/semantic-evm-mlm-chkp1000}
}