from speculators.models.eagle import EagleSpeculator
from transformers import AutoModelForCausalLM

# Load the HASS speculator
hass_model = EagleSpeculator.from_pretrained("nm-testing/hass-llama3.1-8b-layernorms")

# Attach verifier model
verifier = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
hass_model.attach_verifier(verifier)

# Use for speculative decoding
# ... your speculative decoding code here ...

Conversion Details

This checkpoint was converted using:

bash

speculators convert --eagle --layernorms \
  nm-testing/Eagle_Speculator_Llama_3_1_8B_TTT \
  ./hass-layernorms-converted \
  meta-llama/Meta-Llama-3.1-8B-Instruct

License

Apache 2.0