You can deploy this model and start earning money today!

Cheng98/
llama-160m

Name: llama-160m
Availability: InStock

text-generation-inference

endpoints_compatible

Views

No views yet

A toy Llama adapted from JackFram/llama-160m with special tokens added.

This checkpoint can be loaded into MASE's LlamaQuantized

python

from transformers.models.llama import LlamaTokenizer
from chop.models.manual.llama_quantized import (
    LlamaQuantizedConfig,
    LlamaQuantizedForCausalLM,
)

name="Cheng98/llama-160m"
tokenizer = LlamaTokenizer.from_pretrained(name)

# override the quant_config to quantized the model
# default does not quantize llama
config = LlamaQuantizedConfig.from_pretrained(
    name,
    # quant_config="./quant_config_na.toml"

)

llama = LlamaQuantizedForCausalLM.from_pretrained(
    name,
    config=config,
)