You can deploy this model and start earning money today!

yujiepan/
Llama-2-7b-int8asym-sparse-up30-gate30-down50

Name: Llama-2-7b-int8asym-sparse-up30-gate30-down50
Availability: InStock

Views

No views yet

[internal use]

python

import transformers
from optimum.intel.openvino import OVModelForCausalLM
model_id = '<local folder or model_id on HF>'
ov_model = OVModelForCausalLM.from_pretrained(model_id)
tokenizer = transformers.AutoTokenizer.from_pretrained(model_id)
pipe = transformers.pipelines.TextGenerationPipeline(model=ov_model, tokenizer=tokenizer)
output = pipe('Hello, I am a ', max_new_tokens=16)

yujiepan/Llama-2-7b-int8asym-sparse-up30-gate30-down50

yujiepan/
Llama-2-7b-int8asym-sparse-up30-gate30-down50