Quantization made by Richard Erkhov.
base_model: meta-llama/Llama-2-7b-hf
inference: true
model_type: llama
pipeline_tag: text-generation
datasets:
This repo contains a
Llama 2 7B finetuned for code generation tasks using the
Evolved CodeAlpaca dataset.
Below we share some code snippets on how to get quickly started with running the model.
By leveraging a pre-sparsified model's structure, you can efficiently fine-tune on new data, leading to reduced hyperparameter tuning, training times, and computational costs. Learn about this process
here.
This model may be run with the transformers library. For accelerated inference with sparsity, deploy with
nm-vllm or
deepsparse.
1# pip install transformers accelerate
2from transformers import AutoTokenizer, AutoModelForCausalLM
3
4tokenizer = AutoTokenizer.from_pretrained("neuralmagic/Llama-2-7b-evolcodealpaca")
5model = AutoModelForCausalLM.from_pretrained("neuralmagic/Llama-2-7b-evolcodealpaca", device_map="auto")
6
7input_text = "def fibonacci(n):\n"
8input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
9
10outputs = model.generate(**input_ids)
11print(tokenizer.decode(outputs[0]))
Model evaluation metrics and results.
Coming soon.
For further support, and discussions on these models and AI in general, join
Neural Magic's Slack Community
