Views
No views yet
neuronx compatible checkpoints for meta-llama/Llama-2-7b-hf.
You can find detailed information about the base model on its Model Card.neuron format using specific input_shapes and compiler parameters detailed in the paragraphs below.optimum-neuron documentation for an explanation of these parameters.optimum-neuron1>>> from optimum.neuron import pipeline
2
3>>> p = pipeline('text-generation', 'aws-neuron/Mistral-7B-Instruct-v0.1-neuron-4x2048-24-cores')
4>>> p("My favorite place on earth is", max_new_tokens=64, do_sample=True, top_k=50)
5[{'generated_text': 'My favorite place on earth is the ocean. It is where I feel most
6at peace. I love to travel and see new places. I have a'}]neuronx. When using with 🤗 optimum-neuron, use the repo revision specific to the version of neuronx you are using, to load the right serialized checkpoints.1{
2 "batch_size": 4,
3 "sequence_length": 2048,
4}1{
2 "auto_cast_type": "bf16",
3 "num_cores": 24,
4}