Views
No views yet
neuronx compatible checkpoints for codellama/CodeLlama-7b-hf.
You can find detailed information about the base model on its Model Card.neuron format using specific input_shapes and compiler parameters detailed in the paragraphs below.optimum-neuron documentation for an explanation of these parameters.optimum-neuron1>>> from optimum.neuron import pipeline
2
3>>> p = pipeline('text-generation', 'aws-neuron/CodeLlama-7b-hf-neuron-24xlarge')
4>>> p("import socket\n\ndef ping_exponential_backoff(host: str):",
5 do_sample=True,
6 top_k=10,
7 temperature=0.1,
8 top_p=0.95,
9 num_return_sequences=1,
10 max_length=200,
11)
12[{'generated_text': 'import socket\n\ndef ping_exponential_backoff(host: str):\n """\n Ping a host with exponential backoff.\n\n :param host: Host to ping\n :return: True if host is reachable, False otherwise\n """\n for i in range(1, 10):\n try:\n socket.create_connection((host, 80), 1).close()\n return True\n except OSError:\n time.sleep(2 ** i)\n return False\n\n\ndef ping_exponential_backoff_with_timeout(host: str, timeout: int):\n """\n Ping a host with exponential backoff and timeout.\n\n :param host: Host to ping\n :param timeout: Timeout in seconds\n :return: True if host is reachable, False otherwise\n """\n for'}]neuronx. When using with 🤗 optimum-neuron, use the repo revision specific to the version of neuronx you are using, to load the right serialized checkpoints.1{
2 "batch_size": 1,
3 "sequence_length": 2048,
4}1{
2 "auto_cast_type": "fp16",
3 "num_cores": 12,
4}