You can deploy this model and start earning money today!

keras/
vicuna_1.5_7b_en

Name: vicuna_1.5_7b_en
Availability: InStock

keras-hub

text-generation-inference

text-generation

2306.05685

llama2

Views

No views yet

Model Overview

Vicuna is a chat assistant trained by fine-tuning Llama 2 on user-shared conversations collected from ShareGPT.Weights are release under the Llama 2 Community License Agreement and Keras model code are released under the Apache 2 License.

Model type: An auto-regressive language model based on the transformer architecture.

Fine tuned from model: Llama 2

Uses: The primary use of Vicuna is research on large language models and chatbots. The primary intended users of the model are researchers and hobbyists in natural language processing, machine learning, and artificial intelligence.

Installation

Keras and KerasHub can be installed with:

pip install -U -q keras-hub
pip install -U -q keras

Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instruction on installing them in another environment see the Keras Getting Started page.

Presets

The following model checkpoints are provided by the Keras team. Full code examples for each are available below.

Preset name	Parameters	Description
`vicuna_1.5_7b_en`	6.74B	7 billion parameter, 32-layer, instruction tuned Vicuna v1.5 model.

Paper: https://arxiv.org/abs/2306.05685

Example Usage

python

import keras
import keras_hub
import numpy as np

Use generate() to do text generation.

python

vicuna_lm = keras_hub.models.LlamaCausalLM.from_preset("vicuna_1.5_7b_en")
vicuna_lm.generate("### HUMAN:\nWhat is Keras? \n### RESPONSE:\n", max_length=500)

# Generate with batched prompts.
vicuna_lm.generate([
    "### HUMAN:\nWhat is ML? \n### RESPONSE:\n",
    "### HUMAN:\nGive me your best brownie recipe.\n### RESPONSE:\n",
],max_length=500)

Compile the generate() function with a custom sampler.

python

vicuna_lm = keras_hub.models.LlamaCausalLM.from_preset("vicuna_1.5_7b_en")
vicuna_lm.compile(sampler="greedy")
vicuna_lm.generate("I want to say", max_length=30)

vicuna_lm.compile(sampler=keras_hub.samplers.BeamSampler(num_beams=2))
vicuna_lm.generate("I want to say", max_length=30)

Use generate() without preprocessing.

python

prompt = {
    # `1` maps to the start token followed by "I want to say".
    "token_ids": np.array([[1, 306, 864, 304, 1827, 0, 0, 0, 0, 0]] * 2),
    # Use `"padding_mask"` to indicate values that should not be overridden.
    "padding_mask": np.array([[1, 1, 1, 1, 1, 0, 0, 0, 0, 0]] * 2),
}

vicuna_lm = keras_hub.models.LlamaCausalLM.from_preset(
    "vicuna_1.5_7b_en",
    preprocessor=None,
    dtype="bfloat16"
)
vicuna_lm.generate(prompt)

Call fit() on a single batch.

python

features = ["The quick brown fox jumped.", "I forgot my homework."]
vicuna_lm = keras_hub.models.LlamaCausalLM.from_preset("vicuna_1.5_7b_en")
vicuna_lm.fit(x=features, batch_size=2)

Call fit() without preprocessing.

python

x = {
    "token_ids": np.array([[1, 450, 4996, 17354, 1701, 29916, 12500, 287, 29889, 0]] * 2),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 0]] * 2),
}
y = np.array([[450, 4996, 17354, 1701, 29916, 12500, 287, 29889, 0, 0]] * 2)
sw = np.array([[1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2)

vicuna_lm = keras_hub.models.LlamaCausalLM.from_preset(
    "vicuna_1.5_7b_en",
    preprocessor=None,
    dtype="bfloat16"
)
vicuna_lm.fit(x=x, y=y, sample_weight=sw, batch_size=2)

Example Usage with Hugging Face URI

python

import keras
import keras_hub
import numpy as np

Use generate() to do text generation.

python

vicuna_lm = keras_hub.models.LlamaCausalLM.from_preset("hf://keras/vicuna_1.5_7b_en")
vicuna_lm.generate("### HUMAN:\nWhat is Keras? \n### RESPONSE:\n", max_length=500)

# Generate with batched prompts.
vicuna_lm.generate([
    "### HUMAN:\nWhat is ML? \n### RESPONSE:\n",
    "### HUMAN:\nGive me your best brownie recipe.\n### RESPONSE:\n",
],max_length=500)

Compile the generate() function with a custom sampler.

python

vicuna_lm = keras_hub.models.LlamaCausalLM.from_preset("hf://keras/vicuna_1.5_7b_en")
vicuna_lm.compile(sampler="greedy")
vicuna_lm.generate("I want to say", max_length=30)

vicuna_lm.compile(sampler=keras_hub.samplers.BeamSampler(num_beams=2))
vicuna_lm.generate("I want to say", max_length=30)

Use generate() without preprocessing.

python

prompt = {
    # `1` maps to the start token followed by "I want to say".
    "token_ids": np.array([[1, 306, 864, 304, 1827, 0, 0, 0, 0, 0]] * 2),
    # Use `"padding_mask"` to indicate values that should not be overridden.
    "padding_mask": np.array([[1, 1, 1, 1, 1, 0, 0, 0, 0, 0]] * 2),
}

vicuna_lm = keras_hub.models.LlamaCausalLM.from_preset(
    "hf://keras/vicuna_1.5_7b_en",
    preprocessor=None,
    dtype="bfloat16"
)
vicuna_lm.generate(prompt)

Call fit() on a single batch.

python

features = ["The quick brown fox jumped.", "I forgot my homework."]
vicuna_lm = keras_hub.models.LlamaCausalLM.from_preset("hf://keras/vicuna_1.5_7b_en")
vicuna_lm.fit(x=features, batch_size=2)

Call fit() without preprocessing.

python

x = {
    "token_ids": np.array([[1, 450, 4996, 17354, 1701, 29916, 12500, 287, 29889, 0]] * 2),
    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 0]] * 2),
}
y = np.array([[450, 4996, 17354, 1701, 29916, 12500, 287, 29889, 0, 0]] * 2)
sw = np.array([[1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2)

vicuna_lm = keras_hub.models.LlamaCausalLM.from_preset(
    "hf://keras/vicuna_1.5_7b_en",
    preprocessor=None,
    dtype="bfloat16"
)
vicuna_lm.fit(x=x, y=y, sample_weight=sw, batch_size=2)

keras/vicuna_1.5_7b_en

Model Overview

Links

Installation

Presets

Example Usage

Example Usage with Hugging Face URI

keras/
vicuna_1.5_7b_en