Quantization made by Richard Erkhov.
tiny-T5ForConditionalGeneration - MLX
Quick start for LLMs
Install mlx-lm:
You can use mlx-lm from the command line. For example:
mlx_lm.generate --model peft-internal-testing_-_tiny-T5ForConditionalGeneration-mlx --prompt "hello"This will download a model from the Hugging Face Hub and generate
text using the given prompt.
To chat with an LLM use:
This will give you a chat REPL that you can use to interact with the LLM. The
chat context is preserved during the lifetime of the REPL.
For a full list of options run --help on the command of your interest, for example:
Original model description:
library_name: transformers
tags:
Tiny T5ForConditionalGeneration
PEFT copy of trl-internal-testing/tiny-T5ForConditionalGeneration, minimal model built for unit tests.
