Quantization made by Richard Erkhov.

tiny-T5ForConditionalGeneration - MLX

Install mlx-lm:

You can use mlx-lm from the command line. For example:

This will download a model from the Hugging Face Hub and generate text using the given prompt.

To chat with an LLM use:

This will give you a chat REPL that you can use to interact with the LLM. The chat context is preserved during the lifetime of the REPL.

For a full list of options run --help on the command of your interest, for example:

library_name: transformers tags:

This is a minimal model built for unit tests in the TRL library.