import { pipeline } from "@huggingface/transformers";

// Create a text generation pipeline
const generator = await pipeline(
  "text-generation",
  "onnx-community/moondream2.text_model-ONNX",
  { dtype: "q4" },
);

// Generate a response
const text = "Once upon a time";
const output = await generator(text, { max_new_tokens: 128 });
console.log(output[0].generated_text);

Example output

Once upon a time, there was a young girl named Lily who loved to read books. She would spend hours lost in the pages of her favorite stories, imagining herself as the main characters. One day, while reading a book about a magical forest, Lily had an idea. She wanted to create her own magical forest, filled with all the wonders and adventures she had read about in her books.

Lily knew that she needed to gather the right materials to create her magical forest. She decided to use her imagination and creativity to make the forest come to life. She gathered her friends and family to help her, and together they began to build the forest

Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using 🤗 Optimum and structuring your repo like this one (with ONNX weights located in a subfolder named onnx).

RichardErkhov/onnx-community_-_moondream2.text_model-ONNX-4bits

Original model description:

Usage (Transformers.js)

RichardErkhov/
onnx-community_-_moondream2.text_model-ONNX-4bits