You can deploy this model and start earning money today!

lucianoon/
t5-small-lora-tweetsumm-onnx

Name: t5-small-lora-tweetsumm-onnx
Availability: InStock

Views

No views yet

T5-Small + LoRA (TweetSumm) — ONNX for Transformers.js

ONNX export of lucianoon/t5-small-lora-tweetsumm with the LoRA adapter merged into the base weights, quantized to INT8 for in-browser inference with Transformers.js.

🖥️ Live demo (runs in your browser): lora-tweetsumm-demo
🧬 Training code and rank ablation: github.com/lucianoon/lora-tweetsumm

Usage (Transformers.js)

import { pipeline } from "@huggingface/transformers";

const summarizer = await pipeline(
  "text2text-generation",
  "lucianoon/t5-small-lora-tweetsumm-onnx",
  { dtype: "q8" },
);

const dialogue = "Customer: I need to cancel order #12345.\nAgent: Done! Refund in 3-5 days.";
const out = await summarizer("summarize: " + dialogue, { max_new_tokens: 48 });
console.log(out[0].generated_text);

Files

File	Precision	Size
`onnx/encoder_model_quantized.onnx`	INT8	34 MB
`onnx/decoder_model_merged_quantized.onnx`	INT8	56 MB
`onnx/encoder_model.onnx`	FP32	135 MB
`onnx/decoder_model_merged.onnx`	FP32	222 MB

Export pipeline: peft merge_and_unload → optimum-cli export onnx (task text2text-generation-with-past) → onnxruntime dynamic quantization.