Sample text embeddings produced by a durable producer/consumer GPU queue with
resume-on-crash support. Source code:
github.com/narinzar/distributed-embedding-generation-queue.
Model: sentence-transformers/all-MiniLM-L6-v2 (384-dimensional vectors).
Pipeline: a durable SQLite task queue (WAL mode) feeds a GPU worker pool.
Items are claimed atomically, embedded in batches, written as .npy… See the full description on the dataset page:
https://huggingface.co/datasets/narinzar/distributed-embedding-generation-queue.
