An 8-bit build of the LLM2Vec Llama-3-8B text encoder used by NVIDIA ARDY for text-to-motion generation, packaged for Voxta.

ARDY conditions on a single 4096-dimensional pooled vector per prompt, produced by LLM2Vec: Meta-Llama-3-8B-Instruct with bidirectional attention, the McGill-NLP MNTP adapter, and the McGill-NLP supervised adapter. Assembling that at runtime means a 16GB download, a 15.4GB resident encoder, and a two-stage adapter merge on every launch. This repo is that same encoder with both adapters already merged and the final weights quantized to 8-bit.

This is the higher-fidelity option. For the smallest footprint, see the NF4 build.

The adapters were trained against fp16 base weights. Merging them into weights that have already been quantized compounds the error badly: measured cosine similarity against the bf16 reference drops to 0.77. This build merges first at bf16 and quantizes the final weights, which is the correct order.

Twelve motion prompts, encoded through the full LLM2Vec pipeline, compared to the unquantized bf16 assembly:

Relative geometry is preserved: across the prompt set, maximum pairwise-similarity delta is 0.010 and every prompt still lands nearest its own reference vector.

Note that a diffusion model is chaotic with respect to its conditioning. For a fixed seed this encoder produces a different, equally valid motion rather than a degraded one, so seeds are not comparable across precisions. As a control, switching bf16 to fp16 (a change nobody would call a quality loss) already shifts joints by 6.5mm on average.

Requires bitsandbytes and a CUDA device.

llm2vec_config.json ships in this repo, so pooling behaviour (mean pooling over the prompt tokens, instruction skipped) is restored automatically.

Meta Llama 3 is licensed under the Meta Llama 3 Community License, Copyright (c) Meta Platforms, Inc. All Rights Reserved.