This dataset contains the clusters discovered in the vocabulary embeddings of the olmo-7b-sft model.
The 50280 vocabulary embeddings are separated into 512 clusters by k-means, which show pattern correlations probably undesirable for diverse generation.
We also prompt GPT-4o to summarize the commonality of vocabularies in the same cluster, which can be used for further analysis.
This dataset is a part of the work on diverse LLM generation. [Paper], [Github]
