The all config may contain duplicates.
Use the clustered sample_k* configs which contain semantically distinct rows.
Source
Rows
Norquinal/claude_evol_instruct_210k
270641
Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
181082
Norquinal/WizardLM_alpaca_claude_evol_instruct_70k
61617
kalomaze/Opus_Instruct_25k
25102
Solenopsisbot/real-slop
24970
Norquinal/claude_multiround_chat_30k
16942
SicariusSicariiStuff/Claude_32K
15229… See the full description on the dataset page:
https://huggingface.co/datasets/agentlans/claude.
