HiTZ/

Magpie-Llama-3.1-70B-Instruct-Filtered-1M

Dataset generated using meta-llama/Llama-3.1-70B-Instruct with the MAGPIE codebase. The unfiltered dataset can be found here: /HiTZ/Magpie-Llama-3.1-70B-Instruct-Unfiltered

min_repetition = 100

def test_no_repetition(text: str): # Count the frequency of each word in the text word_count = Counter(text.split()) # Check if any word appears more than min_repetition times return all(count <= min_repetition for count in word_count.values())

def… See the full description on the dataset page: https://huggingface.co/datasets/HiTZ/Magpie-Llama-3.1-70B-Instruct-Filtered-1M.