Dataset Card for "sampling-distill-train-data-kth-shift256"
Training data for sampling-based watermark distillation using the KTH s=256s=256s=256 watermarking strategy in the paper On the Learnability of Watermarks for Language Models. Llama 2 7Bwith decoding-based watermarking was used to generate 640,000 watermarked samples, each 256 tokens long. Each sample is prompted with 50-token prefixes from OpenWebText (prompts not included in the samples).
