Targeted Paraphrasing Model for Adversarial Data Generation
This repository provides the (UN)-Targeted Paraphrasing Model, developed as part of the research presented in the paper: "Finding a Needle in the Adversarial Haystack: A Targeted Paraphrasing Approach For Uncovering Edge Cases with Minimal Distribution Distortion."
The model is designed to generate high-quality paraphrases with enhanced fluency, diversity, and relevance, and is tailored for applications in adversarial data generation.
The training process utilized a meticulously curated dataset comprising 560,550 paraphrase pairs from seven high-quality sources:
APT Dataset (Nighojkar and Licato, 2021)
Microsoft Research Paraphrase Corpus (MSRP) (Dolan and Brockett, 2005)
PARANMT-50M (Wieting and Gimpel, 2018)
TwitterPPDB (Lan et al., 2017)
PIT-2015 (Xu et al., 2015)
PARADE (He et al., 2020)
Quora Question Pairs (QQP) (Iyer et al., 2017)
Filtering steps were applied to ensure high-quality and diverse data:
Removal of pairs with over 50% unigram overlap to improve lexical diversity.
Elimination of pairs with less than 50% reordering of shared words for syntactic diversity.
Filtering out pairs with less than 50% semantic similarity, leveraging cosine similarity scores from the "all-MiniLM-L12-v2" model.
Discarding pairs with over 70% trigram overlap to enhance diversity.
The refined dataset consists of 96,073 samples, split into training (76,857), validation (9,608), and testing (9,608) subsets.
Model Description
The paraphrasing model is built upon FLAN-T5-large and fine-tuned on the filtered dataset for nine epochs. Key features include:
Performance: Achieves an F1 BERT-Score of 75.925%, reflecting superior fluency and paraphrasing ability.
Task-Specificity: Focused training on relevant pairs ensures high-quality task-specific outputs.
Enhanced Generation: Generates paraphrases introducing new information about entities or objects, improving overall generation quality.
Applications
This model is primarily designed to create adversarial training samples that effectively uncover edge cases in machine learning models while maintaining minimal distribution distortion.
Additionally, the model is suitable for general paraphrasing purposes, making it a versatile tool for generating high-quality paraphrases across various contexts. It is compatible with the Parrot paraphrasing library for seamless integration and usage. Below is an example of how to use the model with the Parrot library:
1from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
23model_name ="alykassem/FLAN-T5-Paraphraser"45# Load the tokenizer6tokenizer = AutoTokenizer.from_pretrained(model_name)78# Load the model9model = AutoModelForSeq2SeqLM.from_pretrained(model_name)1011# Example usage: Tokenize input and generate output12input_text ="Paraphrase: How are you?"13inputs = tokenizer(input_text, return_tensors="pt")1415# Generate response16outputs = model.generate(**inputs)17decoded_output = tokenizer.decode(outputs[0], skip_special_tokens=True)1819print("Generated text:", decoded_output)20
In Parrot
python
1from parrot import Parrot
2import torch
3import warnings
4warnings.filterwarnings("ignore")56# Uncomment to get reproducible paraphrase generations7# def random_state(seed):8# torch.manual_seed(seed)9# if torch.cuda.is_available():10# torch.cuda.manual_seed_all(seed)1112# random_state(1234)1314# Initialize the Parrot model (ensure initialization occurs only once in your code)15parrot = Parrot(model_tag="alykassem/FLAN-T5-Paraphraser", use_gpu=True)1617phrases =[18"Can you recommend some upscale restaurants in New York?",19"What are the famous places we should not miss in Russia?"20]2122for phrase in phrases:23print("-"*100)24print("Input Phrase: ", phrase)25print("-"*100)26 para_phrases = parrot.augment(input_phrase=phrase)27for para_phrase in para_phrases:28print(para_phrase)
Citation
If you find this work or model useful, please cite the paper:
@inproceedings{kassem-saad-2024-finding,
title = "Finding a Needle in the Adversarial Haystack: A Targeted Paraphrasing Approach For Uncovering Edge Cases with Minimal Distribution Distortion",
author = "Kassem, Aly and
Saad, Sherif",
editor = "Graham, Yvette and
Purver, Matthew",
booktitle = "Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = mar,
year = "2024",
address = "St. Julian{'}s, Malta",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.eacl-long.33/",
pages = "552--572",
}
