You can deploy this model and start earning money today!

eskim86/
PassphraseGPT

Name: PassphraseGPT
Availability: InStock

safetensors

code

text-generation

openai-community/gpt2

finetune

mit

Views

No views yet

PassphraseGPT

This repository includes PassphraseGPT, a pre-trained GPT-2 model and its source code, designed to learn and generate passphrases. PassphraseGPT was developed for the research paper "Open Sesame! On the Security and Memorability of Verbal Passwords", accepted at IEEE Security & Privacy 2025. The study evaluates the memorability and security of user-chosen verbal passwords through guessing attacks. Trained on various phrase datasets, the model can be used to generate passphrases for custom applications or further research. This model was inspired by PassGPT, developed in the study by Rando et al. (GitHub: https://github.com/javirandor/passgpt). If you're interested, feel free to check out their work at the link.

Installation

bash

# Clone the repository
git clone https://github.com/yourusername/PassphraseGPT.git
cd PassphraseGPT

# Install dependencies
pip install -r requirements.txt

Requirements

Python 3.10+
PyTorch 2.0.1
transformers 4.45.2
tokenizers 0.20.1
datasets 3.3.0
PyYAML 6.0.2
tqdm 4.67.1
numpy 1.26.4
wandb 0.19.6 (for experiment tracking)
CUDA toolkit 11.7 (for GPU acceleration)

A complete list of dependencies can be installed via the requirements.txt file:

Configuration

All components use a single YAML configuration file to ensure consistency across the training and generation pipeline. Here's an example configuration:

yaml

# Execution-wide parameters
config_args:
    seed: 14
    maxchars: 16 # Maximum tokens to be in your verbal passwords
    subsample: -1 # -1 means no subsampling training data
    tokenizer_path: 'tokenizer/wordpiece/' # Path or huggingface name for your tokenizer
    train_data_path: 'data/passphrase/phrases.txt' # Path to your training data
    #eval_data_path: 'data/passphrase/test_set.txt' # Path to evaluation data (optional)

# Details for model architecture (GPT2Config parameters)
model_args:
    n_head: 12
    n_layer: 8

# Training arguments (Transformers TrainingArguments)
training_args:
    per_device_train_batch_size: 512
    gradient_accumulation_steps: 1
    logging_steps: 250
    save_total_limit: 1
    num_train_epochs: 3
    overwrite_output_dir: true
    fp16: false
    output_dir: './pretrain/PassphraseGPT/' # Where to store checkpoints
    report_to: "wandb"
    save_steps: 5000

# Password generation parameters
generation_args:
    out_path: './generated/' # Path to store the generations
    filename: "generated_phrases.txt" # Filename where generations will be stored
    num_generate: 10000 # Number of passwords to generate
    batch_size: 1000 # Batch size for generation
    seed_offset: 0 # Random seed offset for generation
    device: 'cuda' # Device to run execution ('cuda' or 'cpu')
    num_beams: 1 # Number of beams for sampling
    top_p: 100 # Probability for nucleus sampling (0-100)
    top_k: null # Sample only from k tokens with highest probability (null = disabled)
    temperature: 1.0 # Sampling temperature

Training a WordPiece Tokenizer

python tokenizer_train.py --config config.yaml

This will train a BertWordPieceTokenizer on your dataset and save it to the path specified in the configuration file.

Training the Model

The model training script (not shown) will use the same configuration file to ensure consistency with the tokenizer and generation steps.

python train.py --config config.yaml

Generating Passphrases

python generate.py --config config.yaml

This will generate passphrases using the trained model and save them to the path specified in the configuration file.

Switching Between CPU and GPU

To switch between CPU and GPU, simply modify the device parameter in the generation_args section of your configuration file: