Audio question-answering dataset generator supporting diverse datasets like ESC-50, UrbanSound8K, and GISE. It dynamically concatenates variable-length audio clips to reach exact target durations. Creates 15 task types across three reasoning families: simple singular, multihop singular, and multihop inter-task temporal reasoning.
Quick Start
bash
1# 1. Install dependencies2pip install -r requirements.txt
34# 2. Preprocess datasets (e.g., ESC-50, UrbanSound8K) (required for duration-based tasks)5python preprocess_esc50.py --config config.yaml
6# Or for UrbanSound8K: python preprocess_urbansound8k.py --config config_urbansound8k.yaml78# 3. Generate datasets9python main.py --config config.yaml
10# Or use the helper scripts for specific datasets:11# ./run_pipeline_urbansound8k.sh12# ./run_pipeline_gise.sh
Configuration
Edit config.yaml (or config_urbansound8k.yaml / config_gise.yaml) to set:
Task duration: task_duration_size (hours) per task
Clip duration range: min_clip_duration to max_clip_duration (seconds)
Dataset paths: Point to your source dataset location (e.g., ESC-50, UrbanSound8K, GISE)
Variable Length Handling: Audio clips with native durations are automatically concatenated and trimmed to reach specific target durations while ensuring metadata correctness.
Enable/disable tasks: Set enabled: true/false for each task
Key Files
config.yaml, config_urbansound8k.yaml, config_gise.yaml - Configuration parameters for different datasets
main.py - Pipeline entry point (runs all tasks)
preprocess_esc50.py, preprocess_urbansound8k.py - Preprocess datasets for duration tasks
tasks/task_*.py - Individual task generators
tasks/multihop_base.py - Shared base class for multihop tasks
Tasks
Simple Singular Temporal Reasoning
Direct one-step questions over one temporal/acoustic property.
Task
Question
Example
COUNT
"How many unique sounds?" / "How many times does X occur?"
Audio with distinct sound types or repetitions
DURATION
"Which sound is longest/shortest?" / "Which is longer?"
Compare sound durations and pairwise comparisons
ORDER
"Which sound is first/last/after X?"
Temporal sequence questions
VOLUME
"Which sound is loudest/softest?"
Loudness comparison
SILENCE GAP
"Which sounds have the longest silence?"
Compare silence durations between sequential sounds
OVERLAP
"Which sound overlaps with X?" / "Do X and Y overlap?"
Identify partially overlapping sound events
DURING/CONTAINS
"Which sound occurs during X?"
One sound temporally contained entirely within another
Multihop Temporal Reasoning — Singular Task
Multi-hop questions within one task family. The model first applies a temporal condition (before, after, between), then answers a question of the same type.
Task
Question
Reasoning Hops
CONDITIONAL COUNT
"How many X sounds occur after Y?"
temporal filter → count
CONDITIONAL DURATION
"Which sound after Y lasts the longest?"
temporal filter → duration comparison
BETWEEN EVENTS
"Which sound occurs between X and Y?"
temporal window → identification
EVENT DENSITY
"Which half of the audio has more events?"
region segmentation → count comparison
Multihop Temporal Reasoning — Inter-Task
Multi-hop questions combining two or more temporal/acoustic properties.
Run full pipeline (uses python main.py under the hood):
bash
1# Make executable and run (from pipeline/)2./run_pipeline.sh
34# With custom config, tasks, and output5./run_pipeline.sh --config my_config.yaml --tasks count,order --output ./my_dataset
67# Run only multihop tasks8./run_pipeline.sh --tasks conditional_count,conditional_duration,between_events,event_density,duration_gap,temporal_arithmetic,temporal_loudness,multi_hop
Run the LLM answer generation across splits (uses llm_answer_generator.py):
bash
1# Processes open_text CSVs across splits/tasks defined in the script2./run_llm_answers_all.sh
34# Or run per-file with the helper script directly5python llm_answer_generator.py --input /path/to/count_open_text.csv --mode open_text --task count
Advanced Usage
bash
1# Run specific tasks only2python main.py --tasks count order conditional_count multi_hop
34# Use custom config (e.g., for UrbanSound8K)5python main.py --config config_urbansound8k.yaml
67# Custom output directory8python main.py --output /path/to/output
910# Preprocess with custom parameters11python preprocess_esc50.py --config config.yaml \12 --threshold-strategy noise_floor \13 --noise-floor-percentile 2.0\14 --noise-floor-delta-db 5.0
