Example metadata can be found below, context represents the prompt that is presented to the model. Database schemas follow the encoding method proposed by Shaw et al (2020).

"query": "SELECT count(*) FROM singer",
"question": "How many singers do we have?",
"context": "How many singers do we have? | concert_singer | stadium : stadium_id, location, name, capacity, highest, lowest, average | singer : singer_id, name, country, song_name, song_release_year, age, is_male | concert : concert_id, concert_name, theme, stadium_id, year | singer_in_concert : concert_id, singer_id",
"db_id": "concert_singer",

Model Results

Evaluation set: Spider/dev

Evaluation metrics: [Test-Suite-Execution, Execution Accuracy]

Model	Data	Run	Execution Accuracy	Test-Suite Execution Accuracy
T5-3B	semiotic/SynQL-Spider-Train	00	0.7021	0.5996
T5-3B	semiotic/SynQL-Spider-Train	01	0.6992	0.5464
T5-3B	semiotic/SynQL-Spider-Train	02	0.7002	0.5861

semiotic/T5-3B-SynQL-Spider-Train-Run-00

Model Card for T5-3B/SynQL-Spider-Train-Run-00

Model Context

Model Results

semiotic/
T5-3B-SynQL-Spider-Train-Run-00