You can deploy this model and start earning money today!

irisaparina/
llama-3-8b-instruct-ambiqt-infilling-lora

Name: llama-3-8b-instruct-ambiqt-infilling-lora
Availability: InStock

peft

safetensors

meta-llama/Llama-3.1-8B-Instruct

adapter

Views

No views yet

Model Card

Model Details

Model Description

This is a LoRA adapter for the meta-llama/Llama-3.1-8B-Instruct model, fine-tuned for the task of interpretation infilling in ambiguous text-to-SQL semantic parsing. The adapter is trained on the AmbiQT dataset, as described in the paper:

Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsing
Irina Saparina, Mirella Lapata
Findings of ACL 2025

The model is designed to take an ambiguous natural language question, database context, and a set of initial interpretations, and generate any missing interpretations in natural language. This modular approach improves coverage and recall for ambiguous queries in text-to-SQL tasks.

More details on how to train and evaluate are in this repo.

Developed by: Irina Saparina, Mirella Lapata
Model type: LoRA adapter for Llama 3.1 8B Instruct
Language(s): English
License: CC BY-NC-SA 4.0
Finetuned from model: meta-llama/Llama-3.1-8B-Instruct

Model Sources

Repository: https://github.com/saparina/disambiguate-then-parse
Paper: Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsing (Saparina & Lapata, Findings 2025)

Uses

Direct Use

This adapter is intended for use in modular semantic parsing pipelines, specifically for the interpretation infilling step. Given a database context, an ambiguous question, and a set of initial interpretations, the model generates any missing interpretations in natural language.

Example prompt:

The task is to review the provided context, question, and existing interpretations, and determine if any additional interpretations are missing. If there are missing interpretations, list them on separate lines without explanations. If all interpretations have already been covered, simply state: "All possible interpretations are covered."

Given the following context:

{}

Question:

{}

Existing interpretations:

{}

Provide any missing interpretations or confirm that all possible interpretations are covered.

How to Get Started with the Model

You can load the adapter using the PEFT library and Hugging Face Transformers:

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel, PeftConfig

base_model = "meta-llama/Llama-3.1-8B-Instruct"
adapter_path = "path/to/this/adapter"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, adapter_path)

The model should be combined with the generator of the initial interpretations (e.g., Llama 3.1 8B Instruct) and text-to-SQL parser (e.g., Qwen2.5 Coder 32B Instruct). See more details in the repo.

Training Details

Training Data

Dataset: AmbiQT (ambiguous text-to-SQL questions with multiple interpretations)
Data preprocessing: See code repository for details.

Training Procedure

Base model: meta-llama/Llama-3.1-8B-Instruct
Adapter: LoRA (PEFT)
LoRA rank: 16 (α=16)
Noise: NEFTune (α=5)
Epochs: 15
Batch size: 8
Learning rate: 5e-5 (cosine schedule, warmup ratio 0.01)
Weight decay: 0.01
Gradient clipping: 0.3
Hardware: NVIDIA A100 GPU
Framework: PEFT 0.14.0

Evaluation

Testing Data, Factors & Metrics

Test sets: AmbiQT, Ambrosia (out-of-domain)
Metrics: Single and full interpretation coverage, recall, precision (see paper for details)

Results

See Table 1, 7 in the paper for detailed results. The proposed system achieves state-of-the-art coverage and recall for ambiguous question interpretation infilling.

Technical Specifications

Architecture: Llama 3.1 8B Instruct + LoRA adapter (rank 16)
Objective: Supervised fine-tuning for interpretation infilling
Hardware Type: NVIDIA A100 GPU
Hours used: <10 hours per run

Citation

If you use this model, please cite:

BibTeX:

bibtex

@inproceedings{saparina2025disambiguate,
  title={Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsing},
  author={Saparina, Irina and Lapata, Mirella},
  booktitle={Findings of ACL},
  year={2025},
  url={https://aclanthology.org/2025.findings-acl.863/}
}

Model Card Contact

Irina Saparina

For more details, see the code repository and the paper.

irisaparina/llama-3-8b-instruct-ambiqt-infilling-lora