AI2sql
AI2sql is a state-of-the-art LLM for converting natural language questions to SQL queries.
Model Description
This model card presents the finetuning of the Mistral-7b model using the PEFT library and bitsandbytes for loading large models in 4-bit. The notebook demonstrates finetuning with Low Rank Adapters (LoRA), allowing only the adapters to be finetuned instead of the entire model. The process is designed for ease of use with Google Colab and is applicable for models supporting device_map.
Training Data
The finetuning involves a dataset on finance from Wikisql, using 10% of the data to showcase the process. The data is prepared in a prompt format for better comprehension by the model.
Training Procedure
The training involves several steps:
- Installing Necessary Packages: Installation of required libraries from their source.
- Model Loading: Using QLoRA quantization to load the model, reducing memory usage.
- Dataset Preparation: Tokenizing and splitting the dataset for training and testing.
- Applying LoRA: Utilizing PEFT for applying low-rank adapters to the model.
- Running the Training: Implementing training with specific arguments, showcasing the process with a demo setup.
- Evaluating the Model: Qualitative evaluation through inferences.
How to Use
The trained adapters can be shared on the Hugging Face Hub for easy loading. Users can directly load the adapters from the Hub and employ the model for tasks such as generating SQL queries.
Limitations and Bias
This finetuning process is specific to the Mistral-7b model and may not generalize to other models. The focus on finance data might limit the model's applicability to other domains.
Ethical Considerations
Users should be aware of potential biases in the training data, especially given its focus on finance, and should consider this when applying the model to real-world scenarios.
Acknowledgements
This work utilizes resources and tools from Hugging Face, including the PEFT library, bitsandbytes, and other associated libraries. The process is designed to be accessible and implementable using Google Colab.
Training procedure
The following bitsandbytes quantization config was used during training:
- quant_method: bitsandbytes
- load_in_8bit: False
- load_in_4bit: True
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: nf4
- bnb_4bit_use_double_quant: True
- bnb_4bit_compute_dtype: bfloat16
Framework versions
