richardr1126/

spider-skeleton-wizard-coder-ggml

Spider Skeleton Wizard Coder GGML Summary

• This model was created by finetuning WizardLM/WizardCoder-15B-V1.0 on an enhanced Spider context training dataset: richardr1126/spider-skeleton-context-instruct.
• Finetuning was performed using QLoRa on 3x RTX6000 48GB.
• This is the GGML version of the model.
• If you want just the QLoRa/LoRA adapter: richardr1126/spider-skeleton-wizard-coder-qlora

Running the GGML model

• The best way to run this model is to use the 4-bit GGML version on koboldcpp, with CuBlas support.

Spider Dataset

Spider Skeleton WizardCoder - test-suite-sql-eval Results

• ChatGPT was evaluated with the default hyperparameters and with the system message You are a sophisticated AI assistant capable of converting text into SQL queries. You can only output SQL, don't add any other text.
• Both models were evaluated with --plug_value in evaluation.py using the Spider dev set with database context.
  • --plug_value: If set, the gold value will be plugged into the predicted query. This is suitable if your model does not predict values. This is set to False by default.

Citations

@misc{luo2023wizardcoder,
      title={WizardCoder: Empowering Code Large Language Models with Evol-Instruct}, 
      author={Ziyang Luo and Can Xu and Pu Zhao and Qingfeng Sun and Xiubo Geng and Wenxiang Hu and Chongyang Tao and Jing Ma and Qingwei Lin and Daxin Jiang},
      year={2023},
}

@article{yu2018spider,
  title={Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task},
  author={Yu, Tao and Zhang, Rui and Yang, Kai and Yasunaga, Michihiro and Wang, Dongxu and Li, Zifan and Ma, James and Li, Irene and Yao, Qingning and Roman, Shanelle and others},
  journal={arXiv preprint arXiv:1809.08887},
  year={2018}
}

@article{dettmers2023qlora,
  title={QLoRA: Efficient Finetuning of Quantized LLMs},
  author={Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},
  journal={arXiv preprint arXiv:2305.14314},
  year={2023}
}

Disclaimer