WeatherGPT: Generate valid JSON from weather descriptions

Task

Example

Installation

Environment

Implementation

• Generate data
  • ChatGPT with few-shot variable examples
• Train model
  • QLoRA
  • DeepSpeed Stage 2
  • 4 bit quantization
  • Gradient checkpointing
  • Mistal AI 7B
• Evaluate
  • Output can be parsed
  • ChatGPT labeling

Reproduce

1. Install requirements

1. Make .env and fill it with your values (please take a look at .env.template)
2. [Optional] Generate data

1. Prepare data and model

1. Train the model

1. Fuse LoRA weights

1. Evaluate

Data generation

Why generated?

• ChatGPT was chosen for data collection because I couldn't find similar datasets and because this method scales to other domains and many companies will need to do this in one way or another
• Previously, NLP engineers suffered from the lack of datasets, but now they can be generated and this will serve as a great starting point for problem-solving
• The cost of compiling the initial dataset has decreased from thousands of dollars to a few bucks

Prompt

Example

Example of output

Results of data generation

• There were 5848 examples generated for training (including the validation set), which took about 10 minutes
• The ChatGPT model was used because it is 30 times cheaper and much faster. In real projects, I would use ChatGPT, GPT4, as well as open models, to obtain as diverse a dataset as possible
• The examples turned out to be quite lively and met the requirements of the task

Training

• Model: Mistral-7B-v0.1
• Boilerplate (QLoRA, DeepSpeed Stage 2, 4 bit quantization, Gradient checkpointing): xllm

Methods

• QLoRA (and 4 bit bnb quantization). The preferred method of fine-tuning, as it usually ensures a higher quality than full fine-tuning due to effective management of catastrophic forgetting. And of course, it optimizes the memory utilized during training
  • I use LoRA for all linear layers except lm_head and embeddings
    • The original paper does not investigate which layers are better to apply. Please check AdaLoRA paper
  • I also use a fairly high rank for low-rank optimization: 64 (alpha is 32)
• DeepSpeed Stage 2 (with CPU offloading). I'm using Deepspeed for training on multiple GPUs. Stage 2 was used because there are observed issues with the use of Stage 3 and quantized models
• Gradient Checkpointing. Very strong optimization of used memory at the expense of slowing down training

xllm details

• Prepare. The data preprocessing and model download step has been separated to avoid doing it on each of the processes in distributed learning mode
• Train. Training the model and saving checkpoints. I save checkpoints in HuggingFace Hub. Since I am training through LoRA, those weights are specifically saved.
• Fuse. Fusing LoRA weights with the backbone model and push it to HF Hub
• [Optional] Quantize. GPTQ quantization of the model

Task details

Results of the training

• LoRA weights
• Fused model
• W&B link

Evaluation

Metrics

Why no BLEU / ROUGE / etc?

Output can be parsed

ChatGPT labeling

ChatGPT labeling instruction

Results

ChatGPT labeling

Future works

• Improve evaluation
  • Need to add a method that compares the JSON response with the generated JSON. We know the types of fields. For numeric fields, you should use MSE, and for text fields, you should use the proximity of text embeddings, having previously selected the model
• If the quality of the current model is satisfactory, it should be deployed into production (in a quantized version) using TGI at least for a limited number of users.
• It is crucial to gather real data from production to fine-tune the model. Then these data need to be labeled, train a discriminator model (which would assess the quality of responses), filter the data and further train the model. For this task, I wouldn't apply RL, only the ReST ([link](It is crucial to gather real data from production to fine-tune the model. Then these data need to be labeled, train a discriminator model (which would assess the quality of responses), filter the data and further train the model. For this task, I wouldn't apply RL, only the ReST (link) method, which I would improve. Such actions on labeling and further training should be performed regularly. Ideally, an infrastructure for constant retraining should be developed. The recommended frequency depends on the traffic volumes. Usually, for manual re-learning the frequency is monthly, for automatic – weekly. Also, because we apply labeling, we can track model improvements. This will be particularly useful when the labeling instruction is stabilized. With the help of the discriminator we can adjust the hyperparameters for training and inference, for example, generation parameters. Also, with the discriminator, we can immediately assess several hypotheses from the generative model and deliver only the best one to the user. Currently, this method is not widely used due to the significantly increasing load on the generative model, so I would focus on the further training of the generative model using the discriminator.)) method, which I would improve. Such actions on labeling and further training should be performed regularly. Ideally, an infrastructure for constant retraining should be developed. The recommended frequency depends on the traffic volumes. Usually, for manual re-learning the frequency is monthly, for automatic – weekly. Also, because we apply labeling, we can track model improvements. This will be particularly useful when the labeling instruction is stabilized. With the help of the discriminator we can adjust the hyperparameters for training and inference, for example, generation parameters. Also, with the discriminator, we can immediately assess several hypotheses from the generative model and deliver only the best one to the user. Currently, this method is not widely used due to the significantly increasing load on the generative model, so I would focus on the further training of the generative model using the discriminator.
• If the quality of the current model is not satisfactory, similar steps will need to be taken with synthetic data, deploy it into production, and then perform the same steps with the data from production.