📣 P.S.: Please reach out to us, if you would be interested in supporting compute resources. Here are our recent achievements in LLM space: https://upaya.ai/
Model Details
Birbal-7B-V1 is fine-tuned on our curated dataset of 200k size for nearly 3 epochs. Our approach for dataset preparation is focused on finding most-relavant examples from large pool of tasks spanning across NLP, Maths, Commonsense, etc. Hence, we expect model to perform well on different tasks including unseen tasks.
If a record does not contain any instruction, here is the training format:
## Input:
<input>
## Response:
<response>
It will performed best if queried in the same way.
Downstream Use
Birbal-7B-V1 is fine-tuned on our curated dataset that contain examples from large number of tasks spanning across NLP, Maths, QA, etc. Hence, we expect the model to perform well on in general on various kinds of tasks.
How to Get Started with the Model
It is quite easy! Merge Birbal-7B-V1 peft model with Mistral-7B model and start running inference!
Training Details
We used Mistral-7B as a base model and fine-tuned it on a single RTX 4090 GPU for 24 hours as per the competition rules. Fine-tuning was performed using 4-bit QLoRA.
Training Data
Here is high-level diagram of our data preparation strategy:
