Llama 3 8 billion model was finetuned using unsloth package on a cleaned Bangla alpaca dataset. After that the model was quantized in 4-bit. The model is finetuned for 2 epoch on a single T4 GPU.
Pros & Cons of the Model
Pros
The model can comprehend the Bangla language, including its semantic nuances
Given context model can answer the question based on the context
Cons
Model is unable to do creative or complex work. i.e: creating a poem or solving a math problem in Bangla
Since the size of the dataset was small, the model lacks lot of general knowledge in Bangla
