cmarkea/bloomz-3b-dpo-table-qa-latex is a fine-tuned version of the cmarkea/bloomz-3b-dpo-chat model, specialized for table-based question answering (QA) tasks. The model has been trained on the table-vqa dataset, which was developed by Crédit Mutuel Arkéa, and it processes tables provided in their LaTeX source format.
This model is optimized for multilingual environments, supporting both French and English, and is especially effective in extracting and interpreting tabular data from documents. It has been fine-tuned for 2 days on an A100 40GB GPU and operates in bfloat16 precision to maximize resource efficiency.
Key Features
Domain: Table-based question answering (QA), particularly for extracting information from LaTeX-format tables.
Language Support: French and English, making it suitable for multilingual environments.
This model is highly applicable in fields where tabular data needs to be queried and analyzed, such as financial reports, academic papers, and technical documentation.
Usage
Here’s an example of how to use this model for table-based question answering:
python
1import torch
2from transformers import pipeline
34device =0if torch.cuda.is_available()else-15table ='''\begin{tabular}{|c|c|c|}
6 \hline
7 Model & MAE-{$TKE$}-low & MAE-{$TKE$}-high\\
8 \hline
9 U-FNET & 0.0048 & $1.09 \times 10^{-5}$\\
10 \hline
11 \end{tabular}'''12question ="What is the MAE-TKE-high value for the U-FNET model?"13prompt = table +'\n'+ question
1415model = pipeline("text-generation","cmarkea/bloomz-3b-dpo-chat", device=device)16result = model(f"</s>{prompt}<s>", max_new_tokens=512)17print(result)
The model processes tables written in LaTeX format, so be sure to provide your tables in that form.
Performance
This model was evaluated on 200 question-answer pairs extracted from 100 tables in the table-vqa test set. Each table had two question-answer pairs: one in French and one in English.
The evaluation used the LLM-as-Juries method, employing three judge models (GPT-4o, Gemini1.5 Pro, and Claude 3.5-Sonnet). The scoring was adapted to the table QA context, with a scale from 0 to 5 to ensure precision in assessing the model’s performance.
