This is my first shot at training a LLM. Consider this a work in progress. My goal is to produce a fully working LLM geared toward bookkeeping.
I'm looking for more bookkeeping and accounting datasets to make this a better product.
Now for the deets....
Qwen3-14B-Bookkeeper-v1
Qwen3-14B trained to be a Specialized AI Bookkeeper – Fine-Tuned for Real-World Accounting & Bookkeeping (see below)
Overview
Qwen3-14B-Bookkeeper-v1 is a powerful, fine-tuned version of Qwen3-14B designed specifically for bookkeepers, accountants, and small business owners.
It excels at everyday bookkeeping tasks such as:
- Categorizing bank transactions
- Suggesting accurate double-entry journal entries
- Detecting potential fraud or anomalies
- Explaining accounting rules and standards
- Reconciling balances and preparing simple reports
- Answering professional accounting questions
The model retains Qwen3's excellent reasoning abilities while becoming highly specialized in accounting workflows.
Key Features
- Transaction Mastery – Handles real-world descriptions, amounts, and balances with precise categorization and journal entry suggestions
- Fraud Detection – Flags suspicious patterns based on trained synthetic financial data
- Professional Knowledge – Understands GAAP concepts, expense types, income recognition, and more from exam-level training
- Natural & Helpful Chat – Responds clearly and conversationally, perfect for client-facing or team use
- Efficient & Fast – Runs smoothly on consumer GPUs (e.g., RTX 4090) or via Ollama/LM Studio after GGUF quantization
Training Data
This model was fine-tuned on a carefully balanced mixture of high-quality datasets (total ~100k examples):
| Dataset | Purpose | Examples Used | Link |
|---|
| unsloth/OpenMathReasoning-mini | Preserve strong step-by-step reasoning | Full cot split | Link |
| mlabonne/FineTome-100k | General conversational fluency | Full train split | Link |
| kohdified/synthetic-financial-data | Core transaction categorization, journal entries & fraud detection | 40,000 sampled & formatted | Link |
| gbharti/finance-alpaca | Finance & accounting Q&A and explanations | Full dataset | Link |
| brucewlee1/mmlu-professional-accounting | Deep professional accounting knowledge | Full available split | Link |
Mix ratio: ≈70% bookkeeping/finance data + 25% reasoning + 5-10% general conversation
Fine-tuning performed with Unsloth on Google Colab A100/H100 (QLoRA, 16-bit merge, GGUF export).
Ollama
An Ollama Modelfile is included for easy deployment.
This was trained 2x faster with
Unsloth
