This is my first shot at training a LLM. Consider this a work in progress. My goal is to produce a fully working LLM geared toward bookkeeping. I'm looking for more bookkeeping and accounting datasets to make this a better product.
Now for the deets....
Qwen3-14B-Bookkeeper-v1
Qwen3-14B trained to be a Specialized AI Bookkeeper – Fine-Tuned for Real-World Accounting & Bookkeeping (see below)
Overview
Qwen3-14B-Bookkeeper-v1 is a powerful, fine-tuned version of Qwen3-14B designed specifically for bookkeepers, accountants, and small business owners.
It excels at everyday bookkeeping tasks such as:
Categorizing bank transactions
Suggesting accurate double-entry journal entries
Detecting potential fraud or anomalies
Explaining accounting rules and standards
Reconciling balances and preparing simple reports
Answering professional accounting questions
The model retains Qwen3's excellent reasoning abilities while becoming highly specialized in accounting workflows.
Key Features
Transaction Mastery – Handles real-world descriptions, amounts, and balances with precise categorization and journal entry suggestions
Fraud Detection – Flags suspicious patterns based on trained synthetic financial data
Professional Knowledge – Understands GAAP concepts, expense types, income recognition, and more from exam-level training
Natural & Helpful Chat – Responds clearly and conversationally, perfect for client-facing or team use
Efficient & Fast – Runs smoothly on consumer GPUs (e.g., RTX 4090) or via Ollama/LM Studio after GGUF quantization
Training Data
This model was fine-tuned on a carefully balanced mixture of high-quality datasets (total ~100k examples):
Dataset Purpose Examples Used Link
unsloth/OpenMathReasoning-mini Preserve strong step-by-step reasoning Full cot split Link
mlabonne/FineTome-100k General conversational fluency Full train split Link
kohdified/synthetic-financial-data Core transaction categorization, journal entries & fraud detection 40,000 sampled & formatted Link
gbharti/finance-alpaca Finance & accounting Q&A and explanations Full dataset Link
brucewlee1/mmlu-professional-accounting Deep professional accounting knowledge Full available split Link
Mix ratio: ≈70% bookkeeping/finance data + 25% reasoning + 5-10% general conversation
Fine-tuning performed with Unsloth on Google Colab A100/H100 (QLoRA, 16-bit merge, GGUF export).
qwen3-14b-bookkeeper-gguf-q4km : GGUF
This model was finetuned and converted to GGUF format using Unsloth.
Example usage:
For text only LLMs: ./llama.cpp/llama-cli -hf Myyyyyyyyyyyyyy/qwen3-14b-bookkeeper-gguf-q4km --jinja
For multimodal models: ./llama.cpp/llama-mtmd-cli -hf Myyyyyyyyyyyyyy/qwen3-14b-bookkeeper-gguf-q4km --jinja
Available Model files:
qwen3-14b.Q4_K_M.gguf
Ollama
An Ollama Modelfile is included for easy deployment.
This was trained 2x faster with Unsloth
