TraceAlchemy-Gemma-4-E4B-Finance-IT BF16
TraceAlchemy-Gemma-4-E4B-Finance-IT is a finance-focused instruction-tuned Gemma 4 E4B model trained to improve careful financial reasoning, financial table understanding, unit and scale handling, sign and direction checks, and final-answer consistency.
This repository contains the merged BF16 model. The LoRA adapter has already been merged into the base model weights, so this repo can be loaded directly as a full Hugging Face Transformers model.
The model was fine-tuned and merged using
Unsloth.
Model Summary
This model was trained as a finance reasoning assistant with a focus on:
- Financial statement reasoning
- Revenue, margin, growth, and ratio calculations
- SEC-style table and excerpt extraction
- Unit and scale conversion, such as thousands to millions
- Sign and direction reasoning
- Multi-step table reasoning
- Final-answer consistency checking
- General finance instruction following
The goal of this run was not to teach the model static finance facts. Instead, the goal was to improve the model’s behavior on finance reasoning workflows. The training data emphasizes showing calculations, checking units, avoiding unsupported assumptions, and producing clear final answers.
Base Model
- Base model:
unsloth/gemma-4-E4B-it
- Fine-tuned model name:
TraceAlchemy-Gemma-4-E4B-Finance-IT
- Architecture class: Gemma 4 E4B instruction model
- Training method: LoRA supervised fine-tuning
- Final format in this repository: merged BF16 Hugging Face model
Gemma 4 E4B is treated as an effective E4B-class model. During training, the loaded parameter count was approximately 8.1B parameters including embeddings.
What This Repository Contains
This is the merged BF16 model, not just the LoRA adapter.
That means the training flow was:
1Base Gemma model
2+ trained LoRA adapter
3→ merged 16-bit model
4→ uploaded as full BF16 model
The important model file is:
This file contains the merged model weights. You should not expect to see:
1adapter_model.safetensors
2adapter_config.json
Those files belong in the LoRA adapter repository, not the merged BF16 repository.
Training Configuration
| Setting | Value |
|---|
| Base model | unsloth/gemma-4-E4B-it |
| Training method | LoRA SFT |
| Base loading during training | 8-bit |
| Merge format | BF16 |
| Max sequence length | 16,384 |
| LoRA rank | 64 |
| LoRA alpha | 64 |
| Learning rate | 2e-5 |
| Epochs | 1 |
| Per-device batch size | 2 |
| Gradient accumulation steps | 8 |
| Effective batch size | 16 |
| Optimizer | adamw_8bit |
| Training examples | 10,000 |
| Evaluation examples | 630 |
| Total training steps | 625 |
| Training runtime | ~2.93 hours on A100-class GPU |
Training Completion
The run completed successfully with the following final training output:
1global_step: 625
2epoch: 1.0
3train_runtime: 10558.7348 seconds
4train_samples_per_second: 0.947
5train_steps_per_second: 0.059
The final run completed all 625 planned training steps.
Validation and Evaluation During Training
The model was evaluated every 50 training steps on a held-out evaluation set of 630 examples.
Validation loss is reported as language-modeling loss on the held-out evaluation split. It is not the same thing as benchmark accuracy, but it is useful for checking whether the model is improving on examples it is not directly training on.
The validation trend was strong during the run:
| Step | Validation Loss |
|---|
| 50 | 0.765309 |
| 100 | 0.409274 |
| 150 | 0.318539 |
| 200 | 0.286068 |
| 250 | 0.271857 |
| 300 | 0.263259 |
| 350 | 0.257434 |
The validation loss dropped from 0.765309 at step 50 to 0.257434 by step 350, showing steady improvement on held-out finance examples during training.
This suggests the model was not only fitting the training examples, but also improving on the validation set.
More benchmarks soon.
Dataset Mix
The training set used a 10,000-example finance-focused mixture.
The data recipe was built around a real finance-data anchor from previous experimentation, then expanded with targeted synthetic examples designed to address specific finance reasoning failure modes.
Real Finance Data
| Source | Train | Eval |
|---|
| FinanceBench | 120 | 30 |
| TAT-QA | 2,200 | 100 |
| ConvFinQA | 1,800 | 100 |
| FinanceReasoning | 750 | 100 |
| Finance-Instruct-500k | 400 | 50 |
| Total real examples | 5,270 | 380 |
Synthetic Finance Data
The synthetic examples targeted specific reasoning skills:
| Synthetic Category | Train | Eval |
|---|
| Hard SEC-style extraction | 1,500 | 50 |
| Final-answer consistency checks | 1,200 | 50 |
| Sign, scale, and direction reasoning | 900 | 50 |
| Multi-step distractor tables | 800 | 50 |
| Hard unit-scale conversion | 330 | 50 |
| Total synthetic examples | 4,730 | 250 |
Final Dataset Size
| Split | Examples |
|---|
| Train | 10,000 |
| Eval | 630 |
Dataset Credits
This model was trained using examples derived from or inspired by the following datasets:
Additional synthetic examples were generated for targeted finance reasoning skills, including SEC-style extraction, scale and unit conversion, sign and direction reasoning, multi-step table reasoning, and final-answer consistency checks.
What Was Tested
During training, the model was tested through validation loss on a held-out evaluation split containing both real and synthetic finance examples.
The evaluation set included:
- FinanceBench-style question answering
- TAT-QA table and text reasoning
- ConvFinQA conversational finance QA
- FinanceReasoning-style calculation and reasoning examples
- Finance-Instruct examples
- Synthetic SEC-style extraction checks
- Synthetic unit and scale conversion checks
- Synthetic sign and direction checks
- Synthetic final-answer consistency checks
- Synthetic multi-step distractor table checks
This validation setup was intended to check whether the model could generalize beyond the exact training examples while staying focused on finance reasoning behavior.
Further external benchmark testing is planned.
More benchmarks soon.
Files in This Repository
The expected main files include:
1model.safetensors
2config.json
3tokenizer.json
4tokenizer_config.json
5processor_config.json
6chat_template.jinja
7README.md
The model.safetensors file is the full merged BF16 model.
For an approximately 8.1B parameter model, a BF16 model file around 16 GB is expected because BF16 uses about 2 bytes per parameter.
Loading the Model
Example loading code:
1import torch
2from transformers import AutoModelForCausalLM, AutoProcessor
3
4model_id = "trjxter/TraceAlchemy-Gemma-4-E4B-Finance-IT-bf16"
5
6processor = AutoProcessor.from_pretrained(model_id)
7
8model = AutoModelForCausalLM.from_pretrained(
9 model_id,
10 torch_dtype=torch.bfloat16,
11 device_map="auto",
12)
13
14messages = [
15 {
16 "role": "user",
17 "content": "A financial table says amounts are in thousands. Revenue is 302,578 and gross profit is 91,842. Express revenue in millions and calculate gross margin.",
18 }
19]
20
21inputs = processor.apply_chat_template(
22 messages,
23 tokenize=True,
24 add_generation_prompt=True,
25 return_tensors="pt",
26).to(model.device)
27
28outputs = model.generate(
29 input_ids=inputs,
30 max_new_tokens=256,
31 do_sample=False,
32)
33
34print(processor.decode(outputs[0], skip_special_tokens=False))
Depending on your local environment and installed Transformers version, Gemma 4 support may require a recent Transformers release.
Intended Use
This model is intended for finance reasoning and educational/research workflows, especially tasks involving:
- Financial statement reasoning
- SEC-style table and excerpt extraction
- Revenue, margin, growth, and ratio calculations
- Unit and scale conversion
- Sign and direction checks
- Final-answer consistency checking
- General finance instruction following
Limitations
This model should not be treated as a source of guaranteed financial truth.
It may still:
- Make calculation mistakes
- Misread financial tables
- Mis-handle units or scales
- Produce unsupported conclusions
- Hallucinate details not present in the prompt
- Fail on complex accounting, legal, or investment questions
For real investment, accounting, legal, or business decisions, verify outputs against original filings, audited statements, and qualified professionals.
Related Artifacts
This BF16 repository is part of the TraceAlchemy Gemma finance run.
Expected related repositories:
- LoRA adapter:
trjxter/TraceAlchemy-Gemma-4-E4B-Finance-IT-lora
- Merged BF16 model:
trjxter/TraceAlchemy-Gemma-4-E4B-Finance-IT-bf16
- GGUF model:
trjxter/TraceAlchemy-Gemma-4-E4B-Finance-IT-gguf
BF16 vs Adapter vs GGUF
This repo is the merged BF16 model.
| Repository Type | What It Contains | Use Case |
|---|
| LoRA adapter | Small adapter delta weights | Use with the original base model |
| Merged BF16 | Full merged model weights | Transformers/HF inference or further conversion |
| GGUF | Quantized merged model files | llama.cpp, LM Studio, Ollama, local inference |
This repository does not require loading the adapter separately because the adapter has already been merged into the full BF16 model weights.
Training and Merge Process
This model was fine-tuned and merged using
Unsloth.
The merge flow was:
1Base Gemma model + LoRA adapter
2→ merged 16-bit model
3→ saved as full BF16 Hugging Face model
4→ Hugging Face upload
Unsloth
This model was trained and merged with
Unsloth.
