You can deploy this model and start earning money today!

NeshVerse/
Ultra_Flash_Financial_SFT_Nanbeige_4.1-3B

Name: Ultra_Flash_Financial_SFT_Nanbeige_4.1-3B
Availability: InStock

Views

No views yet

Ultra Flash Financial Sales Assistant

A Specialized Model for Financial Analysis & Sales Forecasting

Sequentially merged double-LoRA model built on Nanbeige4.1-3B

🚀 Model Overview

Ultra Flash Financial Sales Assistant is a specialized language model created by sequentially merging two domain-specific LoRA adapters onto the Nanbeige/Nanbeige4.1-3B foundation. This model combines deep financial analysis capabilities with advanced sales forecasting expertise, delivering a unified solution for business intelligence tasks.

This is a merge of two models:

The base model Nanbeige/Nanbeige4.1-3B
The financial adapter NeshVerse/Flash_Financial_SFT_Nanbeige_4.1-3B, followed by a sequentially merged sales adapter

Model Architecture

text

┌─────────────────────────────────────┐
│    Nanbeige4.1-3B (Base Model)   │
│         (3.9B parameters)        │
└───────────────┬─────────────────────┘
                │
    ┌───────────▼───────────┐
    │  Financial LoRA(r=4)│
    │  37,463 samples     │
    │  Loss: 0.52         │
    └───────────┬───────────┘
                │
    ┌───────────▼───────────┐
    │   Sales LoRA (r=8)  │
    │  37,463 samples     │
    │  Loss: 0.49         │
    └───────────┬───────────┘
                │
    ┌───────────▼───────────┐
    │   Merged Final Model│
    │  Perplexity: 1.60   │
    └───────────────────────┘

📊 Training Details

Adapter 1: Financial Domain

Source: NeshVerse/financial-sft-dataset (37,463 examples)
Training Time: 222.7 minutes (3.7 hours)
Initial Loss: 3.91 → Final Loss: 0.5178
Final Perplexity: 1.69
LoRA Configuration: r=4, alpha=8
Trainable Parameters: 1.7M (0.043% of base)

Adapter 2: Sales Domain

Source: NeshVerse/sales-forecast-dataset (37,463 examples)
Training Time: 232.8 minutes (3.9 hours)
Initial Loss: 0.61 → Final Loss: 0.4888
Best Validation Loss: 0.4887 (Step 4,600)
Final Perplexity: 1.60
LoRA Configuration: r=4 → r=8 (zero-padded), alpha=8

Combined Training Summary

Metric	Value
Total Training Time	455.5 minutes (7.6 hours)
Total Training Samples	74,926
Total Steps	9,366
Final Perplexity	1.60
Final Eval Loss	0.4723
Model Size (merged)	4 GB
Framework	Unsloth (2x faster, 80% less memory)
Precision	4-bit QLoRA (training & inference)

💡 Capabilities

Financial Analysis

📊 Numerical Aggregation - Sum, average, count, variance calculations
📈 Temporal Analysis - Monthly, quarterly, and annual summaries
📑 Structured Data Parsing - Extract insights from tables and reports
💰 Revenue Analysis - Growth trends, profit margin calculations

Sales Forecasting

🔮 Trend Prediction - Pattern recognition and future projections
📅 Quarterly Analysis - Seasonal adjustments and performance metrics
📊 Product Performance - SKU-level trend analysis
💵 Revenue Projections - Data-driven forecasting models

Combined Intelligence

📊 Financial Impact of Sales Trends
💹 Integrated Revenue Forecasting
📈 Business Intelligence Insights

🔧 Quick Start

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "NeshVerse/Ultra_Flash_Financial_SFT_Nanbeige_4.1-3B"

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.float16,
    trust_remote_code=True,
)

# Financial Analysis Example
financial_prompt = """Analyze the quarterly performance:
Q1 2024: Revenue $2.1M, Costs $1.2M
Q2 2024: Revenue $2.4M, Costs $1.3M
Q3 2024: Revenue $2.9M, Costs $1.5M

Calculate:
1. Quarterly growth rates
2. Average profit margin
3. Q4 revenue projection at current growth"""

inputs = tokenizer(financial_prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=250,
    temperature=0.7,
    do_sample=True,
    top_p=0.9
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

📈 Performance Metrics

Validation Progress (Sales Adapter)

Step	Train Loss	Val Loss	Gap	Perplexity
200	0.5516	0.5373	-0.0143	1.71
600	0.5331	0.5282	-0.0049	1.70
1,000	0.5224	0.5227	+0.0003	1.69
2,000	0.5030	0.5086	+0.0056	1.66
3,400	0.4999	0.4948	-0.0051	1.64
4,600	0.4888	0.4887	-0.0001	1.63

Model Comparison

Model	Perplexity	Eval Loss	Training Time
Base (Nanbeige4.1-3B)	>2.5	>0.9	-
Flash Financial (First)	1.69	0.52	222.7 min
Ultra Flash (Merged)	1.60	0.47	455.5 min

🔄 Merging Process

python

# Sequential LoRA merge implementation
from transformers import AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch

# 4-bit quantization config
quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True
)

# Load base model
model = AutoModelForCausalLM.from_pretrained(
    "Nanbeige/Nanbeige4.1-3B",
    quantization_config=quant_config,
    device_map="auto",
    trust_remote_code=True
)

# Merge Financial LoRA (r=4)
model = PeftModel.from_pretrained(
    model, 
    "NeshVerse/Flash_Financial_SFT_Nanbeige_4.1-3B"
)
model = model.merge_and_unload()

# Convert and merge Sales LoRA (r=4 → r=8)
# (Zero-padding applied for rank conversion)
model = PeftModel.from_pretrained(
    model, 
    "path/to/converted-sales-lora"
)
model = model.merge_and_unload()

# Save merged model
model.save_pretrained("ultra-flash-merged")
tokenizer.save_pretrained("ultra-flash-merged")

⚠️ Limitations & Considerations

4-bit Precision: Minor quality degradation from quantization (estimated 2-3% accuracy loss).
Rank Conversion: Sales LoRA padded from r=4 to r=8 may have slight fidelity reduction.
Context Window: Optimized for 256-512 tokens (expandable to 1024 with performance trade-offs).
Domain Specificity: Best performance on financial and sales tasks; general tasks may underperform.
Language Support: Optimized for English and Chinese financial terminology.
Hardware Requirements: 4GB VRAM minimum for 4-bit inference.
Intended Use: This model is designed for professional business and financial analysis assistance. It is not suitable for providing actual financial, investment, or legal advice.

📚 Dataset Information

Financial Dataset (NeshVerse/financial-sft-dataset): 37,463 examples covering revenue reports, balance sheets, and financial statements. Tasks include aggregation, trend analysis, and ratio calculations. Languages: English (70%), Chinese (30%).
Sales Dataset (NeshVerse/sales-forecast-dataset): 37,463 examples covering sales records, forecasts, and product performance. Tasks include time series prediction, seasonal analysis, and projections. Languages: English (80%), Chinese (20%).

🏆 Use Cases

Quarterly Business Reviews - Generate comprehensive performance summaries.
Sales Pipeline Analysis - Forecast conversion rates and revenue impact.
Financial Statement Analysis - Extract key metrics from complex documents.
Market Trend Prediction - Combine historical sales with market indicators.

📖 Citation

bibtex

@misc{ultra_flash_financial_2024,
  title={Ultra Flash Financial Sales Assistant: Double LoRA Sequential Merge},
  author={NeshVerse},
  year={2024},
  publisher={Hugging Face},
  journal={Hugging Face Model Hub},
  howpublished={https://huggingface.co/NeshVerse/Ultra_Flash_Financial_SFT_Nanbeige_4.1-3B},
  note={Sequential merge of Financial (r=4, 37K samples) and Sales (r=8, 37K samples) LoRA adapters on Nanbeige4.1-3B using Unsloth optimization.}
}

🤝 Acknowledgments

Base Model: Nanbeige/Nanbeige4.1-3B
First Adapter: NeshVerse/Flash_Financial_SFT_Nanbeige_4.1-3B
Framework: Unsloth for 2x faster training with 80% less memory
Compute: Google Colab Pro (Tesla T4 GPU)
Libraries: Transformers, PEFT, BitsAndBytes, PyTorch

Version 1.0.0 | Last Updated: 2024-02-26

Total Training Investment: 7.6 hours, 75K samples

License: Apache 2.0

Built with ❤️ for the open-source AI community