CommerceExpert v1
Fine-tuned LLMs for Salesforce Commerce Cloud error classification
Three open-source models (Llama-3.2-1B, LFM-2.5-Thinking, Qwen-2.5-7B) trained on curated SFCC error data for real-time error categorization and event classification.
📋 Overview
CommerceExpert classifies SFCC runtime errors into standard categories. Choose the model that fits your deployment:
Llama-3.2-1B — Lightweight, edge deployment (Cloudflare Workers, on-device)LFM-2.5-Thinking — Reasoning-focused, explainable predictions (audit trails)Qwen-2.5-7B — Highest accuracy, GPU-required (cloud, batch processing)
All three models are trained on the same dataset and ready for production integration.
🎯 Use Cases
✅ Real-time SFCC error classification
📊 Model Details
Property Value Training Data 1,000 SFCC errors (200 real + 800 synthetic) Eval Data 300 held-out examples Error Classes 9 categories (Network, Auth, Quota, Validation, NotFound, Payment, InternalError, BusinessLogic, Integration) Fine-Tune Method LoRA (r=16, alpha=32) Training Epochs 1 (smoke test) Format Alpaca (Llama, Qwen), ChatML (LFM) License Apache 2.0
🚀 Quick Start
Installation
pip install transformers peft unsloth torchLoad & Infer (Llama)
1 from unsloth import FastLanguageModel
2
3 model , tokenizer = FastLanguageModel . from_pretrained (
4 model_name = "darleison/CommerceExpert-Llama-v1" ,
5 max_seq_length = 2048 ,
6 load_in_4bit = True ,
7 )
8 model = FastLanguageModel . for_inference ( model )
9
10 # Classify an error
11 error_json = '{"code": "ECONNREFUSED", "message": "Connection refused", "context": "api-gateway"}'
12 prompt = f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
13
14 ### Instruction:
15 Classify the following Salesforce Commerce Cloud runtime error into a standard category.
16
17 ### Input:
18 { error_json }
19
20 ### Response:
21 """
22
23 inputs = tokenizer ( prompt , return_tensors = "pt" ) . to ( "cuda" )
24 outputs = model . generate ( ** inputs , max_new_tokens = 16 )
25 result = tokenizer . batch_decode ( outputs ) [ 0 ]
26 print ( result ) # → "Network"
Load & Infer (Qwen)
1 model , tokenizer = FastLanguageModel . from_pretrained (
2 model_name = "darleison/CommerceExpert-Qwen-v1" ,
3 max_seq_length = 4096 ,
4 load_in_4bit = True ,
5 )
6 # Same inference as above
Load & Infer (LFM - with reasoning)
1 model , tokenizer = FastLanguageModel . from_pretrained (
2 model_name = "darleison/CommerceExpert-LFM-v1" ,
3 max_seq_length = 4096 ,
4 load_in_4bit = True ,
5 )
6 # Same inference, longer output includes reasoning chain
7 outputs = model . generate ( ** inputs , max_new_tokens = 256 )
Error Categories
The model predicts one of 9 error classes:
Network — Connection, DNS, timeout errorsAuthentication — Auth failures, invalid tokens, permission deniedQuota — Rate limits, API quotas, concurrent limitsValidation — JSON, schema, required field errorsNotFound — 404, missing resourcesPayment — Payment processing failuresInternalError — Server errors, crashes, exceptionsBusinessLogic — Out of stock, invalid coupon, shipping unavailableIntegration — Webhook failures, external service errors
📈 Performance
Accuracy (on 300-example golden test set)
Model Baseline Fine-Tuned Improvement Llama-3.2-1B [INSERT]% [INSERT]% [INSERT]+% LFM-2.5-Thinking [INSERT]% [INSERT]% [INSERT]+% Qwen-2.5-7B [INSERT]% [INSERT]% [INSERT]+%
Results from evaluation on held-out test set. Baseline = zero-shot pre-trained model.
Inference Speed
Model Tokens/sec (GPU) Tokens/sec (CPU) Model Size Llama-3.2-1B ~500 ~50 ~650 MB LFM-2.5-Thinking ~300 ~30 ~650 MB Qwen-2.5-7B ~100 ~10 ~3.5 GB
⚙️ Training Configuration
1 LoRA :
2 rank : 16
3 alpha : 32
4 dropout : 0
5 target_modules : [ q_proj , k_proj , v_proj , o_proj , gate_proj , up_proj , down_proj ]
6
7 Training :
8 epochs : 1
9 batch_size : 2 (per device)
10 gradient_accumulation_steps : 4
11 learning_rate : 2e-4
12 optimizer : adamw_8bit
13 scheduler : linear
14 warmup_steps : 5
15 weight_decay : 0.01
16
17 Data :
18 seed : 42 (reproducible)
19 train : 1000 examples (800 synthetic , 200 real)
20 test : 300 examples (held - out)
🔬 Data Generation
Training data expanded from seed dataset using:
Error Taxonomy — 9 categories with subcategoriesSynthetic Generation — Realistic error patterns from templatesCartridge Integration — Error codes from 18 real SFCC integrations (payment, shipping, fraud, etc.)B2C Patterns — Real log examples and error formats
Final dataset: balanced class distribution, diverse error contexts, realistic for production SFCC systems.
📦 Integration with PulsarJS
CommerceExpert integrates as the ML classification layer in PulsarJS , an event-centric knowledge graph platform for ecommerce attribution.
1 from unsloth import FastLanguageModel
2
3 class CommerceExpertClassifier :
4 def __init__ ( self , model_name = "llama" ) :
5 models = {
6 "llama" : "darleison/CommerceExpert-Llama-v1" ,
7 "qwen" : "darleison/CommerceExpert-Qwen-v1" ,
8 "lfm" : "darleison/CommerceExpert-LFM-v1" ,
9 }
10 self . model , self . tokenizer = FastLanguageModel . from_pretrained (
11 model_name = models [ model_name ] ,
12 max_seq_length = 4096 ,
13 load_in_4bit = True ,
14 )
15 self . model = FastLanguageModel . for_inference ( self . model )
16
17 def classify ( self , error_dict ) :
18 """Classify an SFCC error."""
19 prompt = f"""...[format prompt]..."""
20 inputs = self . tokenizer ( prompt , return_tensors = "pt" ) . to ( "cuda" )
21 outputs = self . model . generate ( ** inputs , max_new_tokens = 16 )
22 return self . tokenizer . batch_decode ( outputs ) [ 0 ]
23
24 # Usage
25 classifier = CommerceExpertClassifier ( "llama" )
26 error = { "code" : "E_TIER_QUOTA" , "message" : "Quota exceeded" }
27 category = classifier . classify ( error ) # → "Quota"
⚠️ Limitations
Single epoch training — Smoke test only. Production use may benefit from multi-epoch training.Synthetic data — Training data includes generated examples. Real-world error variations may differ.SFCC-specific — Domain-specific to Salesforce Commerce Cloud. Performance on generic errors may be lower.Language — English only. Non-English error messages not tested.Model selection — Choose based on deployment constraints (edge vs. accuracy vs. explainability trade-off).
🔄 Roadmap
Version Focus Status v1.0 Initial release, 9 error classes ✅ Live v1.1 Multi-epoch training, expanded taxonomy 🔜 Planned v2.0 Event classification, remediation suggestions 💡 Future
📚 Reproducibility
All training code and data generation scripts are included:
src/data/data_generator.py — Synthetic error generationsrc/data/cartridge_extractor.py — Extract from GitHub cartridgessrc/data/data_validator.py — Quality assurancesrc/data/data_pipeline.py — Full pipeline orchestrationsrc/eval/eval_compare_models.py — Model evaluation
Training notebooks (Colab) available in training/notebooks/.
📝 Citation
1 @software{CommerceExpert2026,
2 title={CommerceExpert v1: Fine-Tuned LLMs for SFCC Error Classification},
3 author={Filho, Darleison},
4 year={2026},
5 month={April},
6 url={https://huggingface.co/darleison/sfcc-commerce-expert-v1}
7 }
📄 License
Apache 2.0 — Free for research, commercial, and derivative use. See
LICENSE for details.
🙋 Support
Ready to use. Fully reproducible. Open source.
