A fine-tuned Qwen3-14B specialized in Revit API code generation, IFC reasoning, and BIM development patterns.
An experiment in domain-specific fine-tuning demonstrating that focused training on 177,127 Revit/BIM examples can produce a specialized model for $48 in 8 hours on a single GPU. The model was validated on a 40-question Revit C# benchmark, showing competitive performance with frontier models on domain-specific tasks.
Key insight: This demonstrates the value of domain-specific fine-tuning for specialized use cases rather than claiming superiority over general-purpose frontier models.
QLoRA rank 64: Balances expressiveness with efficiency for domain-specific patterns
Packing enabled: Maximizes GPU utilization on variable-length sequences
Cosine schedule + warmup: Stable learning on technical documentation
Low dropout: Unsloth fast patching requires 0 dropout for optimized training
4096 context: Covers typical Revit API code examples with context
Dataset Splits
Split
Examples
Percentage
Purpose
Train
159,414
90%
Model training
Validation
8,856
5%
Hyperparameter tuning & early stopping
Test
8,857
5%
Final evaluation (not used in this benchmark)
Split strategy: Stratified sampling by domain to maintain proportional representation across all 6 BIM domains. Random seed: 42.
Note: The 40-question benchmark is separate from the training data—it tests zero-shot generalization on new Revit API questions not seen during training.
Training Data Distribution
Data Distribution
Domain
Records
%
Description
revit_csharp
143,060
72.7%
Revit API C# code from docs, examples, references
ifc_reasoning
44,571
22.6%
IFC topology, spatial hierarchies, BIM reasoning
aps_schema
4,980
2.5%
APS/Forge cloud API patterns
revit_patterns
3,758
1.9%
Development patterns (IUpdater, events, filters)
revit_python
285
0.1%
pyRevit Python automation
mcp_tools
149
0.1%
MCP tool definitions for AI-BIM integration
Why this distribution?
72.7% revit_csharp: Reflects the primary use case—Revit plugin development is predominantly C#/.NET
22.6% ifc_reasoning: BIM data exchange and interoperability are core to AEC workflows
Domain-tagged system prompts: Each domain uses specialized prompts to activate appropriate model behaviors
Data format: ChatML with domain-specific system prompts. Each record includes <|im_start|>system, <|im_start|>user, <|im_start|>assistant sections.
Key takeaway: Domain-specific fine-tuning achieved competitive performance with <3% of the compute required to train frontier models from scratch.
Intended Use
Primary: Domain-specialized Revit API code generation. This is an experiment demonstrating that domain-specific fine-tuning can achieve competitive results with significantly less compute than training frontier models from scratch.
Validate Revit API usage (catch missing Transactions, null checks, type filter issues)
Reason about IFC spatial hierarchies and property sets
Produce Revit development patterns (IExternalEventHandler, IUpdater, ISelectionFilter)
Limitations:
Optimized for Revit 2025/2026 (.NET 8) API, may not cover older API versions
Strongest on revit_csharp domain; weaker on IFC STEP format generation
Best results under 800 tokens; quality may degrade on very long outputs
Not a general-purpose coding model; use frontier models for non-Revit tasks
The benchmark comparison is asymmetric (fine-tuned vs. zero-shot); Claude with proper system prompts may perform differently
Benchmark Results
40-question Revit C# benchmark - pure code generation focused on practical API usage:
Model
Avg Score
Questions Scored Higher
Parameters
Inference
revit-coder-14b
0.800
25 of 40
14B
Local (Ollama)
Claude Opus 4.6
0.793
15 of 40
~100B+
API
Note: This comparison shows a fine-tuned specialist vs. a zero-shot generalist. The fine-tuned model naturally has advantages on this specific benchmark. In production with proper prompting and examples, Claude may outperform on complex tasks.
By Difficulty
Difficulty
Count
revit-coder-14b
Claude Opus 4.6
Notes
Easy
9
0.800
0.796
Similar performance
Medium
19
0.839
0.801
Fine-tuned model shows strength on practical patterns
Hard
12
0.736
0.779
Claude shows strength on complex multi-class problems
Average Scores by Difficulty
Scoring Components Breakdown
All 40 questions and both models' full responses are published in BENCHMARK_FULL.md.
Benchmark Methodology
Data Independence: The 40 benchmark questions were held out from training data to ensure fair evaluation.
Automated Scoring: Each response is scored on three axes:
Signal Presence (40%): Fraction of expected domain keywords found (e.g., FilteredElementCollector, Transaction, IfcRelAggregates)
Code Quality (30%): Domain-specific structural checks (namespaces, class structure, API patterns)
Important: No reference answers or human evaluation were used. Scores reflect structural patterns, not compilation or execution. This is automated evaluation only.
Asymmetric Comparison: The fine-tuned model received domain training; Claude did not. This tests whether domain-specific fine-tuning provides value, not which model is "better."
Usage
Ollama (Recommended)
bash
1# Pull or create the model2ollama run revit-coder-14b-f16
34# Query5ollama run revit-coder-14b-f16 "Write C# code to collect all walls and group by type name"
Python (transformers)
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
23model_name ="schauh11/revit-coder-14b"# HuggingFace repo4tokenizer = AutoTokenizer.from_pretrained(model_name)5model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")67messages =[8{"role":"system","content":"You are a Revit API expert specialized in C# and .NET 8."},9{"role":"user","content":"Write code to get all rooms and their areas."},10]11text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)12inputs = tokenizer(text, return_tensors="pt").to(model.device)13outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.1)14print(tokenizer.decode(outputs[0], skip_special_tokens=True))
