Model Name: BEMGPT
Finetuned From: Meta Llama 3.2 11B Vision Instruct
Developers: Xiaoqin Fu and Liang Zhang @ TensorBuild Lab, Department of Civil and Architectural Engineering and Mechanics (CAEM), University of Arizona
Release Year: 2025
Overview
BEMGPT is a large language model (LLM) fine-tuned specifically for Building Energy Modeling (BEM). It addresses the complexity of BEM data and the fragmented nature of domain knowledge by incorporating domain-specific expertise through fine-tuning.
By leveraging the generative capabilities of large language models, BEMGPT enhances knowledge representation, reasoning, and automation within the BEM workflow—making energy modeling more intelligent, efficient, and accessible.
EnergyPlus documentation such as engineering reference and input-output reference
Training Methods
LoRA (Low-Rank Adaptation)
QLoRA (Quantized Low-Rank Adaptation)
DoRA (Weight-Decomposed Low-Rank Adaptation)
......
3. Evaluation
Testing Data & Metrics
Metrics
BLEU: Measures token-level overlap between generated and reference texts
ROUGE: Recall-oriented similarity metrics, including ROUGE-1, ROUGE-2, and ROUGE-L
BERTScore: Embedding-based semantic similarity
Score evaluated by the OpenAI model gpt-4o (2024-11-20 version): Human-like correctness score (0.00–100.00)
Example GPT-4o Evaluation Prompt:
prompt = f"""
You are an expert evaluator and semantic grader. Grade the correctness of the provided answer as concise as possible on a decimal ranging from 0.00 to 100.00, where:
100.00 = Completely correct, perfectly matches the reference text, or provides necessary information and details
90.00 = Mostly correct, very accurate and comprehensive, minor details may be missing
80.00 = Moderately correct, mostly accurate but may have minor errors or omissions
70.00 = Partially correct, contains some inaccuracies, or misses information
60.00 = Only little correct, contains few accuracies, or misses most information
0.00 = Only repeated question, no answer, or completely incorrect, useless, and meaningless answer
Decimals are acceptable and encouraged to reflect partial correctness.
......
Use the reference text to evaluate the answer. If the reference text is empty, incomprehensible, unsuitable, or useless, please do not use the reference text and directly evaluate the answer.
Ignore any interrogative sentences within the answer. Return only the decimal score (e.g., 97.23), with no additional text.
Instruction: {instruction}
Question: {question}
Answer: {provided_answer}
Reference Text: "{text}"
"""
Domain-Specific Fine-Tuning for BEM: BEMGPT is the first large language model fine-tuned specifically for BEM, addressing the lack of domain expertise in general-purpose LLMs.
Embedded Domain Knowledge: Unlike Retrieval-Augmented Generation (RAG), BEMGPT integrates domain knowledge directly into the model parameters, making it a cost-effective alternative that reduces inference latency and dependency on external databases.
Foundation for Intelligent, AI-Driven BEM Tools: BEMGPT establishes a new foundation for intelligent automation in BEM workflows, supporting more efficient, scalable, and accessible applications across the building energy sector.
6. Who should use this model?
Design and construction professionals
Architects and designers
Mechanical, Electrical, and Plumbing (MEP) engineers
