Physical control systems, such as building HVAC systems, thermal processes, and power-grid assets, often involve delayed dynamics, safety constraints, and costly online interaction. Because the effect of a control action may appear after several time steps, effective control often requires reasoning over histories of states, actions, and outcomes. Pretrained sequence controllers, such as Decision Transformers, provide a natural way to model this history-dependent control problem by learning policies from offline trajectories, reducing the need for expensive online retraining. However, their performance can degrade when deployment conditions differ from the training data. In building control, for example, actuator wear, sensor drift, occupancy changes, and weather variation can all affect the observed response of the system to the same control action. Online adaptation is therefore needed, but raw prediction error is not always a reliable signal for updating the controller. The key challenge is to identify which part of the observed mismatch is action-sensitive and should guide policy adaptation.
To address this challenge, we propose DARPAN, a Drift-Aware Residual Policy Adapter Network for pretrained physical control. DARPAN keeps the pretrained Decision Transformer frozen and performs online adaptation only through lightweight residual adapters, preserving the offline policy and limiting the number of trainable parameters. Instead of updating from raw prediction error, DARPAN uses a controllability filter trained in simulation to extract the action-sensitive part of the residual and filter out mismatch that is weakly related to the controller's actions. During deployment, a frozen predictor estimates the next state, the controllability filter produces a filtered residual, and a short history of filtered residuals forms a drift signature. This signature is used to select an existing adapter or create a new one when a persistent drift pattern appears. We benchmark DARPAN on SustainGym under actuator faults, sensor drift, delays, and external disturbances.
Code Submission
Anonymized source code accompanying the paper. The codebase implements a
deployment-time adapter for offline-pretrained Decision Transformers, evaluated
on two domains from the SustainGym benchmark: a multi-zone building HVAC
environment (building_env/) and a combined heat-and-power cogeneration
environment (cogen_env/).
The two domains share an identical method but live in separate packages because
their state and action spaces differ. Each package contains:
Frozen backbone. A multi-head Decision Transformer pretrained on offline
trajectories and held fixed at deployment.
Frozen forward predictor / world model. A separately trained dynamics
model used as a differentiable surrogate for the deployment-time update.
Frozen controllability filter. A temporal filter that produces a
per-state mask separating action-influenced residual components from
exogenous disturbance.
Adapter library. A small library of zero-initialized residual adapters
routed by similarity in a drift signature space; new adapters are created
when no existing entry matches.
Baselines. Frozen DT, online finetuned DT, LoRA, hyper-network
conditioning, single-adapter, online RL, and a rule-based controller.
The code targets PyTorch 2.x with CUDA. CPU execution is supported.
The sustaingym package provides the underlying BuildingEnv and CogenEnv
implementations and is required for environment construction.
Configuration
All deployment-time hyperparameters for the building-HVAC experiments are
declared in building_env/configs/main.yaml. The cogeneration runner reads its
configuration from constants at the top of cogen_env/experiments/run_protocol.py.
