We introduce ADAM, An emboDied causal Agent in Minecraft, that can autonomously navigate the open world, perceive multimodal contexts, learn causal world knowledge, and tackle complex tasks through lifelong learning. ADAM is empowered by four key components: 1) an interaction module, enabling the agent to execute actions while documenting the interaction processes; 2) a causal model module, tasked with constructing an ever-growing causal graph (i.e., technology tree) from scratch, which enhances interpretability and diminishes reliance on prior knowledge; 3) a controller module, comprising a planner, an actor, and a memory pool, which uses the learned causal graph to accomplish tasks; 4) a perception module, powered by multimodal large language models, which enables ADAM to perceive like a human player. Our project page is at https://opencausalab.github.io/ADAM

Extensive experiments show that ADAM accurately constructs an almost perfect causal graph, enabling effective task decomposition and execution with increased efficiency and strong interpretability. Additionally, in modified Minecraft scenarios where no prior knowledge is available, ADAM maintains its performance and shows remarkable robustness and generalization capability.

We provide the implementation of ADAM in this repo. We employ Mineflayer, a JavaScript-based framework designed for integration with the commercial Minecraft. For visual processing, we utilize prismarine-viewer, an API for visualizing Minecraft content. The encapsulation of the Mineflayer uses the implementation in VOYAGER.

ADAM requires Python ≥ 3.9 and Node.js ≥ 16.13.0. Our experiment tested the Minecraft environment on the Windows platform. The Azure Cloud Service is also available in the implementation.

git clone https://github.com/OpenCausaLab/ADAM
cd ADAM
pip install -e .

In addition to the Python dependencies, you need to install the following Node.js packages:

cd env/mineflayer
npm install -g npx
npm install
cd mineflayer-collectblock
npx tsc
cd ..
npm install

For users from mainland China, we recommend the following installation method:

cd env/mineflayer
npm config set registry https://registry.npmmirror.com
npm install -g npx
npm install
cd mineflayer-collectblock
npx tsc
cd ..
npm install

The tutorial is from VOYAGER.

Follow the instructions in Minecraft Login Tutorial to set up your Minecraft Instance.

Follow the instructions in Fabric Mods Install to install the mods.

You can run ADAM by:

Our implementation has methods for automatic saving and automatic loading of checkpoints. If you want to continue from a checkpoint, you can use the following code.

If you want to continue from a specific checkpoint, you can use the following code.