MineDreamer is an instructable embodied agent for simulated control and it is developed on top of recent advances in Multimodal Large Language Models (MLLMs) and diffusion models!

MineDreamer can follow instructions steadily by employing a Chain-of-Imagination (CoI) mechanism to envision the step-by-step process of executing instructions and translating imaginations into more precise visual prompts tailored to the current state; subsequently, it generates keyboard-and-mouse actions to efficiently achieve these imaginations,

For more details or tutorials see https://github.com/Zhoues/MineDreamer.