GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds

Zekun Hao, Arun Mallya, Serge Belongie, Ming-Yu Liu


This paper presents GANcraft a volumetric rendering (rendering of the 3D world as 2D images) based approach to model a 3D block world scene with semantic labels as a continuous volumetric function and render view consistent, photorealistic images. In the absence of the paired training data, an image-to-image translation model generates the pseudo ground truth labels for the corresponding photorealistic 3D world.




The model is evaluated based on the FID, KID scores where GANcraft achieves values close to SPADE which is a photorealistic image generator and outperforms other baselines on temporal consistency metric based on human preference scores.

Our Two Cents


Project Page: