SEM-ROVER: Semantic Voxel-Guided Diffusion for Large-Scale Driving Scene Generation

ECCV 2026
Hiba Dahmani1, 2, Nathan Piasco1, Moussab Bennehar1, Luis Roldao1, Dzmitry Tsishkou1, Laurent Caraffa3, Jean-Philippe Tarel2, Roland Brémond2,
1Huawei Paris Research Center
2 COSYS, Gustave Eiffel University
3 LASTIG, IGN-ENSG, Gustave Eiffel University
arXiv Video Code(TBA)

Abstract

We present a framework for generating large-scale outdoor driving scenes in 3D. Unlike prior methods that struggle with geometric consistency or scale, our approach uses a discrete 3D representation called Σ-Voxfield, where each occupied voxel stores a fixed set of colored surface samples. A semantic-conditioned diffusion model generates this representation using local voxel neighborhoods and 3D positional encodings, while progressive spatial outpainting enables expansion to large scenes. A deferred renderer then produces photorealistic images from the generated 3D scene. The method generates diverse, multiview-consistent urban environments without per-scene optimization and supports different sensors, camera trajectories, and large spatial scales at moderate computational cost.

Method Overview

method image

Our framework generates large-scale outdoor scenes through the following steps:

  1. 3D Scene Representation: The scene is first represented as a Σ-Voxfield grid, a discrete 3D structure that stores colorized surface information within occupied voxels.
  2. 3D Scene Generation: A semantic-conditioned diffusion model operates directly on the Σ-Voxfield grid, generating scene content in 3D space using transformer-based processing and 3D positional encodings to ensure geometric consistency.
  3. Large-Scale Scene Expansion: An iterative outpainting strategy progressively extends the generated scene by synthesizing overlapping neighboring regions, enabling scalable generation of large environments while maintaining coherence.
  4. Photorealistic Rendering: The completed Σ-Voxfield grid is processed by a deferred rendering engine, which converts the generated 3D scene into photorealistic images from different viewpoints and sensor configurations.

Results

Our method enables the generation of large-scale, multiview-consistent outdoor scenes while maintaining realistic geometry, appearance, and computational efficiency.

PandaSet
PandaSet
PandaSet
Waymo
Waymo

Scene Inpainting

Local editing is enabled by voxel-space inpainting. SEM-ROVER enables different local generations that remain coherent with the surrounding context across viewpoints.

Infinite scenes generation

Our method enables large scene generation spanning over 1002 m.
method image

BibTeX

@misc{dahmani2026semroversemanticvoxelguideddiffusion,
      title={SEM-ROVER: Semantic Voxel-Guided Diffusion for Large-Scale Driving Scene Generation},
      author={Hiba Dahmani and Nathan Piasco and Moussab Bennehar and Luis Roldão and Dzmitry Tsishkou and Laurent Caraffa and Jean-Philippe Tarel and Roland Brémond},
      year={2026},
      eprint={2604.06113},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2604.06113},
}