Semantic Bayesian image reconstruction with DINOv3 and StyleGAN3

Abstrak

Image reconstruction tasks, such as super-resolution and in-painting, are fundamental challenges in computer vision. Traditional supervised methods often require retraining for new tasks or data distributions, limiting their flexibility and efficiency. The Bayesian Reconstruction through Generative Models (BRGM) framework presents a powerful alternative by leveraging a pre-trained generative model, such as StyleGAN2, as a strong image prior, enabling the solution of various inverse problems without task-specific retraining. This paper aims to enhance and critically evaluate the BRGM framework by systematically exploring the impact of two key components: the perceptual feature extractor and the generative model architecture. We will conduct a comprehensive comparative study across four distinct configurations: (1) BRGM + VGG16 + StyleGAN2, replicating and extending the original setup; (2) BRGM + DINOv3 + StyleGAN2, investigating the effect of a modern, self-supervised vision transformer (ViT) as a perceptual loss network; (3) BRGM + VGG16 + StyleGAN3, evaluating improvements from a more advanced and stable generator; and (4) BRGM + DINOv3 + StyleGAN3, combining the most advanced components to form a state-of-the-art baseline. The performance of each configuration will be rigorously assessed on diverse datasets for the tasks of super-resolution and in-painting, using quantitative metrics (e.g., LPIPS, PSNR, SSIM, NRMSE) and qualitative analysis. Experimental results show that the combination DINOv3 + StyleGAN3 sets a new state‑of‑the‑art for unsupervised image reconstruction, achieving average LPIPS reductions of 0.1–0.2 and PSNR gains of 2–4 dB over the original BRGM baseline.

Identifier

• OpenAlex: https://openalex.org/W7163826006
• DOI: https://doi.org/10.1016/j.eij.2026.100998