DualSplat: Robust 3D Gaussian Splatting via Pseudo-Mask Bootstrapping from Reconstruction Failures

Xu Wang1, Zhiru Wang1, Shiyun Xie1, Chengwei Pan1†, Yisong Chen2
1Beihang University, 2Peking University
†Corresponding author

DualSplat breaks the circular dependency between transient detection and static scene reconstruction through a two-stage pipeline: pseudo-mask bootstrapping from first-pass reconstruction failures, followed by pseudo-mask-guided clean 3DGS optimization.

First page

Abstract

While 3D Gaussian Splatting (3DGS) achieves real-time photorealistic rendering, its performance degrades significantly when training images contain transient objects that violate multi-view consistency. Existing methods face a circular dependency: accurate transient detection requires a well-reconstructed static scene, while clean reconstruction itself depends on reliable transient masks. We address this challenge with DualSplat, a Failure-to-Prior framework that converts first-pass reconstruction failures into explicit priors for a second reconstruction stage. We observe that transients, which appear in only a subset of views, often manifest as incomplete fragments during conservative initial training. We exploit these failures to construct object-level pseudo-masks by combining photometric residuals, feature mismatches, and SAM2 instance boundaries. These pseudo-masks then guide a clean second-pass 3DGS optimization, while a lightweight MLP refines them online by gradually shifting from prior supervision to self-consistency. Experiments on RobustNeRF and NeRF On-the-go show that DualSplat outperforms existing baselines, demonstrating particularly clear advantages in transient-heavy scenes and transient regions.

Pipeline

Pipeline

DualSplat uses a two-stage robust reconstruction strategy: first-pass conservative reconstruction with pseudo-mask extraction, then pseudo-mask-guided second-pass optimization with online mask refinement.

Qualitative Visuals

Pseudo Masks visualization: We visualize the pseudo-masks generated by our method.

Gt Pseudo Mask Render
Gt Pseudo Mask Render
Gt Pseudo Mask Render

Baseline comparisons: We visualize renderings compared to baseline methods.

Gt 3DGS DeSplat DualSplat
Gt 3DGS DeSplat DualSplat
Gt 3DGS DeSplat DualSplat
Gt 3DGS DeSplat DualSplat
Gt RoubustSplat DeSplat DualSplat
Gt RoubustSplat DeSplat DualSplat

Videos

Interpolation from test viewpoints:

Training viewpoints and pseudo masks:

BibTeX

Generating...