Motivation
(a) Gaussian flow under different supervision. We model Gaussian flow under the supervision of optical flow and motion flow respectively. The latter can produce a more direct description of object motion, thereby effectively guiding the deformation of 3D Gaussians. (b) The decoupling of optical flow. We decouple the optical flow into motion flow which is only related to object motion and camera flow which is only related to camera motion.
Method Overview
The overall architecture of MotionGS. It can be viewed as two data streams: (1) The 2D data stream utilizes the optical flow decoupling module to obtain the motion flow as the 2D motion prior; (2) The 3D data stream involves the deformation and transformation of Gaussians to render the image for the next frame. During training, we alternately optimize 3DGS and camera poses through the camera pose refinement module.
Flow Visualization
Visualization of all data flows. Each example corresponds to two rows.
Reconstruction Results
Qualitative comparison on NeRF-DS dataset per-scene. Compared with the state-of-the-art methods, our method can render more reasonable details, especially on dynamic objects.
Qualitative comparison on HyperNeRF dataset per-scene. Compared with the state-of-the-art methods, our method is more robust in reconstructing casual dynamic scenes.