Interactive Scene Flow Estimation

About

We present "Interactive Scene Flow Estimation" -- or longer "Interactive Scene Flow Editing for Improved Image-based Rendering and Virtual Spacetime Navigation", where we take a realtime GPU per-pixel depth and motion estimator and allow users to interactively fix errors of automated algorithms.

Abstract

High-quality stereo and optical flow maps are essential for a multitude of tasks in visual media production, e.g. virtual camera navigation, disparity adaptation or scene editing. Rather than estimating stereo and optical flow separately, scene flow is a valid alternative since it combines both spatial and temporal information and recently surpassed the former two in terms of accuracy. However, since automated scene flow estimation is non-accurate in a number of situations, resulting rendering artifacts have to be corrected manually in each output frame, an elaborate and time-consuming task. We propose a novel workflow to edit the scene flow itself, catching the problem at its source and yielding a more flexible instrument for further processing. By integrating user edits in early stages of the optimization, we allow the use of approximate scribbles instead of accurate editing, thereby reducing interaction times. Our results show that editing the scene flow improves the quality of visual results considerably while requiring vastly less editing effort.

Download Paper

Citation

Kai Ruhl, Martin Eisemann, Anna Hilsmann, Peter Eisert, Marcus Magnor (2015). Interactive Scene Flow Editing for Improved Image-based Rendering and Virtual Spacetime Navigation. Proceedings of ACM Multimedia 2015.

@inproceedings{ruhl2015acmmm,
    author = {Ruhl, Kai and Eisemann, Martin and Hilsmann, Anna and Eisert, Peter and Magnor, Marcus},
    title = {Interactive Scene Flow Editing for Improved Image-based Rendering and Virtual Spacetime Navigation},
    booktitle = {Proceedings of {ACM} Multimedia 2015},
    publisher = {{ACM}},
    month = oct,
    year = {2015}
}