INVE: Interactive Neural Video Editing
We present Interactive Neural Video Editing (INVE), a real-time video editing
solution, which can assist the video editing process by consistently
propagating sparse frame edits to the entire video clip. Our method is inspired
by the recent work on Layered Neural Atlas (LNA). LNA, however, suffers from
two major drawbacks: (1) the method is too slow for interactive editing, and
(2) it offers insufficient support for some editing use cases, including direct
frame editing and rigid texture tracking. To address these challenges we
leverage and adopt highly efficient network architectures, powered by
hash-grids encoding, to substantially improve processing speed. In addition, we
learn bi-directional functions between image-atlas and introduce vectorized
editing, which collectively enables a much greater variety of edits in both the
atlas and the frames directly. Compared to LNA, our INVE reduces the learning
and inference time by a factor of 5, and supports various video editing
operations that LNA cannot. We showcase the superiority of INVE over LNA in
interactive video editing through a comprehensive quantitative and qualitative
analysis, highlighting its numerous advantages and improved performance. For
video results, please see https://gabriel-huang.github.io/inve/
64b6164921c601b486d3a674
Jiahui Huang
claimed_verified
2023-07-19T12:23:14.247Z
63505463f32062e9eb7e4924
https://cdn-avatars.huggingface.co/v1/production/uploads/63505463f32062e9eb7e4924/6RxK_txF65jh8lhkbjesM.jpeg
Gabriel Huang
user
GabrielH
64b6164921c601b486d3a675
Leonid Sigal
64b6164921c601b486d3a676
Kwang Moo Yi
admin_assigned
2023-07-18T08:13:13.537Z
64a591af9b8ae9ccccfdd3e7
/avatars/c6cd43d814956261639fe594bc4aa181.svg
Kwang Moo Yi
user
kmyi
64b6164921c601b486d3a677
Oliver Wang
admin_assigned
2023-07-18T08:13:37.661Z
62fa8bfdc95d426ff8f25fe9
/avatars/71fe9cff4ce79f6a208045984852e5ad.svg
Oliver Wang
user
oliverwang
64b6164921c601b486d3a678
Joon-Young Lee
claimed_verified
2023-07-19T06:28:22.630Z
64b76c7990b38df83381824b
/avatars/0a5bec2ea480fb3f43c4b24d59c50e81.svg
Joon-Young Lee
user
joonyounglee
INVE: Interactive Neural Video Editing
We present Interactive Neural Video Editing (INVE), a real-time video editing
solution, which can assist the video editing process by consistently
propagating sparse frame edits to the entire video clip. Our method is inspired
by the recent work on Layered Neural Atlas (LNA). LNA, however, suffers from
two major drawbacks: (1) the method is too slow for interactive editing, and
(2) it offers insufficient support for some editing use cases, including direct
frame editing and rigid texture tracking. To address these challenges we
leverage and adopt highly efficient network architectures, powered by
hash-grids encoding, to substantially improve processing speed. In addition, we
learn bi-directional functions between image-atlas and introduce vectorized
editing, which collectively enables a much greater variety of edits in both the
atlas and the frames directly. Compared to LNA, our INVE reduces the learning
and inference time by a factor of 5, and supports various video editing
operations that LNA cannot. We showcase the superiority of INVE over LNA in
interactive video editing through a comprehensive quantitative and qualitative
analysis, highlighting its numerous advantages and improved performance. For
video results, please see https://gabriel-huang.github.io/inve/
Interactive Neural Video Editing (INVE) improves real-time video editing by leveraging hash-grids encoding and bi-directional functions, enabling faster processing and a wider range of edit operations compared to Layered Neural Atlas (LNA).
Interactive Neural Video Editing INVE Layered Neural Atlas LNA hash-grids encoding bi-directional functions vectorized editing real-time video editing sparse frame edits