Motion modeling is critical in flow-based Video Frame Interpolation (VFI).
Existing paradigms either consider linear combinations of bidirectional flows or directly predict bilateral flows for given timestamps without exploring favorable motion priors,
thus lacking the capability of effectively modeling spatiotemporal dynamics in real-world videos.
To address this limitation, in this study, we introduce Generalizable Implicit Motion Modeling (GIMM),
a novel and effective approach to motion modeling for VFI.
Specifically, to enable GIMM as an effective motion modeling paradigm,
we design a motion encoding pipeline to model spatiotemporal motion latent from bidirectional flows extracted from pre-trained flow estimators,
effectively representing input-specific motion priors.
Then, we implicitly predict arbitrary-timestep optical flows within two adjacent input frames via an adaptive coordinate-based neural network,
with spatiotemporal coordinates and motion latent as inputs.
Our GIMM can be smoothly integrated with existing flow-based VFI works without further modifications. We show that GIMM performs better than the current state of the art on the VFI benchmarks.

Integrating **GIMM** with **V**ideo **F**rame **I**nterpolation (**GIMM-VFI**). GIMM-VFI utilizes a pre-trained flow estimator \(\mathcal{E}\),
to predict bidirectional flows \((F_{0\rightarrow1}, F_{1\rightarrow0})\) and extracts context features \(A\) as well as correlation features \(C\) from the input frames \((I_0, I_1)\).
Given the timestep \(t\), a generalizable implicit motion modeling (GIMM) module \(\mathcal{G}\)
(detailed in Figure \above) takes the bidirectional flows as inputs and predicts bilateral flows \((F_{t\rightarrow0}, F_{t\rightarrow1})\),
which are then passed into a fame synthesis module \(\mathcal{S}\), together with extracted features \((A, C)\), to synthesize the target frame \(I_t\).

We present input videos (up) and 8X interpolated results by GIMM-VFI (down).

Overlaid inputs

GIMM modeled motion

GIMM-VFI interpolated results

```
@article{guo2024gimmvfi,
author = {Guo, Zujin and Li, Wei and Loy, Chen Change},
title = {Generalizable Implicit Motion Modeling for Video Frame Interpolation},
journal = {arXiv preprint arXiv:},
year = {2024},
}
```