Neural Ordinary Differential Equations

This paper introduces a new family of deep neural network models. Instead of defining a discrete sequence of hidden layers, they parameterize the derivative of the hidden state using a neural network. The continuous-time dynamics allow for adaptive computation, memory efficiency, and inherent trade-offs between accuracy and speed. This approach has implications for modeling sequential data and allows for continuous normalizing flows, enabling density estimation and generative modeling.