Learning from (sparse) observations through the lens of models

Despite the revolution in autonomous in-situ and satellite remote sensing, much of the ocean remains poorly sampled observationally in terms of its spatio-temporal evolution. Inverse methods seek to combine the incomplete knowledge reservoirs from sparse observations and numerical models that require the provision of uncertain inputs to produce optimal state and parameter estimates of the system considered. In its various renderings, methods of optimal estimation and control are being used to learn uncertain parameters (calibration problem) in models or to produce optimal initial states for prediction (extrapolation problem). State estimates serve as optimal reconstructions to improve system understanding (interpolation problem) or to invert for uncertain boundary conditions (flux inversions). Rigorously applied, these methods also enable characterization of uncertainty that arise from different sources (model, observations, forcings). Scientific machine learning methods rely on similar mathematical concepts that underpin optimal estimation and may be successful when the system is heavily sampled. I explore some of the concepts in the context of ocean modeling, based on the unifying framework of differentiable programming. I provide a subjective perspective on opportunities and challenges with a view toward hybrid data assimilation and machine learning to enable online or full-model learning. These ideas are being implemented in the context of general-purpose automatic differentiation applied to an ocean model written in the Julia programming language.