Neural Operators

A neural operator learns a map between function spaces rather than between finite-dimensional vectors. This page states the concept and its two main architectures — DeepONet and the Fourier neural operator — on mathematical grounds, and explains why the latter is the natural parameterization for the repository's jump-diffusion control problems. No measurements appear here; they belong in the empirical results section, and the jump connection is worked out in full in the research note.

From functions to function maps

A plain neural network approximates a map \(f : \mathbb{R}^{d_x} \to \mathbb{R}^{d_y}\) between finite-dimensional spaces. An operator-learning problem replaces both ends by function spaces: given an input function \(a \in \mathcal{A}\) (terminal data, a coefficient field, or a jump measure) and an output function \(u \in \mathcal{U}\) (the value function or the control), the object of interest is the solution operator

\[ G : \mathcal{A} \to \mathcal{U}, \qquad G(a) = u . \]

A neural operator \(G_\theta\) parameterizes this map directly. Two properties distinguish it from solving one instance at a time. First, discretization invariance: the same weights act on the input function regardless of the grid or sensor set on which it is represented, so the approximation is a property of the function, not of one mesh. Second, amortization: a single trained operator evaluates a whole family of instances (every terminal condition, every coefficient, every jump measure) at inference cost, which is the pay-off that justifies the training cost. The general theory is in [@kovachki2023neural].

Universal approximation for operators

The theoretical foundation is a classical result of Chen and Chen: a nonlinear continuous operator \(G\) on a compact set of inputs can be uniformly approximated by a finite expansion of the form

\[ G(a)(y) \;\approx\; \sum_{k=1}^{p} b_k(a)\,\phi_k(y), \]

where the \(b_k\) are continuous functionals of the input function \(a\) and the \(\phi_k\) are functions of the output variable \(y\). The two architectures below are two different ways of realizing this factorization with a neural network: DeepONet learns the \(b_k\) and \(\phi_k\) explicitly as two subnetworks, while the Fourier neural operator realizes the operator as a composition of integral kernels rather than a truncated expansion.

Special case 1: DeepONet (branch/trunk)

DeepONet [@lu2021deeponet] realizes the Chen–Chen factorization with two networks and an inner product. A branch net encodes the input function from its values at a fixed set of sensor points \(x_1,\dots,x_m\) into the coefficients \(b_1,\dots,b_p\); a trunk net encodes the query point \(y\) into the basis functions \(\phi_1,\dots,\phi_p\). The operator is then

\[ G_\theta(a)(y) = \big\langle \mathrm{branch}(a(x_1),\dots,a(x_m)),\; \mathrm{trunk}(y) \big\rangle . \]

The output is evaluated at arbitrary query points \(y\): once the branch latent vector is computed, the trunk can be evaluated at any \(y\) without a grid, which is what makes DeepONet mesh-free on the output side. Its input, however, is a fixed set of sensor values, so the input function is represented pointwise rather than spectrally.

Special case 2: Fourier neural operator (spectral)

The Fourier neural operator [@li2021fourier] takes the alternative route of building the operator as a composition of integral-kernel layers separated by pointwise nonlinearities. Each layer maps

\[ v \;\mapsto\; \sigma\!\big(W v + \mathcal{K} v\big), \qquad (\mathcal{K}v)(x) = \int \kappa(x, y)\, v(y)\, dy, \]

where \(W\) is a pointwise (local) linear map and \(\mathcal{K}\) is a nonlocal integral operator with kernel \(\kappa\). The decisive step is to take a translation-invariant kernel \(\kappa(x,y) = \kappa(x-y)\), so that \(\mathcal{K}\) is a convolution. A convolution is diagonalized by the Fourier transform,

\[ \widehat{\mathcal{K} v}(\xi) = \hat\kappa(\xi)\,\hat v(\xi), \]

so the layer reduces to a spectral multiplier with a learned symbol \(R_\theta\):

\[ v \;\mapsto\; \mathcal{F}^{-1}\!\big(R_\theta \cdot \mathcal{F}(v)\big). \]

This is efficient (the kernel is parameterized by a few Fourier modes) and it bakes in the translation-invariance that convolution expresses. The cost is that both input and output must live on a grid on which the transform is defined, so FNO is grid-bound where DeepONet is mesh-free.

Why the jump generator is a Fourier multiplier

The repository's jump-diffusion control problems are exactly the setting in which the FNO's structure is not an approximation but an identity. For a jump measure \(\nu\), the jump generator is

\[ \mathcal{J}v(x) = \int_{\mathcal{X}} \big(v(x+z) - v(x)\big)\,\nu(dz) = (\nu \ast v)(x) - v(x), \]

a convolution with \(\nu\) minus the identity, whose Fourier symbol is \(\widehat{\mathcal{J}v} = (\hat\nu - 1)\,\hat v\) (see the research note for the derivation and its lattice \(\mathbb{Z}^m\) and group \(\mathbb{Z}_2^m\) counterparts). A single FNO layer with \(R_\theta = \hat\nu - 1\) therefore represents the jump generator exactly. The nonlocal term that makes classical grid methods expensive is the term a spectral operator represents natively; the diffusion term (a local differential operator) and the control supremum (a pointwise nonlinearity) are the parts FNO handles less naturally, and they are the reason a jump-aware operator is a composition rather than one layer.

Choosing between the two

They are complementary: the spectral layer captures the convolution/jump structure, and the branch/trunk head handles mesh-free evaluation and non-function inputs. The stage that turns this into code is 07-neural-operators.