Surface Networks

Why meshes are not point clouds: A cylinder bent along $x$ vs $y$ has identical intrinsic metric (both developable) but different extrinsic mean curvature vectors. Laplacian $\Delta V = -2H\mathbf{n}$ measures only mean curvature magnitude $|H|$, not direction. That means a Laplacian network trained to predict next frame of cloth cannot distinguish a wrinkle forming along vs across. Dirac resolves it.

2.1 Cotangent Laplacian: $L_{ij}= \cot\alpha_{ij}+\cot\beta_{ij}$, $M_V$ Voronoi mass, $\Delta = M_V^{-1}L$. Applied to embedding $V$, $\Delta V = -2 H \mathbf{n}$ – only mean curvature vector. Eigenvalues Weyl law $\lambda_k\sim 4\pi k/Area$. Isotropic diffusion: $x^{k+1}= \rho(A\Delta x+Bx)$ blurs equally in all directions – good for noise removal, bad for wrinkles.

2.2 The cylinder bending failure: Take thin sheet $[-1,1]^2$ bent by 30° along $x$ (crease parallel to $y$) vs same bent along $y$. Both have same $|H|$ distribution (average curvature magnitude equal). Laplacian SN with symmetric aggregation $A\Delta$ produces identical latent – cannot predict if deformation will continue folding same direction (temporal task). Test in paper: Laplacian SN L2 0.029 vs Dirac 0.024 – that gap is entirely directionality.

2.3 Dirac deep dive – quaternion intuition: For each face $f$ with vertices $(i,j,k)$, define quaternion-valued gradient $D_{f,j} = -\frac{1}{2| A_f |} \mathbf{e}_j$ where $\mathbf{e}_j$ is opposite edge embedded in $\mathbb{H}$ as pure quaternion $(0, \mathbf{e}_x,\mathbf{e}_y,\mathbf{e}_z)$. Then $D: \mathbb{R}^{|V|\times d}\to \mathbb{H}^{|F|\times d}$ computes face gradient. Its adjoint $D^*=M_V^{-1} D^H M_F$ brings back to vertices. Crucial identities:

• $\Re(D^* D)=\Delta$ – real part recovers Laplacian, but imaginary parts encode curl = direction of maximal curvature.
• Spectrum of $D$ comes in pairs $\pm \sqrt{\lambda}$ and detects $k_1,k_2$ separately because $D$ acting on position field returns $k_1 \mathbf{d}_1 + k_2 \mathbf{d}_2$ in quaternion basis.
• Chunk dim multiple of 4 required – we treat 4 channels as one quaternion, enabling rotation-equivariant transport: $q\cdot p$ pseudo quaternion multiplication propagates orientation consistently across edges (no flip ambiguity unlike vector fields).

Mind picture: Laplacian is like asking “how much does height vary on average?” Dirac is like asking “in which compass direction does it curve fastest?” For cloth, that compass is wrinkle direction.

Node → Face → Node: Input vertex signal $x_V\in\mathbb{R}^{|V|\times d}$ (positions or SHOT). Compute face gradient $y_F = D x_V \in \mathbb{H}^{|F|\times d}$ (each face 1 quaternion per 4 channels). Apply learnable quaternion linear $W_F \in \mathbb{H}^{d'\times d}$ with mass-renorm $ \tilde D = M_F^{1/2} D M_V^{-1/2}$ to make symmetric, stable spectrum $\sigma(\tilde D)\subset[-1,1]$. Non-linearity $\rho=$ ELU on norm + direction preserving.

Then return $z_V = D^* y_F$ → vertex space. Residual: $x^{k+1}=x^k+z_V$. Stack 6 such Dirac layers shared trunk 64→128→256 dims. Temporal head: 2-layer MLP → predicts $\delta V_{t+1}=V_{t+1}-V_t$, L2 loss $| \hat V - V_{gt}|^2$ averaged. VAE variant: encoder both Dirac layers → $\mu,\log\sigma\in\mathbb{R}^{10}$ latent, decoder Dirac transposed, ELBO loss.

Why chunk 4? Quaternion multiplication needs 4-dim group: we reinterpret channel dim $C=4K$ as $K$ quaternions. Multiplication $q\cdot p$ = Hamilton product allows network to learn rotation-equivariant filters (bend left vs right preserved). Implementation in CUDA via pynvrtc JIT – 5× over PyTorch naive (which unrolls matmul). Block-diagonal batching of 16 meshes (varying |V|) via sparse COO.

Curriculum: First 5 epochs flat sheets only (zero bending) to stabilize Dirac mass matrices $M_F,M_V$ condition number. Then introduce bending 10°→30° linearly.

Theorem 4.1 Stability to deformation: Let $\tau$ be diffeomorphism with bi-Lipschitz constant $\|\nabla\tau\|_\infty\le\epsilon$, dihedral angle change $\le \delta_\theta$. Then for SN depth $L$, Lipschitz product $L_W=\prod\|W^{(l)}\|$,

$$ \| \Phi(\mathcal M_\tau)-\Phi(\mathcal M)\| \le C L_W (\epsilon+\delta_\theta) \|x\|_{H^1}$$

Proof sketch: $D_\tau = D + O(\epsilon)$ since edge vectors rotate $O(\epsilon)$; $M_F$ changes $O(\epsilon)$; composition inherits bound via Sobolev embedding $H^1\to L^2$. Implication: small stretch doesn't explode – crucial for loose garment sim later.

Theorem 4.2 Consistency to discretization: As triangulation refines $h\to0$, $\beta\to1$ (mesh regularity $\beta=$ min angle / max angle), eigenvalues $\lambda_k(D_h)\to\lambda_k(D_{cont})$, with rate

$$ h(\beta)=\prod_{\text{tri }t}\frac{\beta_t-1}{\beta_t-1/2}\to0 $$

Uses Weyl law $\lambda_k\sim4\pi k/Area$ + discrete Dirac convergence of Leske + Crane. Practically: remeshing same shape (Loop subdivision 2×) changes output <2% L2 – verified Table 2 supplement.

Corollary 4.3 Coordinate reconstruction: First 100 eigenfunctions of Dirac span coordinates up to rigid motion – so Dirac trunk is universal approximator for extrinsic shape.

• pynvrtc CUDA JIT: Quaternion batched matmul $Q\in\mathbb{H}^{B\times K\times4}$ compiled at runtime, 18ms forward for |V|=5k, |F|=10k, C=128. Without JIT 92ms. Code in my fork `ops/quaternion_kernel.cu`.
• libigl Python bindings: For $L,M_V,D$ computation: `pip install git+https://github.com/jiangzhongshi/libigl@cluster-pyigl#egg=pyigl` – my branch adds `dirac_operator` returning scipy CSR double + face areas. Also `cotmatrix` from igl.
• Curvature-aware batching: Batch meshes sorted by curvature variance $\operatorname{Var}(k_1-k_2)$ to avoid mixing anisotropic / isotropic in same batch (stabilizes batchnorm).
• Mass renormalization: $ \tilde D = M_F^{1/2} D M_V^{-1/2}$ symmetrizes spectrum; also clamp face mass $|A_f|>1e-7$ to avoid degenerate tiny tris division.
• Repro: `python train_temporal.py --mesh 62415 --dirac --layers 6 --chunk 4 --lr 1e-3 --curriculum` matches paper L2 0.024 after 80 epochs, Quadro M4000 6h.

Temporal elastic shell (main): 500 sequences ×50 frames each, Saint Venant–Kirchhoff non-linear, thin plate $\nu=0.3$, Young's modulus random. Train predict next frame from past 4. Test L2: Dirac SN 0.024 vs Laplacian SN 0.029 vs MoNet 0.032 vs PointNet++ 0.038 vs GCNN 0.035. Wrinkles visually preserved.

Mesh MNIST – digits embossed on thin sheet bending (+ randomly). VAE NLL: Dirac best 44.7 vs Laplacian 48.2 vs PointNet++ 51.1 (n=10k). Latent disentangles digit identity vs bend angle linearly separable (t-SNE). Image:

Fig 2 Mesh MNIST VAE samples – top row digit variation, bottom row bending angle driven by Dirac second channel.

FAUST segmentation 100 human scans, 10 parts (head, torso...). Accuracy Dirac SN 91.2% vs MoNet 88.5% vs GCNN 86.3% vs ACNN 85.1% – extrinsic helps where intrinsic symmetry (left-right leg isometric) ambiguous.

• Geodesic CNN Masci O(NK²) patch resampling → pooling unstable under remesh (remesh same shape accuracy drops 6% vs ours 1.2%).
• ACNN Boscaini anisotropic but umbilic singular – at planar points where $k_1=k_2$ (umbilic), angular bin undefined → needs handcrafted fix, we are singularity-free (quaternion continuous).
• Torus flat embedding TorAlly genus-constrained (torus method requires genus-1 correction, ours genus-agnostic manifold with boundary ok).
• PointNet diffusion max – sample complexity exponential in curvature dimension; ours 10× fewer samples for same L2 because Dirac bakes in connectivity.

This CVPR 2018 Oral was my undergraduate NYU work with Ilya Kostrikov (first-author). It seeded three later threads in my research:

• Progressive Embedding (SIGGRAPH 2019) – robust untangling for bijective maps. Dirac stability proof inspired our progressive collapse-insertion guarantee – both require deformation bound via Dirac-like operator to avoid flips. Implementation reused pynvrtc pattern.
• Quadfoam / A Low-Parametric Rhombic Family (SIGGRAPH 2020) – rhombic microstructure parameterized by principal stretches $k_1,k_2$ detected via Dirac curvature idea. We applied same curvature-aware batching to homogenization dataset split.
• Meta Reality Labs digital humans (FRESA, LCA, HyperBones, PhySkin, MHR, FaceMap): Bone-driven neural garment simulation with hypernetwork conditioning lives on top of extrinsic surface diffusion. Our later HyperBones & PhySkin replace Dirac hand-crafted with learned hypernetwork but keep theorem 4.1 stability bound as regularization $\|\nabla\tau\|$ for loose garments – directly citing this paper's proof in supplemental. Also, Mega-scale Codec Avatars Gaussian deformer uses Dirac features as conditioning for sparse anchors.

For students: if you start with Laplacian GNNs today, try adding Dirac – change 20 lines in PyTorch (replace $M_V^{-1}L$ with $D^* D_{\mathbb H}$) and get free anisotropy. My fork keeps that example minimal `train_dirac_vs_lap.py`.