27 Lab 11 — Output Pinning
Anchor chapter: Chapter 11 — Pinned Neurons and Joint State-Parameter Dynamics.
Goal. Pin the output neuron to a target label, run sheaf diffusion forward, and observe how the input layer re-equilibrates.
Clamp both endpoints of a small trained MLP: input to a training point, output to its target. Solve the two-sided Dirichlet problem (Prop. 11.2) and plot the interior cochain as a “string draped between two endpoints.” Compute the residual Dirichlet energy \(R(\theta; x, y)\) and the per-edge discords. Then add a few slow gradient steps in \(\theta\) and watch the string go slack as weights adapt. Optional: experiment with hidden-neuron pinning (counterfactual editing) — fix an interior vertex to a target value and observe how it reshapes both upstream and downstream cochains.
This lab requires only NumPy, Matplotlib, and SciPy, loaded automatically via Pyodide. Code cells run directly in the page via WebAssembly — no local Python installation needed.
Prefer a local Jupyter environment? Download lab-11-output-pinning.ipynb
27.1 Setup
27.2 1. Build the object
With both input and output pinned, the free cochains are \(z^{(1)}, \ldots, z^{(k)}\) (the hidden pre-activations). The two-sided coboundary \(\delta_{\Omega,\text{both}}\) has one extra pinned column coming from the output vertex \(z^{(k+1)} = y_{\text{target}}\), which adds a boundary-forcing term on the last edge. The residual Dirichlet energy \(R(\theta; x, y) = \tfrac{1}{2}\|\delta_{\Omega,\text{both}} c_\Omega^* - \tilde{b}_{\text{both}}\|^2\) is zero when the weight parameters \(\theta\) are perfectly consistent with the training point; minimising \(R\) over \(\theta\) is the training objective.
27.3 2. Verify a theorem / run an experiment
We plot the full cochain \((x_{\text{in}}, z^{(1)*}, y_{\text{target}})\) as a “string” stretched between the two pinned endpoints. Each hidden coordinate of \(z^{(1)*}\) lies at the position that minimises the total discord given both boundary conditions. We then take gradient steps \(\theta \leftarrow \theta - \eta\, \nabla_\theta R\) and watch the residual energy shrink and the string relax toward the forward-pass cochain.
27.4 Exercises
Hidden-neuron pinning. Fix one hidden neuron \(z^{(1)}_j = v\) to a target value \(v\) (counterfactual editing). Solve for the remaining free cochains and plot how fixing one hidden neuron reshapes both the upstream (\(x\)-side) and downstream (\(z^{(2)}\)-side) discords.
Two-sided vs one-sided. Compare the residual energy \(R(\theta; x, y)\) with the one-sided (input-only) version \(\tfrac{1}{2}\|f(x) - y\|^2\). Under what conditions are they equal? Show that \(R \geq \tfrac{1}{2}\|f(x) - y\|^2\) always holds.
Batch pinning. Extend the two-sided Dirichlet solve to a batch of training points \((x_i, y_i)\), \(i=1,\ldots,B\). For each, solve for \(z_i^{(1)*}\) independently and average the resulting gradient \(\nabla_\theta R\). Compare batch gradient descent on \(R\) vs vanilla MSE loss for a small training set.
String tension. The “slack” of the string is \(\|z^{(1)*} - z^{(1)}_{\text{fwd}}\|\), i.e., how far the pinned solution deviates from the forward pass. Plot this as a function of the gap \(|y_{\text{target}} - f(x)|\). At what gap does the string become taut enough to cause a region switch (sign change in \(z^{(1)*}\))?