NAC

The question

Can a trained network be treated as a compilable program? NAC inspects parameters, learns local algebraic rewrites from behavior, and tries to emit a smaller executable network or an exact symbolic primitive.

The point is not to rename distillation. Each claim has a baseline, checked-in results, a reproduction command, and a statement of where the result stops applying.

What the experiments found

The first control composes compatible one-hidden-layer tanh MLPs exactly by folding adjacent affine maps. Across 100 sampled pairs and 204,800 inputs, the float64 maximum error was 5.55e-16.

A six-coefficient quadratic operator learned the matrix-product cross term from end-to-end functional error. It trained on interface widths 1 through 4, generalized to width 64, and transferred across deeper ReLU, GELU, and sine chains.

The next experiment projected the learned local rule into the exact rational expression a * b. At interface width 1,024, the structured compiler’s median maximum error was 4.05e-5, compared with 4.60e-1 for an unstructured neural operator. The extracted symbolic compiler matched torch.bmm exactly in checked float64 evaluations through width 4,096.

Exact nonlinear collapse

NAC also defines a restricted nonlinear model family that is closed under exact composition and inversion: strictly increasing scalar piecewise-linear spline networks. A chain of 64 random models with 16 knots each compiled to one model with a median maximum error of 4.13e-12 and a measured median CPU speedup of 39.94x.

The same architecture can be trained with backpropagation, frozen, and compiled from its parameters. In the checked runs, compilation added no measurable loss.

The boundary

This is not a universal compiler for arbitrary neural networks. The exact nonlinear result applies to scalar strictly monotone piecewise-linear maps. The open problem is extending useful closure to multivariate models without the number of regions exploding.