Before NN101 | Neural Networks Dissected

The first time I trained a neural network, on MNIST, it felt like following a recipe: add a linear layer, add an activation, add a softmax, run. Backpropagation made sense. Everything after that looked like conventions stacked on top of conventions.

I'd like us to be careful about every mathematical assumption behind neural network design decisions.

But I'm long overdue for my language model post, so I'll just sketch the outline and leave some investigations to readers. Some are still open questions.

A Distributional View

Every sample we see can be regarded as drawn from an extremely high-dimensional generative process. Real data don't fill that space. They lie on a lower-dimensional manifold with its own intrinsic dimensionality (the manifold hypothesis). And even with gigantic datasets, we are still sparse on those manifolds.

From this view, classification means learning a function, a decision boundary, from sparse points. Most of the questions below come back to this frame.

NN Recipe and UAT

A neural network (NN) is capable of approximating functions. The Universal Approximation Theorem (UAT) says that a network with a single hidden layer, wide enough and with a non-polynomial activation, can approximate any continuous function on a compact domain to arbitrary precision.

Two things to notice. First, one hidden layer is enough. Second, the theorem guarantees that good weights exist, not that we can find them.

Let's examine an NN recipe:

1. A linear layer, scaled up 2x/4x
2. An activation
3. Another linear layer, scaled down to output dimension
4. Softmax
5. NLL loss
6. Backpropagation and a gradient step

Let's take a closer look.

Multiple layers and Scaling

One single perceptron would fail on XOR problems: it draws one hyperplane, and XOR is not linearly separable. So why are we satisfied with adding more layers to address this? Especially when we can simply separate it with a nonlinear boundary $x_1 \times x_2$?

Why can adding layers be regarded as "solving" the entire problem, so that we don't have to worry about expressibility later?

There are investigations, such as EML nodes, which can generate all standard elementary functions. Why don't we want to build something on top of them?

UAT says one hidden layer is enough. So why do we go deep?

Clue:

1. linear separability, manifold hypothesis
2. hidden layers remap the inputs into a space where the problem becomes linearly separable
3. count the linear regions a ReLU network can carve out, as a function of width versus depth.

Activation

Why do we need activation?

And intuitively, GeLU should be more expressible than ReLU. Then why is it only widely used in transformers?

Categorization

Softmax just projects any given vector into a distribution. There are infinitely many ways for such a projection. It isn't even a projection in the geometric sense: the Euclidean projection onto the probability simplex is sparsemax. Then why softmax?

Why do we take one-hot encoding for granted? Sure, it's better than squeezing classes into one dimension, since categorical data don't necessarily have a natural order. But how do we address correlation between dimensions, when some classes are more similar than others?

Optimization

Traditional wisdom is that learning is compression, yet neural networks these days rely heavily on overparametrization. Why?

Networks can fit random labels. So why do we use them, and how do we steer them toward the structure we want?

A contrast case: parity functions are notoriously hard for neural networks to learn. Flip any single input bit and the label flips. Why is that hard?

We have observations of double descent. Will there be multiple descents? And how is this different from drawing a lottery and hoping some overparameterized network performs well on the test set?

Clues: multiple descent, the lottery ticket hypothesis

Epilogue

If you have some intuitions about these questions, we should be equipped with enough familiarity to proceed into various fields utilizing neural networks.