The decision boundary is the weight vector

People say a neural network learns and leave it there. A single neuron makes it concrete enough to watch. One neuron is logistic regression, and once you see the geometry, "learning" stops being a metaphor and becomes a line rotating across a plane until it splits two clouds of points.

One neuron, drawn

A neuron takes inputs x, multiplies each by a weight, adds a bias, and squashes the result with a sigmoid into a probability:

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

def neuron(x, w, b):
    return sigmoid(x @ w + b)

The decision boundary is the set of points where the output is exactly one half, which is where w · x + b = 0. In two dimensions that is a line; in higher dimensions, a flat hyperplane. So the whole model is: which side of a line are you on, and how far.

The weights are the boundary

This is the part worth slowing down on. The weight vector w is not an abstract list of parameters. Geometrically:

• w points perpendicular to the boundary, aimed at the positive class. The boundary is the line orthogonal to w; the bias b slides that line along the w direction.
• The quantity w · x is the projection of the input onto the w direction. Classifying a point is asking "how far along w are you," and the bias sets where the cutoff sits.
• The length of w is confidence. A long w makes the sigmoid switch from near-zero to near-one across a razor-thin band at the boundary, a sharp, confident decision. A short w makes a gentle ramp, an uncertain one. Same boundary line, different conviction.

So if you train a neuron and read its weights back out, you are reading the boundary directly: the direction is the tilt, the magnitude is the certainty. The neuron did not memorize the points. It found a rule, encoded as a vector.

The gradient is shockingly clean

To learn, the neuron needs to know how wrong it is. The loss is cross-entropy, which for a prediction a and true label y is -(y log a + (1-y) log(1-a)). It looks like it should produce a messy derivative. It does not. Work through the chain rule for the pre-sigmoid score z:

dL/da = (a - y) / (a (1 - a))         # derivative of cross-entropy
da/dz = a (1 - a)                      # derivative of sigmoid
dL/dz = dL/da * da/dz = a - y          # the a(1-a) cancels exactly

The a(1-a) terms cancel, and you are left with a - y, the raw error, the prediction minus the truth. That cancellation is not a coincidence; cross-entropy is the loss specifically designed to pair with the sigmoid so the gradient comes out this clean. Push it back to the weights with one more chain-rule step and the gradient is just the error times the input:

def step(X, y, w, b, lr):
    a = sigmoid(X @ w + b)
    error = a - y                      # the whole gradient signal
    w -= lr * (X.T @ error) / len(y)
    b -= lr * error.mean()
    return w, b

Read that geometrically and it is a tug of war. Every misclassified point pulls the boundary toward fixing itself, weighted by how wrong it is; correctly classified points pull weakly or not at all. Sum all the pulls and you get the weight update. Run it a few hundred times and the line rotates and slides until it finds the gap between the clouds. That is the entire learning algorithm: forward, measure, get a - y, take a small step, repeat.

The wall: one neuron is one line

The honest limit is built into the geometry. One neuron can only ever produce one straight boundary, so it can only separate classes that a straight line can separate. Hand it the XOR pattern, points where the classes sit in opposite corners of a checkerboard, and there is no line that works. Train it forever and it parks at a coin-flip, fifty percent, because the best straight line through XOR is no better than guessing.

The fix is the reason deep learning exists. Stack neurons into a layer and feed their outputs into another neuron, and the network can bend the boundary, because each hidden neuron carves one half-plane and the next layer combines them into a curved region. One neuron is a line; layers are curves. That single limitation, drawn on the XOR pattern, is the entire motivation for going deep, and you can feel it the moment you watch a lone neuron fail to separate four points.