## What the ReLU Revolution Revealed About Biological Plausibility The activation function was never a fixed biological commitment but a working hypothesis revised under empirical pressure. ## Introduction Few decisions in the design of a neural network have attracted as much attention, and as much revision, as the choice of activation function. On the surface it's a narrow technical question, but the activation function is where the comparison with biology is most literal, since it decides what it means for an artificial neuron to fire. That makes it the natural place for an old question to…