What Is an Affine Layer?
An affine layer, or fully connected layer, is a layer of an artificial neural network in which all contained nodes connect to all nodes of the subsequent layer. Affine layers are commonly used in both convolutional neural networks and recurrent neural networks. A restricted Boltzmann machine is one example of an affine, or fully connected, layer.
For every connection to an affine (fully connected) layer, the input to a node is a linear combination of the outputs of the previous layer with an added bias. The output of a node is then calculated by passing this input through an activation function. Mathematically, this is expressed as:
y = f(Wx + b),
where y represents the output, x represents the input, W represents the weights used in for the linear combination, b
represents the scaling (bias) vector, andf represents the activation function. One should bear in mind that both the weights and the bias are typically learnable parameters.