Before introducing the classification algorithms and training procedure, we provide some definitions:
- $X$ is input:MNIST image is $28\times28$ two dimensional matrix. It is reshaped to $784$ dimensional vector. $X=\left ( x_0, x_1, \dots, x_{783} \right )$。
- $Y$ is output:Output of classifier is 10 class digits from 0 to 9. $Y=\left ( y_0, y_1, \dots, y_9 \right )$,Each dimension $y_i$ represents a probability that the image belongs to $i$.
- $L$ is a image's ground truth label:$L=\left ( l_0, l_1, \dots, l_9 \right )$ It is also 10 dimensional, but only one dimension is 1 and others are all 0.