Created by: demon386

The combination of cross entropy loss and softmax as last-layer's activation is the standard configuration in classification. To improve the numerical stability, many ML libraries combine the two as one layer. The mathematical motivation is explained in https://www.zhihu.com/question/40403377/answer/86783636 and http://freemind.pluskid.org/machine-learning/softmax-vs-softmax-loss-numerical-stability/

For references, see some implementations in other popular ML libraries:

• Caffe: https://github.com/BVLC/caffe/blob/master/src/caffe/layers/softmax_loss_layer.cpp
• Tensorflow: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/api_docs/python/functions_and_classes/shard4/tf.nn.sparse_softmax_cross_entropy_with_logits.md