optimize softmax with cross entropy soft label (#32387)
softmax_with_cross_entropy optimization with soft label. This PR includes optimization of
"SoftmaxWithCrossEntropySoftLabel" : compute log_softmax and then compute loss.
"CrossEntropySoftLabel" : compute loss with softmax as input.
These optimization includes following technics:
read data to buffer with vectorization
compute max and sum in warp
fixed loop size with macro
Performance (computation time):
softmax_with_cross_entropy_0 (forward) : -40.1%
softmax_with_cross_entropy_0 (backward): -41%
Showing
想要评论请 注册 或 登录