Created by: zlsh80826

PR types

Function optimization

PR changes

OPs

Describe

Few changes to give the kernel SoftmaxKernelWithEltadd 1.25x speedup

1. Change the blockReduce behavior as issue #24731 (closed) describe. Every thread calls the blockReduceXXX can directly obtain the reduced value. Thus, shared memory broadcast isn't needed.
2. Number of launch threads for SoftmaxKernelWithEltadd align to 32 instead of power of 2.
3. vectorization when available