* optimize update_loss_scaling_op by fused for loop to one kernel, test=develop

* remove useless while loop and optimize variable name, test=develop

* optimize variable name from out_addrs_tensor to out_addrs_mem, test=develop

* optimize variable name for readable by change prefix identifier from t_ to local_

* [OPs] Bug fix, fix the segment mean for illegal syncthreads usage.

This reverts commit 4b7242b0.

Co-authored-by: NYang Zhang <yangzhang@live.com>

* add sendrecv, test=develop

* new optimize for where_index_op with prefix sum version.

* write a scan prefix sum kernel with stream for where index op.

* optimize where_index by using cub::DeviceScan::InclusiveSum instead of imperfect self-kernel.

* remove CheckTrue struct and rename stide_array for readable.

* optimize variable name for readable.

* optimize function name and annotation.

* optimize pull sparse

* optimize pull sparse

* change macro

* format

* Unset ReserveSpace for inference program.

* Support training from an inference program.

* fix model parallel

* rm parallel_help.py

* add embedding

* support backward return None.

* edit unittest.

* edit code according to CI

* Improve error information

* optimize slice op and slice grad op, test=develop

* optimize variable name and annotation information, test=develop

* skip op has no fp16 kernel

* add ut

* fix airank bert emb order

* move input num check to converter

* add input num check

* add unused var check white list

* add trt runtime version check

* use different wrap, and change to major version check

* update

* fix bug: when x.dim < y.dim, the result of compare_op is inverse to expected result

* support the cuda for fix the compare broadcast bug

* fix tc trt shape

* fix fc dynamic shape

* add fc shape assert

* update

* use ZerosLike instead of NPUMemsetAsync

* fix compile

* Add dlnne engine runtime

* Fix log

* Remove <const_cast> and remove unrelated modify with dlnne, +clang-format

* Fix CMakeList format error

* Add copyright message

* Fix dlnne CMakeList.txt

* Add some paddlepaddle_pass to support more networks

* Fix some format bug

* add c_identity op, test=develop

* Refine Constructor logic of ParallelExecutor

* refine function name

* refine code comment

* refactor_check_finite_and_scale_npu_kernel

* fix compile

* add alloc_float_status op

* add alloc_float_status op

* add FloatStatus for check_finite_and_unscale

* refine code

* remove unneccessary logic

* refine for fleet