* support use add instead of sum to do gradient accumulation

* add inplace addto pass

* add grad_add op and inplace addto pass

* remove debug code

* code refine

* fix bug when sereral sum ops inserts at same op_idx

* fix Flags type

* add addto attribute for conv3d

* fix ut

* code clean

* fix type

Fix bug: shapes of Topk outputs are wrong when the parameter k is Tensor 

* Finished ChannelWiseQuantDequantAbsMaxOp and Passed unittests.

* Finished channel-wise quantize strategy in imperative quantization.

* Added Cuda code of ChannelWiseQuantDequantMaxAbsOP
Add Cuda code of ChannelWiseQuantDequantMaxAbsOp

* Add quant_axis for channel_wise quant.

* fixed a bug in unnitests, which will not trigger axis = 1 case and cannot meet the coverage rate requirement.

* Added some assert infomation and fixed some coding style mistakes.

* fix fleet util and gloo

* fix worker endpoints

* fix

* fix UT

* fix gloo

* fix gloo

* update gloo

* update gloo

* update gloo

* update gloo

* update gloo

* fix gloo wrapper for hdfs

* add file gloo and UT

* fix UT

* fix UT

* fix UT

* hide public method of RoleMaker

* fix UT

* GPU fleetrun support gloo

* parameterserver fleetrun support gloo

* add UT

* add UT

* fix UT

* fix get server endpoint

* fix get server endpoint

* fix UT

* hide public method of rolemaker

* hide public method of rolemaker

* hide public method of rolemaker

* Update test_fleet_rolemaker_new.py

* hide public method of rolemaker

* hide public method of rolemaker

* fix worker endpoints

* fix gloo wrapper for hdfs

* GPU fleetrun support gloo

* parameterserver fleetrun support gloo

* fix get server endpoint

* pass compatibility

* add compatibility registry

* add unittests for different padding

* add assert

* drop errmsg

* Add env value to  log to stdout; 2.Add logger name

* Optimize log messages in dygraph-to-static

* Replace logging.warn and warnings.warn with logging_utils.warn

and optimize the c++ implmentation of empty op as PR#26659 reviews,
and add bool for shape op.

* test=develop, util documents

enhance reduce op which can reduce tensor with arbitrary rank 

* fix floordiv

* Refine the unittest to support py38

    test=develop

Change ProgramTranslator.enable_declarative to ProgramTranslator.enable_to_static to meet API 2.0

* add op_function_generator.exe retry in windows, test=develop

* modify test_imperative_using_non_zero_gpu from use two gpus to one gpu, test=develop

* remove objgraph

* remove graphviz

* fix ut

* add adaptivelsgd

* Todo fix the code to avoid the conflict.

* support load state_dict from save_params/persistables

* remove failed unittest

* add load eof check & unittest

* remove eof check

* fix error message

* add input_spec & output_spec for translated_layer

* update error message

* fix ports conflict when launching multi-nodes in paddlecloud;test=develop

* add DISTRIBUTED_TRAINER_ENDPOINTS env for cloud;test=develop

* optimize slice TRT plugin

This patch removes unnecessary barrier for data transfer of needed offset,
so data transfer can be overlap with GPU kernel execution.

This patch also fixes incorrect name of slice plugin. That is, replaces
"layernorm" with "slice"

test=develop

* add serialize/deserialize to slice plugin

* add static shape slice trt plugin

* fix slice trt op convertor dynamic shape bug

* fix format by clang-format

* fix pylint format error

* fix problems commented by peiyang
Co-authored-by: NRyan Jeng <rjeng@nvidia.com>

* optimize errror report

* add test case for pad op converter

* fix some spelling mistake commented by peiyang

* replace sequence length attr to input