* Add bfloat16 softmax and gelu

* Add pass attr bfloat16_enabled_op_types

* Changes from review

* - Added sum bf16 oneDNN

test=develop

* - Fix to UT of sum bf16

test=develop

* add conv_bias pass version test

* update according to reviews

* enhance the op_version_registry, test=develop

* add unittests, test=develop

* enhance the op_version_registry, test=develop

* fix bugs, test=develop

* revert pybind_boost_headers.h, test=develop

* fix a attribute bug, test=develop

* add fuse_bn_add_act pass

* add multiple exception type

* define all exception & polish compile pystack

* mapping paddle error to python exception

* polish static mode error format

* fix failed unittests

* fix dytostatic test_error

* fix check_nan_inf failed

* add unittest for coverage

* revert some code try to solve compile error

* refactor enforce & error change

* polish code & add unittest

* add sharding

* add compile limit for paddle enforce

* polish elementwise_op_function.cu.h

* fix failed unittest

* fix windows compile failed

* detail polish

* revert no type constructor

* fix bug of fetch_async_op_handle

* revert some changes of test_buffer_shared_memory_reuse_pass

* revert some changes of test_buffer_shared_memory_reuse_pass

* conv dilated mkldnn support: forward and backward pass

* add mkldnn conv_transpose dilation UT
test=develop

* remove unnecessary PADDLE_ENFORCE

* add int8 and bf16 dilated conv UT

* update according to reviews

* Implement the function of OueScaleForTraining/OutScaleForInference in dygraph

test=develop

* op error info

* style

* code format

* create pin var bug

* add multitask

* add multitask, test=develop

* fix code style, test=develop

* add partail push dense, test=develop

* fix has_kay in py3, test=develop

* fix, test=develop

* fix, test=develop

* fix, test=develop

* add sumary feature

* refine printting tensor

* add sci_mode

* add sample code

* fix indent error

* fix _format_item

* polish code

* support item indent

* add ut

* set place for ut

* fix py2 issue

* fix ut

* refine fleetrun.ps_launch

* update fleet run for multi device support

* ps_graph support ps-gpu

* fix heter save

* add heter save unittest

* fix unittest & simple code

* update fleetrun

* fix fleetrun

* fix launch barrier

* fix role maker

* add paddlecloud rolemaker unittest

* rename heter_worker_device_guard

test=develop

- compilation fix

test=develop

* - condidate fix to issue #25537

test=develop

* - UT for transpose NHWC

test=develop

* unique op for cuda is added

* add support for cuda

* Add cuda support for unique op.

* Add support for int32_t and int64_t.

* For old version, process by cpu

* Add VisitDataType for thrust

* refine broadcast_op_handle

* refine some error messages

* refine some files

* fix bug

* fix bug

* fix bug

* follow comments

* follow comments

* fix test_dist_fleet_heter_ctr & peformance update

* add xpu in heter mode
test=develop

* BOOST_CONST_GET; PADDLE_THROW
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* refine
test=develop

* refine
test=develop

* refine
test=develop

* refine code
test=develop

* Add pass compatibility check for four mkldnn pass, test=develop

* use iwyu clean include, test=develop, test=win

* compilation error, test=develop

* fix compilation error2, test=develop

* fix compilation error3, test=develop

* fix compilation error4, test=develop

* fix compilation error5, test=develop

* fix compilation error6, test=develop

* fix compilation error7, test=develop

* fix compilation error8, test=develop

* fix compilation error8, test=develop

* fix compilation error10, test=develop

* fix compilation error11, test=develop

* fix bug sequececonv_eltadd_relu_fuse_pass, output error when sequence_conv's padding_start > 0

* fix seqconv_eltadd_relu_fuse_pass unitest error

* support use add instead of sum to do gradient accumulation

* add inplace addto pass

* add grad_add op and inplace addto pass

* remove debug code

* code refine

* fix bug when sereral sum ops inserts at same op_idx

* fix Flags type

* add addto attribute for conv3d

* fix ut

* code clean

* fix type

* refine error msg in var_handle.h, test=develop

* refine all_reduce_op_handle

* fix some error msg

* refine variable_visitor

* refine threaded_ssa_graph_executor

* refine inplace related files

* refine executor related files

* refine fetch_op_handle.cc

* fix bug

* follow comments