* add reducer

* refine envent for memorycopy

* add concat&split for allreduce

* apply concat & split for fuse tensor

* fix nccl dep

* fix the untest, compile problem and ddp initialize problem

* fix untest for mac & add some comments & solve the repeated param in sublayers

* fix untest for windows & fix document

update expand as op to use the shape of the target tensor instead of the target tensor itself. (#29020)

* update, test=develop

* alias yolo_loss & decode_yolo_box to paddle.vision. test=develop

add paddle.subtract, optimize paddle.maximum and paddle.minimum 

Add eigen gru and fix the dropout bug in the rnn 

[Dynamic-to-Static] Support **kwargs as input of the function which is decorated by `jit.save.to_static` (#29098)

* remove -DSUPPORTS_CUDA_FP16 in cuda.cmake

* comile with cuda9

* add some unittest

* notest;test=coverage

* add unittest for trt plugin swish && split

* update ernie unittest

* fix some error message

* remove repeated judgement of CUDA version in mbEltwiseLayerNormOpConverter

* fix comile errror when CUDA_ARCH_NAME < Pascal"

* fix comile error

* update unittest timeout

* compile with cuda9

* update error msg

* fix code style

* add some comments

* add define IF_CUDA_ARCH_SUPPORT_FP16

* rename IF_CUDA_ARCH_SUPPORT_FP16 to CUDA_ARCH_FP16_SUPPORTED

* add symlink force for unittest

* open unittest

* update, test=develop

* split train_mode and has_grad

* fix format

* fix ci problems

* fix sample code

* add lars to fleet meta optimizer

* add lamb to proto

* add lamb to fleet meta optimizer

* fixed syntax bug

* fixed syntax bug

* fixed syntax error in lamb, add config setter of lamb in distributed_strategy

* trigger unitest to rerun

* add new unitest func for lamb

* revise unitest for lars and lamb

* revise dgc meta unitest

* revise lars document in distribute_strategy

* revise lars lamb document in distributed_strategy.py

* revise lars lamb document in distributed_strategy.py

* add weight decay exclude logic to lars

* restore optimzier.py

* restore optimizer.py as develop except lars

* add epsilon and exclude fn to distributed_sttrategy

* add lars epsilon

* revise unitest for fleet lars and lamb

* revise lars lamb unitest for CI coverage

* revise lars argument api

* revise lars argument api

* revise lars argument api

* revise api doc of lars

* fix op role

* add sharding save and add_sync_comm_for_test function

* add comm_analyse to utlis

* revise sharding_utils

* add sharding saving unittest

* revise sharding utils for unittest

* revise sharding en doc

* update sharding utils api

* add doc for sharding

* fixed bug in sharding var size count

* update varsize count in sharding

* fix sharding num_nccl_comm

* Revert "fix sharding num_nccl_comm"

This reverts commit d51587c15e9323acf226ddd36154275f0d1daf76.

add API serialize_program, serialize_persistables, save_to_file, deserialize_program, deserialize_persistables, load_from_file. (#29034)

* remove BatchSampler type check. test=develop

* Add bf16 pool2d and unify bf16 unit tests

* Add change default ops test

* default not show cpp statck & add hint

* fix failed unittest

* fix failed unittests

* change load dict file one by one to warn

* add unittests for coverage

* polish error message

* fix cond error

* Rename variables when use 'jit.load'

* Check whether the original graph contains the variable with the same name

* add comment

* rename output/input of op and edit unittest

* modify the code according to CI

* edit code according to CI

* edit code according to CI

* edit code according to CI

* edit code according to CI

* edit code according to CI

* edit code according to CI

* fix syncbn

* add unittest

add uint8 for reshape operator

* Support pop for dict in dy2stat

* Move convert_pop to convert_operators.py and polish convert_pop

* fix tensor detach to zero copy

* fix tensor detach to zero copy

* polish parallel api impl & doc details

* add unittest for coverage

* remove spawn test in py2.7

* add parallel api into white list

As the title

* add soft_label and axis for CrossEntropyLoss and improve performance,test=develop

* fix conflict in nn/functional/loss.py, test=develop

Add support for using tuple as tensor.shape (For example: a, b, c, d = x.shape)

* add lamb optimizer and unittest

* fix lamb

* fix lamb v2 op

* fix sampling id

* fix lamb sample code

* Update lamb.py

* fix doc

* fix doc

* Update lamb.py

* upgrade comment string to raw string

* fix string in

* fix string with ' '

* revert update on comments

* upgrade only necessary

* fix sample code checker

* fix comments with '''