* sharding support dp

* sharding support mp

* sharding support pp

* fix warning error , test=document_fix

* fix warning error , test=document_fix

* fix warning error , test=document_fix

* fix warning error , test=document_fix

* fix warning error , test=document_fix

* fix warning error , test=document_fix

* fix warning error , test=document_fix

* add test_cumprod_op

* Revert "add test_cumprod_op"

This reverts commit c96cf6dff5d09ae7d8cc72c1e8ae4369a153aa19.

* recommit

* add error message

* test input(x) initialize

* test use cpu

* update test code

* add test type

* add test case

* solve ci problem

* add complex case test

* add complex case test

* fix review problem

* fix conflict

* fix some docs

* change test case

* change test case

* fix review problems again

* fix docs

* fix inclusivescan bug

This PR supports gradient clip (ClipGradByGlobalNorm) when training with AMP(auto mixed precision).

* add llvm::SmallVector to paddle

* rename small vector file

* merge paddle small vector to one file

* add small_vector_test

* modify smallvector test argument type

* add string header

* import ska flat_hash_map

* add define NOMINMAX macro to fix windows build failed bug

* add brackets to std::max in flat_hash_map

* move flat_hash_map directions

* modify namespace to paddle

* modify namespace to paddle

* modify namespace to paddle

* modify namespace to paddle

* rm not used map.h and replace with op_info

* add as_extra to dropout op and lrn op

* refine details

* fix dropout op maker

* fix extra op for expand, expand_as, tile, unstack

* Update expand_v2_op.cc

* fix bn/in/squeeze/syncbn extra

* update bn

* update

* update

* add opdef extra

* add reduce mean

* update style

* fix scatter gather bug:

* fix windows ci

* conv3d

* remove const_cast

* modify ut

* disable dynamic shape for trt6.0

* remove trt5

* add asExtra for nce op

* fix unittest error in macos

* remove asExtra for is_test

* mark extra attr for unsqueeze2

* debug for inference

* init matrix_rank op, add matrix_rank CPU code and test

* add GPU kernel, remove svd_eigen.h

* add CPU kernel when tol is tensor

* add cpu and gpu code when tol is tensor

* fix CI-ROCM error

* add matrix_rank API describe, fix PR-CI-Py3 error

* fix PR-CI-Windows error, add matrix_rank API test

* delete useless comments

* fix review

* add my code in svd_helper.h

* update doc commets

* remove spaces

* add backward inplace for dygraph

* fix bug

* support gradient accumulation