* update

* add conv yaml

* add backward

* remove useless code

* fix bug

* fix bug

* revert fluid dygraph conv2d

* remove useless infermeta function

* fix meta fn deluplicat error

* conv using custom impl

* remove amp include

* fix bug

* use final op in maskrcnn and hrnet

* refine
Co-authored-by: Nphlrain <phliuhongyu@126.com>

* [Eager] Support SelectedRows in eager mode

* Remove unnecessary codes

* Adapt new dygraph flag

* refactor eager flags

* fix flags error when we switch from eager to dygraph

* fix ci problem

* fix ci

* fix ci

* merge develop and fix code style

* merge develop and fix code style

* fix op test error

* fix op test error

* fix op test error

* fix op test error

* fix op test error

* merge develop

* add DistributedFusedLamb op

* polish code

* fix compile error

* compatible with pten changement

* fix rocm compile error

* improve converage

* update upstream/develop

* fix cast_with_ptr.h

* add FLAGS_distributed_lamb_divide_nranks_when_allreduce=1

* fix clip before allreduce

* add use_master_param_norm

* code polish

* fix bug

* fix ROCM ci

* refine clip

* delete unused code

* refine logic for clip

* support multi precision update for LAMB

* hide some api

* fix ci uts

* fix lamb output of dygraph

* remove some changes to some PR

* try to fix Py3 CI compile error

* fix test_imperative_optimizer, add lars ut, add layer_norm ut

* fix ut, fix format

* fix ut

* fix windows ci

* add fp16 for clip_by_norm api

* support ClipByGlobalNorm for fp16 in dygraph

* add unittest for dygraph clipGlobalNorm

* refine unittest for dygraph clipGlobalNorm for mac and windows

* refine unittest

* add unittest for fp64

* refine unittest for fp64

This PR supports gradient clip (ClipGradByGlobalNorm) when training with AMP(auto mixed precision).

* fix paddle.optimizer test=document_fix

* fix paddle.optimizer test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix nn formula bugs test=document_fix

* fix nn formula bugs test=document_fix

* fix nn formula bugs test=document_fix

* upgrade comment string to raw string

* fix string in

* fix string with ' '

* revert update on comments

* upgrade only necessary

* fix sample code checker

* fix comments with '''

* add sharding

* fix dtype of gradientclipbyglobalnorm

* fix dtype bug of GradientClipbyGlobalnorm

* Upgrade `no_grad` decorator

test=develop

- match torch decorator usage (i.e., with parenthesis)
- handle generator functions
- add `paddle.no_grad` alias

* Switch from `functools` to `decorator`

preserves signature

* Reword decorator usage note

* test=develop, test=document_fix

* test=develop, test=document_fix
Co-authored-by: Nswtkiwi <1208425345@qq.com>

梯度裁剪的策略进行了升级，配合修复相应的 裁剪API、minimize、ParamAttr 的API英文文档。

对应API变动的文档： #23224 

对应中文文档PR：PaddlePaddle/FluidDoc#1942

speed create clip op

* fix grad clip， clip op belongs to Backward op when running in Parameter Server mode.

* fix GradientClipByNorm english doc
test=develop
test=document_fix

* Fix ErrorClipByValue, GradientClipByGlobalNorm, GradientClipByValue

* add API.spec

* fix en docs of apis, test=develop, test=document_fix

* follow chunwei's comments, test=develop

* test=develop, fix docker with paddle nccl problem

* test=develop, Add Variable api and refine dygraph related API

* test=develop, Add Variable api and refine dygraph related API

* test=develop, refine test for new api and error info

* test=develop, refine error info and test_layers

* test=develop, add API.spec

* test=devleop, fix to_string python2 and python3 compat error and refien doc

* test=devleop, add API spec

* test=devleop, update API spec

* test=devleop, update API spec

* test=develop, invoke ci

* test=develop, fix example code

* test=develop, update API spec

* test=develop, add compat test and fix inplace campat dict error

* expose set_gradient_clip, test=develop, test=document_preview, test=preview

* expose gradient clip, test=develop, test=document_fix

* refine doc, test=develop

* follow lanxiang's comments, test=develop, test=document_fix

* test=develop, fix docker with paddle nccl problem

* test=develop, hide no_support api and add ut for it

* fix regularizer lod bug; test=develop

* fix exception bug and one_hot expand; test=develop

* [DOC] Fix api docs, test=develop, test=document_preview

* [DOC] Fix api annotation: fluid.layers.tensor_array_to_tensor. test=develop, test=document_preview

* test=develop, test=document_preview

update MD5 of tensor_array_to_tensor

* BugFix: fix api comment of paddle.fluid.clip.GradientClipByValue
* test=develop, test=document_preview

* expose square
test=develop

* fix activation
test=develop

* Add square API
test=develop

* add necessary op

* code refine

* fix API.spec
test=develop

* fix unit test
test=develop

* add unit test sparse_grad_clip
test=develop

* fix API.spec
test=develop

* remove mac test for test_gradient_clip
test=develop

* remove selectedrows_mul_tensor
test=develop