* support multi precision update for LAMB

* hide some api

* fix ci uts

* fix lamb output of dygraph

* remove some changes to some PR

* try to fix Py3 CI compile error

* fix test_imperative_optimizer, add lars ut, add layer_norm ut

* fix ut, fix format

* fix ut

* fix windows ci

* add fp16 for clip_by_norm api

* support ClipByGlobalNorm for fp16 in dygraph

* add unittest for dygraph clipGlobalNorm

* refine unittest for dygraph clipGlobalNorm for mac and windows

* refine unittest

* add unittest for fp64

* refine unittest for fp64

This PR supports gradient clip (ClipGradByGlobalNorm) when training with AMP(auto mixed precision).

* fix paddle.optimizer test=document_fix

* fix paddle.optimizer test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix nn formula bugs test=document_fix

* fix nn formula bugs test=document_fix

* fix nn formula bugs test=document_fix

* upgrade comment string to raw string

* fix string in

* fix string with ' '

* revert update on comments

* upgrade only necessary

* fix sample code checker

* fix comments with '''

* add sharding

* fix dtype of gradientclipbyglobalnorm

* fix dtype bug of GradientClipbyGlobalnorm

* Upgrade `no_grad` decorator

test=develop

- match torch decorator usage (i.e., with parenthesis)
- handle generator functions
- add `paddle.no_grad` alias

* Switch from `functools` to `decorator`

preserves signature

* Reword decorator usage note

* test=develop, test=document_fix

* test=develop, test=document_fix
Co-authored-by: Nswtkiwi <1208425345@qq.com>

梯度裁剪的策略进行了升级，配合修复相应的 裁剪API、minimize、ParamAttr 的API英文文档。

对应API变动的文档： #23224 

对应中文文档PR：PaddlePaddle/FluidDoc#1942

speed create clip op

* fix grad clip， clip op belongs to Backward op when running in Parameter Server mode.

* fix GradientClipByNorm english doc
test=develop
test=document_fix

* Fix ErrorClipByValue, GradientClipByGlobalNorm, GradientClipByValue

* add API.spec

* fix en docs of apis, test=develop, test=document_fix

* follow chunwei's comments, test=develop

* test=develop, fix docker with paddle nccl problem

* test=develop, Add Variable api and refine dygraph related API

* test=develop, Add Variable api and refine dygraph related API

* test=develop, refine test for new api and error info

* test=develop, refine error info and test_layers

* test=develop, add API.spec

* test=devleop, fix to_string python2 and python3 compat error and refien doc

* test=devleop, add API spec

* test=devleop, update API spec

* test=devleop, update API spec

* test=develop, invoke ci

* test=develop, fix example code

* test=develop, update API spec

* test=develop, add compat test and fix inplace campat dict error

* expose set_gradient_clip, test=develop, test=document_preview, test=preview

* expose gradient clip, test=develop, test=document_fix

* refine doc, test=develop

* follow lanxiang's comments, test=develop, test=document_fix

* test=develop, fix docker with paddle nccl problem

* test=develop, hide no_support api and add ut for it

* fix regularizer lod bug; test=develop

* fix exception bug and one_hot expand; test=develop

* [DOC] Fix api docs, test=develop, test=document_preview

* [DOC] Fix api annotation: fluid.layers.tensor_array_to_tensor. test=develop, test=document_preview

* test=develop, test=document_preview

update MD5 of tensor_array_to_tensor

* BugFix: fix api comment of paddle.fluid.clip.GradientClipByValue
* test=develop, test=document_preview

* expose square
test=develop

* fix activation
test=develop

* Add square API
test=develop

* add necessary op

* code refine

* fix API.spec
test=develop

* fix unit test
test=develop

* add unit test sparse_grad_clip
test=develop

* fix API.spec
test=develop

* remove mac test for test_gradient_clip
test=develop

* remove selectedrows_mul_tensor
test=develop

* fix api format and examples

test=develop

* Update executor.py

test=develop

* Update nn.py

* Update nn.py

test=develop

* Update nn.py

test=develop

test=develop

* Yuyang fix global gradient clip

* Share LoDs

* Revert unnecessary changes

* Fix bug in sequence_slice_op

* hide operator API

* update

* update api.spec

* fix merge

* fix test