* add weight_norm, test=develop

* move paddle.fleet to paddle.distributed.fleet

* add auto_cast, test=develop

* add loss scaler, test=develop

* add comments, test=develop

* refine code, test=develop

* refine code, test=develop

* do not set flags automatically, test=develop

* fix custom op bug, test=develop

* add more test, test=develop

* refine enable logic, test=develop

* enable amp test with GPU, test=develop

* add unittest

* add test for found_inf

* follow comments

* follow comments

* remove global variable, use singleton

* add some notes

* update comments

* update comments

* update comments

* add use_dynamic_loss_scaling argument

* refine found_inf

* refine found_inf

* Directory migration, test=develop

* Change imperative from paddle init to paddle framework, test=develop

* Fixed jit bug, test=develop

* default static mode, test=develop

* fixed format and create parameter belongs to framework, test=develop

* Fixed import package, test=develop

* fix __init__ format, test=develop

* fixed alias problem

* fixed paddle.enable_imperative problems, test=develop

* Add unittest

* delete install_check comment

* Fixed unittest timeout

* fixed unittest error

* move Program default_xx_program to static package

* optimize unittest method

* fixed framework __init__ format

* fixed jit path

* delete alias

* move jit to paddle

* Fixed unittest format

* fixed paddle.default_main_program

* Fixed save load API in paddle __init__.py

* fixed ci paddle.imperative.to_variable

* test=develop,test=document_fix, remove the out args

* fleet_util move to paddle.fleet
Co-authored-by: NWuHaobo <wuhaobo1994@gmail.com>
Co-authored-by: Ntangwei12 <tangwei12@baidu.com>

* add fleetrun command for distributed running; test=develop

* Remove slim from paddle framework
test=develop
Co-authored-by: Nwanghaoshuang <wanghaoshuang@baidu.com>

Integrated Trainer of Parameter Server (API add `fluid.contrib.layers.sparse_embedding` only) (#22957)

* Integrated Trainer of Parameter Server

refactor fleet api under paddle.fleet
update DistributedStrategy

* experimental refactoring, test=develop

* add TranslatedLayer & remove StaticModelRunner, test=develop

* revert tracedlayer change, test=develop

* fix test_mnist unittest error, test=develop

* add doc & examples, test=develop

* polish doc details, test=develop

* add imperative.jit module, test=develop

* change TranslatedLayer pos, test=develop

* adjust jit module import path, test=develop

* polish doc based review result

* add SaveLoadConfig.separate_params to save paraams separately

* add Layer.buffer support, test=develop

* polish doc details based review result, test=develop

* polish details baesd review comments, test=develop

* add empty str check for param, test=develop

* add unittests, test=develop

* polish details based review comment, test=develop

* remove blanks in comment, test=develop

* polish doc details, test=develop

* update imperative doc link, test=develop

* add api attr for load, test=develop

* Add the imperative quantization aware training.
*  This is the python part of Imperative QAT. test=develop

* add paddle.fleet.DistributedStrategy for 2.0

* add base name alias; test=develop

* fix embedding bug; test=develop

* fix io error; test=develop

* add metric; test=develop

* add import palce; test=develop

* add setup; test=develop

* fix incubate reader; test=develop

* fix initializer error; test=develop

* Add hapi.text and corresponding unit test.
test=develop

* Remove hapi.text apis' reuse parameter args for coverage.
test=develop

* Fix TransformerCell and TransformerBeamSearchDecoder example codes.
test=develop

* Fix example codes in hapi.text.
test=develop

* Add some apis in hapi.text into example code white list.
test=develop

* Fix example code of DynamicDecode in hapi.text.
text=develop

* Rename Model.self as model in test_text.py
test=develop

* add vision

* fix predict, test=develop

* add unittest for vision apis, test=develop

* fix typos

* add hapi models api, test=develop

* fix code format, test=develop

* fix typos, test=develop

* fix sample code import, test=develop

* fix sample codes, test=develop

* add decompress, test=develop

* rm darknet, test=develop

* rm debug code, test=develop

* Merge hapi into Paddle

Hapi is a high level API for training and inference.
The main modules include Model, Loss, Metrics, Dataset.
Also includes common modules and models in NLP and computer vision, such as BERT, ResNet.

These modules are developed by:
0YuanZhang0, guoshengCS heavengate, LielinJiang, qingqing01, xyzhou-puck huangjun12, wangxiao1021, zhangyang.

* solve mklml memory leak on windows

test=develop

* remove unused msvcr120.dll

test=develop

* Move complex into dir incubate, test=develop

* Fix imports, test=develop

* Fix docs, test=develop

* Forbid import functions from paddle.incubate directly, test=develop

* Init complex number neural network, test=develop

* Improve doc writing, test=develop

* Fix elementwise add & sub, test=develop

* Fix elementwise mul act, test=develop

* a) add ut for complex variable; b) remove arg act in elementwise_ops. test=develop

* add DataLoader, Dataset, BatchSampler

* Optimize the error messages of paddle CUDA API, test=develop

* fix the error messages of paddle CUDA API, test=develop

* Refactoring PADDLE_ENFORCE_CUDA_SUCCESS, and apply to curand/cudnn/cublas/NCCL,test=develop

* remove build_ex_string,test=develop

* merge conflict,test=develop

* test=develop
Co-authored-by: Nwuxing03 <wuxing03@baidu.com>

* fix judge pslib & ranspiler

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* aAdd new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Add new API : randn

test=develop

* Remove the NGraph engine from PDPD repository
1. Each operator was removed from the operator's directory
2. Each test was removed from the unittest directory
3. The parallel executor support was removed from the PDPD
4. The CMake file was removed from the PDPD
5. The NG flags were removed from the repository
test=develop

* Remove ngraph from:
1. Cmake file
2. Python file
test=develop

* add functional conv

* add test and doc for function convs, test=develop

* update ConvTransposeOp's InferShape and error message, test=develop

* add L1Loss

* support L1Loss, test=develop

* add test, test=develop

* fix batch, test=develop

* follow comments, test=develop

* get pslib version
* test=develop

This reverts commit 66fce9e8.

Fleet Parameter Server API Integrated