remove the AdamOptimizer、SGDOptimizer、MomentumOptimizer、ModelAverage、LookaheadOptimizer、FtrlOptimizer、DecayedAdagradOptimizer、DpsgdOptimizer in fluid and relocate the ExponentialMovingAverage、PipelineOptimizer、GradientMergeOptimizer and change optimizer base for LarsMomentumOptimizer and RecomputeOptimizer (#55970)

* change the optimizer base for SGDOptimizer

* change the optimizer base for SGDOptimizer

* replace the SGDOptimizer with SGD

* fix bug of sgd

* change the optimizer base for MomentumOptimizer

* fix the remaining tests

* remove the Momentum in fluid/optimizer.py

* fix bug

* fix bug

* fix bug

* fix bug

* Update test_resnet_cinn.py

* Update test_resnet_prim_cinn.py

* fix bug

* fix bug

* fix bug

* remove the ModelAverage in fluid

* remove the LookaheadOptimizer in fluid

* fix bug

* remove AdamOptimizer in fluid

* Update test_image_classification_fp16.py

* fix bug

* relocate the ExponentialMovingAverage in fluid

* restore the static api

* remove the FtrlOptimizer in fluid

* remove the DecayedAdagradOptimizer in fluid

* remove the DpsgdOptimizer in fluid

* fix bug

* fix codestyle

* fix bug

* fix bug

* relocate the PipelineOptimizer

* relocate the GradientMergeOptimizer

* fix bug

* fix bug

* fix bug

* fix doc

* Update __init__.py

* Update test_fleet_qat_meta_optimizer.py

* change optimizer base for LarsMomentumOptimizer

* fix bug

* fix conflict

* fix code-style

* fix sample codes

* fix bug

* fix bug

* fix cinn bug

* fix bug

* fix bug

* Update qat_optimizer.py

* Update __init__.py

* fix bug

* change optimizer base for RecomputeOptimizer

* fix bug

* fix bug

* Update test_imperative_optimizer_v2.py

* mv all unittests test

* fix error

* fix error

* fix

* fix

* del unittests

* fix paddle_build.sh

* fix

* fix test

* fix add test

* fix

* fix

* fix

* merge develop

* fix

* fix

* fix

* fix

* fix

* merge develop

* fix test_async_read_write

* fix test_async_read_write

* merge develop

* fix

* fix import legacy_test

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix bug

* fix

* fix coverage test bug

* fix

* fix

* fix

* fix

* fix

* fix code sstyle

* fix code

* fix code

* fix

* fix

* fix

* del test_sequence_enumerate_op.py

* fix

* replace paddle.fluid.layers.data and remove io.data

* partial commit

* partial commit

* partial commit

* partial commit

* partial commit

* partial commit

* remove data in fluid.layers.io.__all__

* fix errors

* fix unitests

* fix unitest

* fix unitests

* fix unitest

* fix unitest

* fix unitests

* fix unitest

* fix test_layers unitests

* fix typro

* fix unitest

* fix unitest

* fix unitest

* fix typro

* fix unitest test_model_cast_to_bf16

* fix test_reducescatter

* fix collective unitest

* fix collective unitests

* fix collective unitests

* add coverage

* fix add layers.data

* re run ci

* fix some typro

* fix samplecode error

* fix samplecode error

* clear fluid apis in fleet and passes

* fix model.py

* fix model.py

* fix cpp_pass.py

* clear loss.py

* change test file

* fix some test_*.py

* fix adaround.py

* fix evaluator.py

* fix CI bug

* fix CI bug

* fix decode.py

* fix detection.py

* fix ci bug

* rm test_sigmoid_cross_entropy_with_logits_op_ipu.py and fix __init__.py

* fix ci bug

* fix ci BUG

* isort all files

* revert conflicting files

* revert conflicting files

* revert conflicting files

* revert conflicting files

* revert conflicting files

* update config

* re-blacken python code

* temporarily disable date and diff_py_file

* skip a format

* [CodeStyle] remove all future import

* revert test_error.py

* restore future import in example code

* change fluid.mean to paddle.mean

* reverse some old code examples

* use tempfile to place temporary files

* update

* revert test_communicator

* fix test_dist_base

* use yapf to format all python file

* yapf exclude two unittests file for they rely on writing and reading file, and format will break them

* disable diff_py_file because too many diff files cause command following failed

* pybind core

* set use psgpu

* pipeline adaptive for heterps;test=develop
* fix finalize hang;test=develop
* add is_compiled_with_heterps for dataset;test=develop
* fix hashtable core when pass ins_num=0;test=develop

* add trainprofiler for heterps in oneps; test=develop

* add set_use_ps_gpu; test=develop

* pscore support heterps

* fleet cmake

* fleet wrapper

* macro

* solve conflict

* solve conflict

* add unitest

* paddle enforce

* unitest

* unitest

* unitest

* oneps (3/4)
Co-authored-by: NMrChengmo <cmchengmo@163.com>
Co-authored-by: Nmalin10 <malin10@baidu.com>
Co-authored-by: Nchengmo <chengmo@baidu.com>

* fix test_dist_fleet_heter_ctr & peformance update

* rename communicator meet->BatchesCounter

* fix parame recv for sparse

* geo sparse init from pserver

* optimize init from pserver

* add large scale optimizer fuse(SGD/ADAM)

* rectification init_worker and exe.run startup program

* solve the initial configuration about fleet and rolemaker
Co-authored-by: NseiriosPlus <tangwei12@baidu.com>

* move paddle.fleet to paddle.distributed.fleet

* add paddle.fleet.AsyncOptimizer
Co-authored-by: Ndongdaxiang <dongdaxiang@baidu.com>

This reverts commit 66fce9e8.

Fleet Parameter Server API Integrated

* add sync communicator and implement

* add half_async in the communicator
* fix DistributedStrategy

* fix correctness of the communicator

* fix a bug in send thread when sending var context is empty, test=develop

* add lookup_table_prefetch_op and prefetch optimize, test=develop

* remove remote prefetch GPU supported

* word2vec force with CPU, test=develop

* test dist remote lookup table force with CPU, test=develop

Async exe support communicator

* added lrn op test=develop

* Added CreateConstant method test=develop

* avoid duplicates test=develop

* Enable momentum operator for a ngraph engine
test=develop

* Update tests
test=develop

* Unnecessary line of the code as intended was removed
test=develop

* fix compile issue with_distribute test=develop

* simplified logic test=develop

* use ngraph dependency test=develop

* set cpu only test=develop

* update test and eliminate fp16 test test=develop

* Added accuracy ngraph op test=develop

* fixed name type test=develop

Currently, our tests run with 2 GPUs, the init time is absurdly long:
about 4s for each process.  Currently, we run each OP test on
different processes. This PR:

1. create cmake function py_test_modules which will generate the
Makefile that runs a list of Python unittest module in a single Python
process.

2. move all "python unittest compatible" (e.g., used the unittest
package, not just a regular python file). from fluid/tests to
fluid/tests/unittests.

3. cmake now will run all OP tests in fluid/tests/unittests in a
single process, except the time-consuming tests, they are separated
into different processes to utilize parallelism. Please make sure to
use the unittest package if you put the python test file in
fluid/tests/unittests

4. remove all exit(0) from fluid/tests/unittests/*.py, exit(0) is used
to disable unittest, we can not do it when running all tests in a
single process since it will terminate the process without running the
other tests. Instead, the test is disabled in
fluid/tests/unittests/CMakeLists.txt. FIXME is added for each disabled
item. Please disable the unittest from
fluid/tests/unittests/CMakeLists.txt, instead of adding exit(0) to the
Python file, for all Python file in fluid/tests/unittests/.

5. add an option WITH_FAST_BUNDLE_TEST. When OFF, will run the unit
tests in separate process so that they can be tested individually.

The English of the previous API is bad.