Integrated Trainer of Parameter Server (API add `fluid.contrib.layers.sparse_embedding` only) (#22957)

* Integrated Trainer of Parameter Server

* refine strategy compiler and meta optimizers
make async as a_sync

* simplify buffered reader to improve DataLoader performance

* fix 22 failed unittests

* fix cuda pinned context condition

* fix test_reader_reset failed

* fix two failed unittests

* change unittest place

* polish error messaage

* polish cast op GetExpecctedKernelType

* remove debug info in unittest

* add more settings for distributed strategy
Basically, DistributedStrategy has several parts of configurations:
- BuildStrategy: the same as paddle.fluid.BuildStrategy, but the distributed arguments are moved out of BuildStrategy
- ExecutionStrategy: the same as paddle.fluid.ExecutionStrategy
- collective communication configs: nccl_comm_num, hierarchical allreduce and so on
- distributed algorithms: async_update(mainly used in PS), lars, lamb and so on

* polish framework error message part3

* polish details

* fix error message print error

* Added DNNL cache management for DyGraph

* move FLAGS_use_mkldnn to more general CMakeLists, getu use of the flag in ClearGradients

* missing file

* Fixes after review

* Bringing back original idea of place for 'use_mkldnn' flag to be accessible from platform nad imperative.

* Removed duplicate and added docs

* Fixes for CI

* polish framework error meg part2

* polish details

* fix error msg report in ir/, about 19 files, test=develop

* modified some unclear descriptions, test=develop

* modified some unclear descriptions, test=develop

* modify unit test pass_test.cc, because the error report in pass.cc is used by pass_test.cc, test=develop

test=develop

* update ResetHolder, test=develop

* add TensorShare for lite engine, test=develop

* tensor data changed from copying to sharing, test=develop

* supports xpu runtime, test=develop

* fix code styles, test=develop

* polish some error message in framework, test=develop

* fix unittest error, test=develop

* replace PADDLE_ENFORCE, test=develop

* polish details based review comment, test=develop

* fix error msg report in ir/, about 5 files, test=develop

* fix error msg report in ir/, about 5 files, test=develop

* fix error msg report in ir/, about 5 files, test=develop

refactor fleet api under paddle.fleet
update DistributedStrategy

* fix paddle/fluid/framework/ir/ error msg reoprt, test=develop

* modify error msg reoprt in ir/, about errortype, grammar, supplementary infor, test=develop

* modified some unclear descriptions, test=develop

* Modify the problem that report msg is less than 20 characters, test=develop

* fix paddle/fluid/framework/ir/mkldnn/ error msg reoprt, test=develop

* modify error msg reoprt, about errortype, grammar, supplementary infor, test=develop

* modified some error descriptions, test=develop

* fix paddle/fluid/framework/ir/multi_devices_graph_pass/ error msg reoprt, test=develop

* fix paddle/fluid/framework/ir/memory_optimize_pass/ error msg reoprt, test=develop

* fix paddle/fluid/framework/ir/fuse_optimizer_ops_pass/ error msg reoprt, test=develop

* fix paddle/fluid/framework/ir/memory_optimize_pass/ error msg reoprt about PADDLE_ENFORCE, test=develop

* modify error msg reoprt,about errortype，grammar. test=develop

* modify error msg reoprt,about PADDLE_ENFORCE to PADDLE_ENFORCE_XXX, test=develop

* modify error msg reoprt,about PADDLE_ENFORCE to PADDLE_ENFORCE_XXX, and %s to %d, test=develop

* modified some error descriptions, test=develop

* fix softmax_with_cross_entropy cuda kernel overflow bug, test=develop

* replace old macro & for condition, test=develop

* polish details, test=develop

* polish pe exception process logic, test=develop

* fix unittest, test=develop

* add unittests, test=develop

* add device_worker for pipeline, test=develop

* cat bad alloc exception; test=develop

* add unitest; test=develop

* move bad alloc catch to the first place; test=develop

* polish error message; test=develop

* polish error message; test=develop

* add mutex header; test=develop

* fix multhead matmul's instable
test=develop

* fix multihead matmul bug
test=develop

* fix converage problem
test=develop

* add paddle.fleet.DistributedStrategy for 2.0

* add encrypt api for inference lib

* Paddle-TensorRT support slim QAT. test=develop

* add comments. test=develop

* use RenameInput instead of ResetInputs. test=develop

* don't re-generate header file if content doesn't change. test=develop

* add copy_if_different function. test=develop

* Start implementing int8 eltwise add

test=develop

* - Fix to Michal PR

* - Fix

test=develop

* - Lint fixes

test=develop

* - Added checking if elementwise_mul can be used

test=develop

* - Added attribs to skip_attrs_set

test=develop

* - Improved broadcasting

test=develop

- fixes to compilation

- fix

- fix

- Lint fixes

test=develop

* - removed redundant condition

test=develop
Co-authored-by: NMichal Gallus <michal.gallus@intel.com>

* Add a StatValue class in the backend to represent a stat.
* Add a singleton StatRegistry to maintain the collection of stats.
* For the sake of code neatness, we only support type of int and float, which can cover most of the scenarios.

* use allow list instead of white list, test=develop

* reduce include, test=develop

Support CMatchAucCalculator based on CMatchRankAucCalculator with a new parameter ignore_rank

* remove gmock from ut

test=develop

* coverage enabled for r+t+m fuse pass

test=develop

test=develop

* add the support of device index for device_guard.

* Fix the field length in LoD scenario
* Fix the missed lod info when copy tensor in dump field
* Add some log to make debug easy