* support layer_norm prim op bf16 dtype

* polish code

* resolve conflict

* static graph autogen code for check_finite_and_unscale_ op

* bug fixed

* [AutoParallel] update while control_flow with pipeline

* update process group instantiate

* fix micro_bsz for reshard

* update api for micro batch size

* add strategy for dp optimization

* mv all unittests test

* fix error

* fix error

* fix

* fix

* del unittests

* fix paddle_build.sh

* fix

* fix test

* fix add test

* fix

* fix

* fix

* merge develop

* fix

* fix

* fix

* fix

* fix

* merge develop

* fix test_async_read_write

* fix test_async_read_write

* merge develop

* fix

* fix import legacy_test

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix bug

* fix

* fix coverage test bug

* fix

* fix

* fix

* fix

* fix

* fix code sstyle

* fix code

* fix code

* fix

* fix

* fix

* del test_sequence_enumerate_op.py

* fix

This reverts commit 4bcb5cc4.

* support activation prim op bf16 dtype

* remove useless code

* add deal with np.int type

* add assertEqual for dtype.

* fix bug

* fix by comment

* fix np.integer.

* fix the strong Wolfe conditions for line search

* pass the max_line_search_iters parameter into line search method (strong_wolfe)

* Reimplement the check_nan_inf function as check_numerics kernel.

* Remove the cpu implemention to phi.

* Add ifdef for the including of omp.h.

* Move the use of FLAGS_check_nan_inf_level out of header file.

* Implement a common PrintAndThrowError function.

* Fix the error using of __NVCC__, which should be instead with __CUDA_ARCH__.

* Add dependency of phi.

* Polish codes and unittest.

* modify gradOpMaker

* modify concat bug

* modify concat bug

* delete unnecessary block

* modify fill_any_like value from 1 to 0

* modify fill_any_like dtype from other to -1

* ci_bug

* [Auto Parallel] Reorganize the fold structure

* [Auto Parallel] Fix some import errors

* Copy QAT files from PaddleSlim

* Integrate QAT API into Paddle

* Replace eval function

* Reduce test_quant_aware run time

* Apply new formatter on modified files

* Remove code check for Paddle version check

* Copy quant_post_quant_aware UT from PaddleSlim

* Integrate test_quant_post_quant_aware UT into PaddlePaddle

* Apply new formatter on modified files

* Remove redundant code and add unittests

* Add new unittests

* Update the time limit of new unittests

* create phi so

* fix ci bugs

* fix py3 bugs

* add file

* fix py3 bugs

* fix windows bugs

* perfect so

* fix py3 bugs

* delete all static target in phi

* fix windows bugs

* fix py3 bugs

* fix ci bugs

* fix windows bugs

* fix bugs: gflags can't be linked by dynamic and static lib

* fix bugs that can not load 3rd party

* fix ci bugs

* fix compile bugs

* fix py3 bugs

* fix conflict

* fix xpu bugs

* fix mac compile bugs

* fix psgpu bugs

* fix inference failed

* deal with conflict

* fix LIBRARY_PATH bug

* fix windows bugs

* fix onednn error

* fix windows compile bugs

* fix windows compile bugs

* fix test_cuda_graph_static_mode_error aborted

* fix windows bugs

* fix mac-python3 error

* fix hip compile bugs

* change mode to static

* change to static mode

* fix ci bugs

* fix py3 bugs

* fix windows bugs

* fix bugs

* add static flag

* add PADDLE_API

* change position of PADDLE_API

* fix windows bugs

* change mode to dynamic lib

* fix windows static bugs

* deal with conflict

* fix windows unit bug

* fix coverage

* deal with conflict

* fix windows-inference

* fix py3 bugs

* fix bugs when compile type_info

* fix compile bugs

* fix py3 bugs

* fix windows bugs

* fix windows openblas

* fix xpu bugs

* fix enforce_test in windows

* update code according comment

* fix windows cmake bug

* fix windows bugs

* fix windows bugs

* delete cinn unittest

* fix cinn bugs

---------
Co-authored-by: lzydev <1528794076@qq.com>

* global view process_group

* fix import

* fix attr

* fix tunner init comm

* fix test error

* fix test_exectuor_feed_non_tensor

* fix test error

* [AMP OP&Test] Support float16 in selu

* fix

* Enable check_cinn on some tests

Tests: bitwise, compare, shape, assign_value, sum, expand_v2,
lookup_table, lookup_table_v2

* Enable more CINN tests

Tests with CINN: expand_v2, matmul, matmul_v2, mul, norm, one_hot_v2
Add target select in cinn_launch_op

* Revert test_mul_op

* Improve op unit tests

* add host memory stats

* add ut

* [CustomDevice] fix auto_paralell

* update

* update

* update

* fix processing logic of the arange function when dtype is empty.

* update commit version

* fix ValueError when end is None.

* add unitest for new case.

* fix tensor type.

* remove paddle.to_tensor(), add more test unit.

* remove useless line.

* fix enable_static

* add new test unit.

* fix by comment.

* [0D-Tensor] Support elementwise_add

* support elementwise_add ZeroDim2&3

* unify code

* remove useless code

* polish

* python/paddle/distributed/fleet/meta_parallel/pipeline_parallel.py

* polish

* polish