* [AutoParallel] support bloom

* fix import

* align amp and bf16

* update func name

* clipbyglobalnorm and add_n support bf16

* upgrade amp strategy api

* update bf16 unittest

* fix static clip

---------
Co-authored-by: Nliangjianzhong <liangjianzhong@baidu.com>
Co-authored-by: NAurelius84 <zhangliujie@baidu.com>

* replace assert false with AssertionError

* 修改配置文件多余的部分

* fix flash_attention

* Update mp_layers.py

* set order for hybridparallel

* fix bug

* fix ->

* fix ->

* fix ->

* add topology

* fix utest

* add patterns

* add unittest

* remove super call with parameters

* fix bug

* add_up004_for_ruff

* 修改配置文件并清除object

* fix md

* remove no used fluid beam_search_decoder

* move Layer and related helper to paddle.nn.common

* modify Layer references from dygraph.layers.Layer to paddle.nn.common.layers

* stash changge

* remove fluid layer_object_helper, layers.py

* remove fluid layers init

* add setip

* fix unitest

* delete layers in fluid.dygraph

* merge paddle.tensor.stat,py

* fix circle import

* fix curcle import

* remove redundant in_dygraph_mode import

* revoce paddle.nn.common.* in fluid.__init__

* recovery nn.rnn

* paddle.frame use lazy import import paddle.jit to avoid circle import

* remove left dygraph.layers ref

* merge develop

* fix import error

* fix test error

* fxi merge error

* fix test fluid.Layer

* fix test error

* fix test error

* fix import error

* fix import error

* fix comments

* fix circle import

* fix rnn import error

* fix circle import

* migrate fill_constant to paddle.tensor

* move fill_constant to paddle.tensor and repalce the reference

* add missing fill_constant replacement

* fix typro

* remove unused import fill_constant

* fix zeros import error

* fix circle import

* fix layers.zeros

* fix unitest

* fix unitests

* fix unitest

* use paddle.full replace fill_constant in samplecode

* fix sample code

* recovery xpu test

* recovery xpu test

* fix circle import

* fix utils import error

* fix utils error

* fix circle import

* redo

* fix circle import

* fix prim fill constant import

* fix type error

* fix increase error

* fix test error

* fix fill_constant

* fix: fix broadcast object list

* fix: fix scatter object list

* fluid clean: remove fluid.ir to framework.ir and some funcs form fluid.layer.io to incubate.

* delete fluid.ir

* update env setting

* update pass logic

* dist op support bf16

* backward cast update

* update setting

* update backward

* revert amp pass

* update fp16 backward logic

* register c_embedding bf16

* revert engine

* add unitest

* add unitest

* update unitest

* update cmake

* update math

* update math.py

* update unitest

* update unitest

* revise unitest

* revise unitest

* update unitest

* update unitest

* update unitest

* optimizing setup.py develop command

* add libpaddle.so

* modify setup.py

* add python/paddle/distributed/fleet/.gitignore

* add libpaddle.so to .gitignore

* add *.so to python/paddle/libs/.gitignore

* add new gitignore

* refine _found_inf

* find relevant testcase

* remove with_data_parallel

* trigger CI

* do not apply ParameterServerGraphOptimizer

* remove useless optimizer

* remove with_data_parallel in test_dist_base

* fix test_fleet_base_3

* only reserve changes for GraphExecutionOptimizer

* fix bug

* fix test_minst_dgc_nccl

* fix typo

* fix test_dist_mnist_gradient_merge

* rm TestDistMnistNCCL2DGCMultiCards

* fix optimizer conflicts

* fix dist_mnist

* fix test_dist_hapi

* delete test_fleet_graph_execution_meta_optimizer & test_fleet_graph_executor

* temporally not delete unittest

* fix unittests

* fix ci

* recover prune in python/paddle/hapi/model.py

* move fluid.utils to paddle.utils.layers_utils

* fix error

* delete original fluid layers utils

* remove import and old utils

* remove more old utils import

* change import path of fill_constant in the layers_utils.py

* fix mistake

* fix error

* expose in __init__.py

* for comment

* when change the ref of func is_sequence, it should change to the root of is_sequence instead

* for codecheck

* Fix bug of reduce_sum op. When input.numel() > INT32_MAX, its result
is wrong.

* Remove climits.

* Fix bug of hybrid parallel strategy with recompute using bf16.

* Fix bug of recompute_hybrid ctx.amp_dtype

* Fix bug of amp_dtype.

* Fix bug of auto_cast.

* move io

* fix ci bugs

* fix ci bugs

* fix py3 bugs

* fix example code

* fix example code

* fix text

* fix text

* deal with ci bugs

* perfect code according comment

* delete import batch

* [Auto Parallel] Speedup the completion process

* [Auto Parallel] Skip the property of dist_context when deepcopying

* [Auto Parallel] Remove the unnecessary print

* find relevant testcase

* remove with_data_parallel in test_dist_base

* fix bugs

* fix test_minst_dgc_nccl

* fix typo

* fix test_dist_mnist_gradient_merge

* rm TestDistMnistNCCL2DGCMultiCards

* fix dist_mnist

* polish code

* fix different PaddleCloudRoleMaker

* fluid clean: remove parallel and parallel_helper api

* fix: fix the import path.

* fix DataParallel imports issue

* fluid clean: remove dygraph_utils._append_bias_in_dygraph

* fix func name and imports

* remove transpiler

* Revert "remove transpiler"

This reverts commit 46044ccd52011d45d7026786d331f264a6a8f645.

* Revert "Revert "remove transpiler""

This reverts commit 80ad0945401b5b5efebac4baee0ec50a793d4405.

* codestyle

* fix setup

* fix

* fix

* main code

* unitest bug

* revert cmake

* fix dist_attr in data_parallel in optimization

* fix grad_clip pass when pp2

* fix dist_attr