* update env setting

* update pass logic

* dist op support bf16

* backward cast update

* update setting

* update backward

* revert amp pass

* update fp16 backward logic

* register c_embedding bf16

* revert engine

* add unitest

* add unitest

* update unitest

* update cmake

* update math

* update math.py

* update unitest

* update unitest

* revise unitest

* revise unitest

* update unitest

* update unitest

* update unitest

* auto parallel align tool

* modify function get_var's return

* add save and load in align_tool

* modify load function and save function

* add finding different ops in align tool

* full auto parallel align tool

add test file for auto parallel align tool

set timeout for test

modify get_backward_tmp_var function

add annotation for align tool

modify test file

modify code to restart CI

remove timeout

* set timeout

* remove dygraph.parallel.ParallelEnv

* logger.py error: AttributeError: module 'paddle' has no attribute 'distributed'

* move the implenmentation to the root folder

* logger.py import ParallelEnv from paddle.parallel to avoid circular import

* add the comment of why import ParallelEnv from paddle.parallel in logger.py and remove the api interface in the paddle/parallel.py

* outdated Env and note removed

* decouple the logger.py and ParallelEnv

* remove another ref of parallel in init.py

* fix the processing order of passes in pass_base.py

* fix processing order

* add _PASS_PROCESS_ORDER_LIST

* delete some pass in _PASS_PROCESS_ORDER_LIST

* add assert in pass_base.py

* remove fuse_optimizer

* add _fusion_opt_list_rule

* add test_pass_base_list.py

* fix some bug

* add fused_attention

* add some passes to list

* fix ci bug

* fix ci bug

* fix bugs in cluster to device meshes

* add tests

* 1

* recovery annotation

* bugfix

* add FusedLinear pass

* add fused_op_list and renname PASSES to OP_FUSION

* add fused_passes_list to constants.py

* add test_passes.py

* fix test_fused_passes.py

* fix add if float(paddle.version.cuda()) >= 11.6:

* renamed test_fused_passes.py

* fix CMakeList.txt

* [Auto Parallel] Remove some fluid APIs

* [Auto Parallel] Fix the wrong import

* [Auto Parallel] Remove unnecessary comments

* [Auto Parallel] Fix the importing bug

* [AutoParalle] balancing the calculation of global_norm in data parallel

* fix unittest

* update cond pure_data_parallel

* Enable standalone executor for fleet training

* Update code

* Replace use_standalone_executor utils in auto parallel

* Update code

* Diable standalone executor for test_pass_sharding

* Update code

* Set sequential run for auto parallel

* Fix dist_attr bug

* Set sequential run for auto parallel

* [Auto Parallel] Rename methods of ProcessMesh

* [Auto Parallel] Impl the python process_mesh by the c++ one

* [Auto Parallel] Add some minor modifications

* [Auto Parallel] Rename some methods

* [Auto Parallel] Remove unnecessary codes

* [Auto Parallel] Add back some removed files

* [Auto Parallel] Fix bugs

* [Auto Parallel] Fix a bug

* Update process_mesh.cc

* [Auto Parallel] Merge dist attrs of Python into C++

* [Auto Parallel] Add back deleted importing

* [Auto Parallel] Add back removed unittest

* [Auto Parallel] Remove type qualifiers of return types

* [Auto Parallel] Fix some bugs

* [Auto Parallel] Fix a bug of the quant pass

* [Auto Parallel] Fix the code style

* remove deps and prior comm

* grad comm fuse

* add deps for amp&global norm

* stage2 broadcast prior deps

* stage2 grad overlap

* stream_analyzer bugfix

* overlap enable

* dep op namescope

* depend support multiple inputs

* check finite deps

* stage2 param comm overlap

* Set kD2HStream

* grad comm hierarchical

* grad comm hierarchical

* new unitest
Co-authored-by: Nchenruibiao <chenruibiao@baidu.com>

* auto parallel bf16

* [AutoParallel] adapt for clip

* fix unittest

* enable_static

* fix dist_fill_constant_batch_size_like

* fix process_mesh.shape

* update cond of modifying shape_list

* fix input order

* add unittest

* update cmakelist

* [AutoParallel] quantization pass support export

* support subgraph

* move_presist_var_to_global_block

* update unittest

* fix ci-coverage

* fix codestyle

* fix fake_dequantize_op

* remove unused var

* fix ci error and aprroval error

* add unittest for fp16 in test_dequant_linear

* replace mutable data

* fix unittest in non-cuda-core

* fix unittest
Co-authored-by: Ncarryyu <569782149@qq.com>
Co-authored-by: Nwufeisheng <wfs1997@163.com>

* [Auto Parallel] Rename methods of ProcessMesh

* [Auto Parallel] Impl the python process_mesh by the c++ one

* [Auto Parallel] Add some minor modifications

* [Auto Parallel] Rename some methods

* [Auto Parallel] Remove unnecessary codes

* [Auto Parallel] Add back some removed files

* [Auto Parallel] Fix bugs

* [Auto Parallel] Fix a bug

* Update process_mesh.cc

* [Auto Parallel] Fix a bug

* remove fluid.mul

* remove mul in __all__

* del array_write & array_read

* fix import err

* fix import err

* fix example codes

* [AutoParallel] recompute tuning

* fix conflict

* update comment

* bug fix

* update rc algo

* tiny fix

* fix clear process_group

* remove comment

* update segment print

* fix import OpRole

* adapt amp pass and grad_clip pass for opt_tuner

* update tuning config

* fix import

* annotate recompute info on ops and upgrade recompute pass

* add op_namescope for seed op

* record reserved vars

* fix recompute var's dist_attr

* fix strategy unittest

* adapt for fp16

* update unittest

* revert copy opt

* update unittest

* rename set_recompute_segments

* fix unittest

* add cluster_partition and device_meshes to process_meshes funcs

* add unitest

* Merge branch 'reduce_sum' of https://github.com/GhostScreaming/Paddle into mine_fluid_clean_common.

* Fix some bugs.

* Clean APIs in python/paddle/fluid/layers/control_flow.py

* Polish code style.

* Change API.

* Fix some bugs.

* Fix some bugs.

* remove layers.matmul in nets.py

* remove layers.matmul in rnn_impl/test_quantization_pass/auto_parallel_gpt_model/test_auto_parallel_completion_gpt

* remove layers.matmul in other files

* fix

* fix

* remove layers.matmul itself

* remove ref in CMakeLists.txt and tools directory

* remove matmul in fluid.layers.nn.py

* remove matmul in fluid.dygraph.rnn.py && resotre test_matmul_op.py

* replace matmul in fluid.dygraph.rnn.py && clean api_test in test_matmul_op.py

* fix error && restore empty test_auto_search_dist_matmul_op.py

* fix check in test_auto_parallel_partitioner.py

* fix test_dist_matmul && test_flags_mkldnn_ops_on_off

* fix test_fused_attention_op_xpu.py && test_matmul_op_xpu.py

* remove test_auto_search_dist_matmul_op.py

* remove layers.matmul in auto_parallel_gpt_model.py && fix doc in fluid/io.py

* fix for matmul_grad

* fix codestyle

* fix codestyle

* resolve conflicts error

* restore unit test file but not compiled it for later remove

* fix codestyle

* fix wrong unittest skip

* fix unittest delete

* fix scale cost

* fix scale cost

* resolve conflicts error

* resolve conflicts error
Co-authored-by: Njakpiase <jakpia21@gmail.com>

move paddle.fluid.layers.tensor.create_parameter to paddle.tensor.creation.create_parameter (#48579)

* add pattern match

* add unittest

* add pattern for auto search

* add unittest

* add isort config

* isort all files

* remove transpose

* codestyle check

* modify CI_STATIC

* modify CI_STATIC

* modify enable static()

* remove unused import

* fix conflict about stack

* fix conflict about stack

* fix conflict about stack

* endless conflict

* [AutoParallel] selective recompute

* add cmakelist