* Add elementwise_add support into NHWC IR

* update

* fix batch norm grad args def

* fix bug

* fix combine slice bug

* fix slice bug

* update builtin split

* disable using kernel resigter dtype

* polish code

* disable some test

* move matmul spmd rules into phi

* add basic infer spmd utils

* addspmd factory

* fix compile error

* add unittest

* refine infer spmd test and utils

* debug infer spmd test

* adapt python test

* poish details

* change to vector attr arg

* revert needless change

* update matmul spmd rule test

* remove original rule

* polish details

* fix marco error

* add comment

* pass backward test

* fix compile error

* add cmake rule for spmd_rules_test

* add dist meta tensor

* update pybind impl

* add marco for rules

* update

* repalce gflags header

* replace DEFINE_<type> with PD_DEFINE_<type>

* fix bug

* fix bug

* fix bug

* update cmake

* add :: before some paddle namespace

* fix link error

* fix CI-Py3

* allow commandline parse

* fix SetFlagsFromEnv

* fix bug

* fix bug

* fix CI-CINN

* fix CI-Coverage-build

* fix CI-Windows-build

* fix CI-Inference

* fix bug

* fix bug

* fix CI-CINN

* fix inference api test

* fix infer_ut test

* revert infer_ut gflags usage

* update

* fix inference

* remove flags export macro

* revert inference demo_ci gflags usage

* update

* update

* update

* update

* update

* update

* update

* update

* fix bug when turn on WITH_GFLAGS

* turn on WITH_GFLAGS

* fix bug when turn on WITH_GFLAGS

* fix bug when turn on WITH_GFLAGS

* update

* update and add unittest

* add unittest

* fix conflict

* rerun ci

* update

* resolve conflict

* add_arg_mapping_for_fetch

* fix

* fix

* for verify

fluid operator support new comm library

* u

* u

* u

* compatiable new comm library upgrade for c_allgather, c_reduce, c_reduce_scatter and c_scatter.

* Remove useless comments in process_group.py

* Polish code style.

* Fix some problems.

* Remove use fluid api in phi comm_context_manager.

* Add PPADDLE_WITH_CUDA and PADDLE_WITH_NCCL micro judgement.

* Fix bug of HIP architecture.

* Fix some problems.
1. remove useless loggings.
2. Fix conditional compilation for HIP.
3. Fix problems of test_pass_generation_pipeline.py. It calls paddle.distributed.init_parallel_env() at first,
then auto.Engine calls _init_comm(), which will calls process_group.instantiate(). However, init_parallel_env() will call
paddle.distributed.barrier(), it will call CreateNCCLEnvCache and create corresponding NCCLCommContext. But dev_id is not
set, as a result, NCCLCommContext's dev_ctx is not initialized.

* Fix some problems.

* Polish code.

* Polish code.

* Revert compatiable upgrade for communication operators. Their upgrades
will be submitted in another PR.

* Remove StaticTCPStore.

* Remove useless modification.

* Remove useless set_cuda_device_id.

* Polish code.

* Remove fluid header files in phi files.

* Remove useless comments.

* Fix problems of hip arch.

* Fix some problems.

* Polish code.

* Polish code style.

---------
Co-authored-by: hitywt <yuwentao126@126.com>

* Register LegacyKernelDialect & Rigister LegacyKernelOp

* fix code style

* delete LegacyKernelDialect ,register LegacyKernelOp into PaddleKernelDialect

* fix bug

* change as reviewed comments

* bug fix

* bug fix

* try to restart coverage CI

* pass legacy op to kernel pass

* fix code style

* fix code style

* fix code style

* [clang-tidy] enable `hicpp-exception-baseclass` and fix existing errors

* config

* update error format to pass the ci check (at least 20 chars)

* [AutoParallel][NewIR] support calc_sync/comm_sync/send_v2/recv_v2

* pre-commit

* rm unittest

* tiny fix

* api_gen support send_v2's output is empty

* fix format

* python_c_gen support send_v2

When using paddle2cinn, CompilationContext.with_instantiate_variables should be set to false, otherwise CINN will instant and manage variables memory, this leads to double the memory usage, which eventually leads to out of memory error.
This PR will set CompilationContext.with_instantiate_variables to false before context pass to constructing the graph compiler.

* add vjp code gen for SplitOp

* change vjp manual file name

* add pass registry

* add pass registry macro

* remove flag

* open static build flag

* add searchsorted to list

* add register info for fused layernorm

* fix fused_layernorm_kernel output registe info

* fix stft registe info

* add include

* fix registe info

* add skip fake init for fused_layernorm:residual_out

* fix error

* add distributed_fused_lamb_init to StaticBuildBlackList

* set static_build flag to false

* [Fluid] move lars_momentum to phi

* add sig

* fix optional Output

* off check_dygraph

* fix input

* fix operator[]

* fix

* try fix AllocateTmpTensor

* fix

* fix type

* Update paddle/phi/kernels/gpu/lars_momentum_kernel.cu

* fix type

* rollback

* Add Registration

* try fix win

* try fix win

* try use double

* try use operator *(float,const Derived &)

* try auto

* fix

* fix

* fix

* fix dtype

* fix type

* fix index

This reverts commit 1409e4ec.

* fix

* fix

[clang-tidy] No. 53,54 enable cppcoreguidelines-c-copy-assignment-signature and bugprone-use-after-move (#56601)

* update

* fix batch norm grad args def

* fix bug

* fix combine slice bug

* fix slice bug

* update builtin split

【inplace api】Batch add inplace api gt_, ge_, lt_, le_, eq_, not_equal_, logical_and_, logical_or_, logical_xor_, logical_not_, divide_, floor_divide_, bitwise_and_ , bitwise_or_, bitwise_xor_, bitwise_not_ (#55509)

* tmp commit

* add atan2

* add inplace api

* fix error

* add inpalce divide

* add inplace api

* add more inplace

* add more inpalce

* fix logical_not error

* support sinh and cosh in cpu

* support asin, acos, atan, asinh, acosh, atanh in cpu

* fix typro

* fix typro

* mv out atan2 ldexp

* mv out atan2 ldexp

* support sinh and cosh in gpu

* support asin, acos, atan, asinh, acosh, atanh in gpu

* fix ge error

* fix dygraph commpare error

* fix dygraph commpare error

* check complex in python

* fix cast inpalce error

* open inplace test

* fix ops.yaml error

* mv cast inpalce to python

* fix coverage ci

* add last inplace

* fix inplace error

* fix cast error

* fix error

* add nan_to_num_

* fix typro

* fix sparse cast error

* remove gpu 4

* fix static cast error

* tmp commit

* add atan2

* add inplace api

* fix error

* add inpalce divide

* add inplace api

* add more inplace

* add more inpalce

* fix logical_not error

* fix typro

* fix typro

* mv out atan2 ldexp

* mv out atan2 ldexp

* fix ge error

* fix dygraph commpare error

* fix dygraph commpare error

* fix cast inpalce error

* open inplace test

* fix ops.yaml error

* mv cast inpalce to python

* fix coverage ci

* add last inplace

* fix inplace error

* fix cast error

* fix error

* add nan_to_num_

* fix typro

* fix sparse cast error

* remove gpu 4

* fix static cast error

* fix cast error

* fix

* Revert "check complex in python"

This reverts commit c822064261d774dd58ad46a4f90ba8b467700a05.

* add renorm , fix error

* add coverage

* fix cumsum inpalce version error

* add cast inpalce impl

* rm test.log

* fix multiply_dyfunction and add multiply_backward test

* add and use is_same_tensor

* fix typro

* fix sone error

* fix typro

---------
Co-authored-by: NScotty <jmhgchn@gmail.com>
Co-authored-by: NScotty <527407973@qq.com>

Add dimOp, tieProductEqualOp. access constraint_func in SymbolTable. Lowing DenseTensorType. (#56615)

* add symbolicDimProduct & symbolicDimMgr without method shape_constraint related.

* add pd_type.cc to ir_shape CMakeLists.

* add dimOp, tieProductEqualOp. access constraint_func in SymbolTable.

* put DenseTensorType into builtin_type.

* register set_value in new ir

* fix

* register set_value_grad

* fix

* fix

* remove debug info

* add unittest

* fix

* fix

* fix

* fix

* fix

* resolve comments

* simplify with dist marco

* polish error message format

* fix vtable error

* fix cmake error

* fix winsock redefined error

* fix windows compile error

* fix windows conpile failed

* fix merge error

* fix vec compile error

* add port.h into test_cpu_vec

* fix merge error

* try to fix winsock error

* move shuffle_batch to phi

* remove useless codes

* add test_shuffle_batch_op to STATIC_BUILD_TESTS

* move shuffle_batch_kernel.cc to cpu folder

* move shuffle_batch_grad to phi

* rm shuffle_batch_op.h

* change year at file head

* support ir api form prim

* convert vector of int to intarray

* add reference of lbfgs

* add reference of lbfgs

* support ir api for prim

* Add more gen api

* concat python api to concat_grad

* fix gen conflict

* support vjp prim mode in new ir

* remove useless code

* add vjp autogen v1.0

* add test for prim

* resolve type conflict

* modify utils

* remove useless code

* add split op and modify some bug of vectorType

* fix conflict

* add concat python test

* add split python api to vjp

* modify build bug

* modify run bug

* fix conflict bug

* build bug fix

* modify python api bug

* modify test

* fix conflict

* fluid backward recover

* recover conflict

* reply review comments

* modify opruntimeinfo num

---------
Co-authored-by: Ncyber-pioneer <chenzhuo@tju.edu.cn>
Co-authored-by: NCharles-hit <wanghao107@baidu.com>
Co-authored-by: N0x45f <wangzhen45@baidu.com>
Co-authored-by: Nchenzhiyang <1792266893@qq.com>
Co-authored-by: NChen Zhiyang <chenzhiyang99@126.com>

* gen-temp-save

* add concat vjp

* remove useless print

* code style

* remove manual concat vjp

* add embedding backward rule

* update backward api

* revert api

* matmul inferbackward

* update unitest

* support create shard dist tesnor

* support non-zero shard to replicated

* change reshard signature

* support new ir load combine

* update

* polish code

* remove print

* update

* update

* update

* polish code

* fix bug

* polish code

* fix compile bug

* fix bug

* revert code

* remove useless code

* polish code