* refine stride flag

* add spmd and reshard code gen

* add backward reshard code gen

* test matmul forward success

* polish test impl

* add unsafe mutable value

* polish details and add test

* fix unittest time out

* fix typo

* refactor reshard input generate impl

* resolve conflict with develop

* fix compile error

* fix bug

* fix bug

* support new ir primitive operator in static operants

* support more vjp code gen

* support more vjp code gen

* support more vjp code gen

* use code gen

* fix operants codegen

* support more vjp code gen

* Fix ci build error

* set FLAGS_tensor_operants_mode to static in generated_vjp for testing

* fix bugs

* change the order of ops_name of divide_grad

* replace FLAGS_enable_new_ir_in_executor by FLAGS_enable_new_ir_api in codegen and test_vjp_prim

---------
Co-authored-by: NCharles-hit <wanghao107@baidu.com>
Co-authored-by: N0x45f <wangzhen45@baidu.com>

* add meta tensor

* refine code

* fix bug

* fix bug

* [clang-tidy] replenish cppcoreguidelines-narrowing-conversions

* fix

* fix

* update

* fix bug

* fix bug

* fix bug

* fix bug

* rerun ci

* turn off shared_phi

* update

* update

* update

* update

* update

* [NewIR]Part-2.1 Refactor NewIRCompiler to support Group Ops

* fix gflags link error

* fix include ir_printer.h

* fix unittest

* fix conflict

* fix flags

* fix comment

[clang-tidy] No.34,36 enable performance-noexcept-move-constructor,modernize-use-transparent-functors (#56261)

* fix

* fix

* CI

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* CI

* fix

* CI

* fix custom device errro by dist

* polish details

* move matmul spmd rules into phi

* add basic infer spmd utils

* addspmd factory

* fix compile error

* add unittest

* refine infer spmd test and utils

* debug infer spmd test

* adapt python test

* poish details

* change to vector attr arg

* revert needless change

* update matmul spmd rule test

* remove original rule

* polish details

* fix marco error

* add comment

* pass backward test

* fix compile error

* add cmake rule for spmd_rules_test

* add dist meta tensor

* update pybind impl

* add marco for rules

* update

* repalce gflags header

* replace DEFINE_<type> with PD_DEFINE_<type>

* fix bug

* fix bug

* fix bug

* update cmake

* add :: before some paddle namespace

* fix link error

* fix CI-Py3

* allow commandline parse

* fix SetFlagsFromEnv

* fix bug

* fix bug

* fix CI-CINN

* fix CI-Coverage-build

* fix CI-Windows-build

* fix CI-Inference

* fix bug

* fix bug

* fix CI-CINN

* fix inference api test

* fix infer_ut test

* revert infer_ut gflags usage

* update

* fix inference

* remove flags export macro

* revert inference demo_ci gflags usage

* update

* update

* update

* update

* update

* update

* update

* update

* fix bug when turn on WITH_GFLAGS

* turn on WITH_GFLAGS

* fix bug when turn on WITH_GFLAGS

* fix bug when turn on WITH_GFLAGS

* update

* update and add unittest

* add unittest

* fix conflict

* rerun ci

* update

* resolve conflict

* for verify

fluid operator support new comm library

* u

* u

* u

* compatiable new comm library upgrade for c_allgather, c_reduce, c_reduce_scatter and c_scatter.

* Remove useless comments in process_group.py

* Polish code style.

* Fix some problems.

* Remove use fluid api in phi comm_context_manager.

* Add PPADDLE_WITH_CUDA and PADDLE_WITH_NCCL micro judgement.

* Fix bug of HIP architecture.

* Fix some problems.
1. remove useless loggings.
2. Fix conditional compilation for HIP.
3. Fix problems of test_pass_generation_pipeline.py. It calls paddle.distributed.init_parallel_env() at first,
then auto.Engine calls _init_comm(), which will calls process_group.instantiate(). However, init_parallel_env() will call
paddle.distributed.barrier(), it will call CreateNCCLEnvCache and create corresponding NCCLCommContext. But dev_id is not
set, as a result, NCCLCommContext's dev_ctx is not initialized.

* Fix some problems.

* Polish code.

* Polish code.

* Revert compatiable upgrade for communication operators. Their upgrades
will be submitted in another PR.

* Remove StaticTCPStore.

* Remove useless modification.

* Remove useless set_cuda_device_id.

* Polish code.

* Remove fluid header files in phi files.

* Remove useless comments.

* Fix problems of hip arch.

* Fix some problems.

* Polish code.

* Polish code style.

---------
Co-authored-by: hitywt <yuwentao126@126.com>

[clang-tidy] No. 53,54 enable cppcoreguidelines-c-copy-assignment-signature and bugprone-use-after-move (#56601)

【inplace api】Batch add inplace api gt_, ge_, lt_, le_, eq_, not_equal_, logical_and_, logical_or_, logical_xor_, logical_not_, divide_, floor_divide_, bitwise_and_ , bitwise_or_, bitwise_xor_, bitwise_not_ (#55509)

* tmp commit

* add atan2

* add inplace api

* fix error

* add inpalce divide

* add inplace api

* add more inplace

* add more inpalce

* fix logical_not error

* support sinh and cosh in cpu

* support asin, acos, atan, asinh, acosh, atanh in cpu

* fix typro

* fix typro

* mv out atan2 ldexp

* mv out atan2 ldexp

* support sinh and cosh in gpu

* support asin, acos, atan, asinh, acosh, atanh in gpu

* fix ge error

* fix dygraph commpare error

* fix dygraph commpare error

* check complex in python

* fix cast inpalce error

* open inplace test

* fix ops.yaml error

* mv cast inpalce to python

* fix coverage ci

* add last inplace

* fix inplace error

* fix cast error

* fix error

* add nan_to_num_

* fix typro

* fix sparse cast error

* remove gpu 4

* fix static cast error

* tmp commit

* add atan2

* add inplace api

* fix error

* add inpalce divide

* add inplace api

* add more inplace

* add more inpalce

* fix logical_not error

* fix typro

* fix typro

* mv out atan2 ldexp

* mv out atan2 ldexp

* fix ge error

* fix dygraph commpare error

* fix dygraph commpare error

* fix cast inpalce error

* open inplace test

* fix ops.yaml error

* mv cast inpalce to python

* fix coverage ci

* add last inplace

* fix inplace error

* fix cast error

* fix error

* add nan_to_num_

* fix typro

* fix sparse cast error

* remove gpu 4

* fix static cast error

* fix cast error

* fix

* Revert "check complex in python"

This reverts commit c822064261d774dd58ad46a4f90ba8b467700a05.

* add renorm , fix error

* add coverage

* fix cumsum inpalce version error

* add cast inpalce impl

* rm test.log

* fix multiply_dyfunction and add multiply_backward test

* add and use is_same_tensor

* fix typro

* fix sone error

* fix typro

---------
Co-authored-by: NScotty <jmhgchn@gmail.com>
Co-authored-by: NScotty <527407973@qq.com>

* simplify with dist marco

* polish error message format

* fix vtable error

* fix cmake error

* fix winsock redefined error

* fix windows compile error

* fix windows conpile failed

* fix merge error

* fix vec compile error

* add port.h into test_cpu_vec

* fix merge error

* try to fix winsock error

* support create shard dist tesnor

* support non-zero shard to replicated

* change reshard signature

* stride flag open

* add symbolicDimProduct & symbolicDimMgr without method shape_constraint related

* split ddim in phi, add a target ddim, used by pd_type

* add pd_type.cc to ir_shape CMakeLists

* polish dist teensor design

* adjust constructor

* polish details

* polish details design

* fix compile error

* refactor init tensor impl

* fix reshard test

* polish details

* add unittest for coverage

[Inference] Make share_external_data supports bf16 and bool; fix while_op cache_inference_while_scope when using fleet_executor. (#56055)

* 1. make share_external_data supports bf16 and bool; 2. don't drop_kids when cache_inference_while_scope

* fix FLAGS_cache_inference_while_scope

* add unitest

* add unitest

* skip unitest when cudnn_version < 8100

* skip test share_external_data_bf16 when CUDA_ARCH < 80

This reverts commit fd765f61.

* add phi forward api gen impl

* add phi backward gen code

* polish api code gen impl

* polish code gen impl

* remove auto_paralel namespace

* add dygraph forward impl

* add for_auto_parallel cond

* fix code gen errors

* add dygraph backward impl

* resolve conflict with develop

* refactor dist api gen impl

* revert origin api gen impl

* replace template for override func

* fix dnnl marco error

* revert third_party change

* add with distributed marco

* Update grad_tensor_holder.cc details

* merge dist tensor constructor

* change test tensor to replicate

* fx typo

* resolve conflict with develop

* fix out dim error