* proto type of composite grad in paddle

* proto type of composite grad in paddle

* refactor composite api with phi

* fix compile error

* support static graph code-gen for squeeze op

* generate static graph code of unsqueeze

* refine op name

* fix compile error

* add extra output in op_compat

* remove debug log

* fix clang compile error

* support prim switch flag

* support prim switch flag

* fix dygraph error

* merge develop

* add code_gen

* add necessary files without codegen

* fix code_gen bug

* add deps

* modify igmnore

* add ignore

* delete std cout

* add composite logic for backward.py

* add tanh first order grad composite

* support enable_prim flag for static graph

* throw expection when both GrapOpMaker and GradCompOpMaker not been registered

* reorganize the directory of prim api tests

* fix windows error

* add eager_utils

* add eager_utils

* modify code gen

* add composite parse

* add unittest for get_grad_op_desc

* code optimize

* fix static test on windows

* support generate static graph code for imag and real op

* fix windows compile error in test_static_prim

* merge develop

* disable test eager in inference

* prim code gen

* disable eager compile in inference

* rm other file

* rm gitignore file

* code_style

* add eager test

* code_style

* merge develop

* remove useless files

* modify static test

* support bool flag from singlton

* merge develop

* recover git ignore

* fix conflict

* recover git ignore for generated op

* fix test compile error

* remove some tests

* add python test

* fix some name issue

* add composite code gen

* modify backward yaml

* fix static composite grad maker code gen

* remove addtional files

* add some static funcs unit test

* fix some bugs

* fix composite grad maker register code gen

* optimize some functions
Co-authored-by: Nzyfncg <zhangyunfei07@baidu.com>
Co-authored-by: Nwangruting <wangruting@baidu.com>
Co-authored-by: Ncxxly <chenxx_id@163.com>
Co-authored-by: Ncharles-hit <wanghao107@baidu.com>
Co-authored-by: Nxiaoguoguo626807 <100397923+xiaoguoguo626807@users.noreply.github.com>

* [AutoParalle] balancing the calculation of global_norm in data parallel

* fix unittest

* update cond pure_data_parallel

* [Dy2static] Add ParameterRecorder

* filter by shape(tensor)==0

* fix code by review

* fix random failed in CI. (especially coverage)

* fix bugs

* remove API changes to avoid static CI approval

* comm_context and static init

* refactor: move to phi/core/distributed

* refactor: avoid mutable_data usage

* fix: windows sock

* fix: device without nccl
Co-authored-by: Wen Sun <syl1887415157@126.com>

* Enable standalone executor for fleet training

* Update code

* Replace use_standalone_executor utils in auto parallel

* Update code

* Diable standalone executor for test_pass_sharding

* Update code

* Set sequential run for auto parallel

* Fix dist_attr bug

* Set sequential run for auto parallel

* add 0-d support for paddle.kthvalue

* add 0-d support for paddle.mode

* fix coverage test for device

* fix check-bug in windows

* change axis check from LT to LE

* add shape & value check for grad when input is 0d tensor

* zero-tensor

* remove unused

* zero_dim_xpu

* relocate

* add value test

* fix syntax

* [D2SCinn]Add test_cinn unittest and param_grad into skip_gc_vars

* remove print

* acquire custom_deivce stream

* regulate file name and unittest

* flatten op support 0D-tensor

* add test in zero dim py

* fix shape should be list

* short code for ci-coverage

* add backward test

* simple code for ci coverage

* add axis check

* add 0D-tensor test in test_flatten_contiguous_range_op.py

* add axis error test for Coverage CI

* add more test for CI-Coverage

* add more test for CI-Coverage

* [Auto Parallel] Rename methods of ProcessMesh

* [Auto Parallel] Impl the python process_mesh by the c++ one

* [Auto Parallel] Add some minor modifications

* [Auto Parallel] Rename some methods

* [Auto Parallel] Remove unnecessary codes

* [Auto Parallel] Add back some removed files

* [Auto Parallel] Fix bugs

* [Auto Parallel] Fix a bug

* Update process_mesh.cc

* [Auto Parallel] Merge dist attrs of Python into C++

* [Auto Parallel] Add back deleted importing

* [Auto Parallel] Add back removed unittest

* [Auto Parallel] Remove type qualifiers of return types

* [Auto Parallel] Fix some bugs

* [Auto Parallel] Fix a bug of the quant pass

* [Auto Parallel] Fix the code style

sequence_mask fix: when the input length is an empty tensor, the kernel tries to dereference illegal sentinel iterator (#49525)

* support 0D for paddle.sort/argsort

* support 0D tensor for paddle.sort/argsort in xpu

* fix bug

* fix grad and add value assertion

* update nearest_interp, expand_v2, cast for stable diffusion

* update nearest_interp, expand_v2, cast for stable diffusion

* correct shape rank

* Update expand_v2_op.cc

* add pool flops

* add annotations and tests

* rm op_function_generator

* rm op_func_generator.h

* rm op_function

* modify cmake

* rm op_function.h

* rm check for op_function_generator.cc

* reset imperative

* rm python part

* fix imperative

* lint

* lint

* modify legacy_c

* review

* modify

* modify legacy

* rm gen op_functions code

* reset framework

* rm core.ops for test

* core.ops->core.eager.ops.legacy

* not raiseError for xpu

Add 0d Tensor Test Cases for cond, case, switch_case. Since the 3 APIs are control flow APIs, their support for 0d tensor relies on the underneath APIs. This PR just added test cases to prove that the 3 APIs have already handled 0d tensor well.

* remove deps and prior comm

* grad comm fuse

* add deps for amp&global norm

* stage2 broadcast prior deps

* stage2 grad overlap

* stream_analyzer bugfix

* overlap enable

* dep op namescope

* depend support multiple inputs

* check finite deps

* stage2 param comm overlap

* Set kD2HStream

* grad comm hierarchical

* grad comm hierarchical

* new unitest
Co-authored-by: Nchenruibiao <chenruibiao@baidu.com>

This reverts commit 2c444dfa.

* add for-else

* add * for unpacking

* Fix param and out grad names in dy2st for high order grad

* replace matmul with matmul_v2 in fuse passes

* Remove fusion logic from matmul

* removing fusion methods

* add proper name

* adjust namespaces

* move fc from fluid to paddle2.0

* fix unit test

* fix some examples

* fix some examples

* remove fluid.save_dygraph and fluid.load_dygraph use paddle.save and paddle.load instead

* move Sequential to paddle.nn

* modify convert_call_func.py Sequential reference

* remove related unitests

* remove fluid.dynamic.Sequntial

* test remove conver_call_func

* fix conflicts

* fix typro

* fix unitests

* fix sample_code

* fix unitest

* fix __init__

* Implement conv2d_fusion NHWC format using CUTLASS
* Add unit testing for CUTLASS Conv in inference
* Add experimental API for CUTLASS.

* [OpAttr]Fix Ignore AttriteTensor in IndicateDataType bug in grad_op

* add GetExpectedKernelType

* [Zero-Dim] reshape/reshape_/reverse 0D support

* rm comment

* change paddle.to_tensor to paddle.full

* fix docs

* update paddle.full

* relu flops all

* add annotations and tests

* revision for codestyle