* move decayed_adagrad_op to phi

* fix bug

* Improve GC for pipeline parallel

* Delete print

* update

* update cache

* fix compile error

* new ir remove fetch list

* fix pattern rewrite bug

* try to remove constant fold

* revert code

* add pattern rewrite test flag

* fix multi fetch

* remove usless code

* new ir support legacy kernel instraction

* new ir support legacy kernel instruction

* add scope prefix

* update

* update

* update

* update

* fix

* revert channel shuffl test

* polish code

* try to fix windows compile error

* polish code

* update

* update

* revert op test

* Provide opoperands(), opresults() methods for the Operation module
Signed-off-by: Njjyaoao <jjyaoao@126.com>

* Update test_ir_pybind.py

---------
Signed-off-by: Njjyaoao <jjyaoao@126.com>

* fix new ir sgd op bug

* fix bug

* fix bug

* update

* revert code

* [prim][newir] add basic framework for primitive

* support desctensor in new ir

* add vjp interface

* support vjp in new ir

* support vjp in new ir

* polish vjp interface

* fix stop_gradients set

* fix vjp dispatch

* add comment

* add vjp test for new ir

* add test for tanh vjp

* [prim][newir] add basic framework for primitive

* support desctensor in new ir

* support vjp in new ir

* support vjp in new ir

* polish vjp interface

* fix stop_gradients set

* fix vjp dispatch

* add comment

* add vjp test for new ir

* add test for tanh vjp

* add eager and static backend for warp lower level api

* support call_vjp pybind

* polish code and add test for vjp

* remove useless code

* polish code

* remove useless code

* support mean vjp

* add test for mean vjp and support has_vjp function

* fix call_vjp

* polish code

* add primitive ops set for backend

* add vjp test for tanh_

* fix inference CI

* fix inference ci

* modify fluid cmake

* remove useless deps

* add cmake

---------
Co-authored-by: Ncxxly <chenxx_id@163.com>
Co-authored-by: Nzhangbo9674 <zhangbo54@baidu.com>

* add mask

* add backword

* add enforce info

* update scale

* integrate code

* update enforce

* add enforce eq

* add error type

* update enforce

* add test_flash_attention

* Polish codes and fix compiling errors.

* Set num_splits to 0 for flash-attn with tensor mask.

* Fix the compiling error for non flash-attn case.

---------
Co-authored-by: NLiu Yiqun <liuyiqun01@baidu.com>

* make tcp store a global instance

* fix windows compile error

* add attrs and dtype interface

* fix compile bugs

* fix some bugs

* fix windows bugs

* fix cudnn 8.7+ bug on cudnnConvolutionBiasActivationForward

* save_optimized_model_pass support tensorrt

* update

* update

* fix compile

* update

* fix ut timeout

* refactor codegen for cinn

* add to_string to some type which can't be += with string

* fix multi-thread bug caused by static var

* delete dead code and comment

* commit

* commit

---------
Co-authored-by: Nzhoukangkang <zhoukangkang@baidu.com>

* add paddle-symbolic-trace to paddle

* add symoblic trace

* delete swp

* support Layer in symbolic trace

* fix test-symbolic-trace, make symbolic trace return a StaticFunction

* template the error message

* fix some unittest

* Modify the execution mode of test

* Modify the module name

* add dy2static unittest decorator

* change some unittest files by @ast_only_test

* fix unittest.

* test-symbolic-trace

* update test_write_python_container.py

* update

* fix test_param_parse.py

* add submodule and ln -sf in cmakefile

* update

* update

* fix some ast only errors

* update

* Polish ut

* fix unittests

* update

* update

* fix unittests

* update

* test warning ast only

* update

* Ast only some uts

* Fix unitests

* test_error ast only

* update

* update

* Support build_strategy for sot

* update

* import sot as a third party module

* update

* update

* Polish code

* update

* update

* update

* update

* update

* remove old fluid api and use paddle.nn.relu instead

* fix

* comment the print of ast code

* add try-finally block

* fix dy2static stop-gradient bugs

* fix code

* remove unused submodule and minor codestyle fix

* fix

* fix cast error

* fix interpolate meets int64 in static model

* add evalframe support for py311

* fix

* fix err

* switch ENABLE_FALL_BACK=False

* fix

* Fix CI for some unittest

* add ENABLE_SOT

* remove setup.py dependences

---------
Co-authored-by: NNotHaozi <zhangmenghao@baidu.com>
Co-authored-by: Nfeifei-111 <2364819892@qq.com>
Co-authored-by: N0x45f <wangzhen45@baidu.com>
Co-authored-by: NSigureMo <sigure.qaq@gmail.com>

Update Save/Load Interface to 2.0

* sovle conflict bug

* fix bug

* Work for fa-2 padded fwd. Code to be cleaned.

* Work for fa2 unpadded fwd.

* Work for padded-bwd, dk get small diff on np.random.seed(0)

* Anyway I pass paddle's utest, except return softmax without dropout.

* Clean code.

* Modify interface.

* Clean code and add some check.

* Easy compile for dev.

* Fix ci.

* Fix ci-build.

* Add std c++17 option again.

* Limit max job when compiling fa2.

* Remove const_cast

* Add fwd params, to be cleaned.

* Clean code.

* Add bwd params.

* Clean code.

* Add enforce.

* Use v2.0.4

* Pass RNG state to fa2 capi

* Fix review.

* Add assert

* Skip compile for sm less than 80.

* filter new ir inplace var set parameter

* polish code

* fix conflict

* fix typo

* fix bug: feed_with_place should consider variable existence

* fix

* fix build scope

* change method to set feed var name

* remove feed_with_place to placeholder

* fix

* rename to data

* fix

* fix

* refacot aot

* update

* fix bugs

* remove some test

* fix bug

* fix bug

* fix bug

* fix bug

* update

1. `Parallel Compiler`：
    - 合并`FLAGS_cinn_parallel_compile_size`和`FLAGS_cinn_parallel_compile_thread`，通过`FLAGS_cinn_parallel_compile_thread`即可指定编译时使用的线程数，所有的`fusion_groups`将会平均分配到可用的线程上
    - 增强编译完成后返回的信息，除`instruction`外，将`lowered_function`、`source_code`、`source_ptx`返回，供上层进一步使用
2. Debug信息：
    - 新增`FLAGS_ cinn_dump_group_lowered_func`、`FLAGS_cinn_dump_group_source_code`、`FLAGS_ cinn_dump_group_ptx`、`FLAGS_ cinn_dump_group_instruction`，可分别按`fusion_groups`储存编译的每个阶段中的中间代码
    - 重新整理`graph_visualization`，所有的可视化图、单测代码均能正确分组储存
3. Bug修复：
    - 修复`MakeDirectory`不能正确创建文件夹的问题
4. 其他：
    - 清除了一些无用代码

* [clang-tidy] enable modernize-use-emplace

* Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into modernize_use_emplace

* add decorator for new_ir_test

* fix bug and only test in ci-coverage

* fix bug and only test in ci-coverage

* fix

* fix bugs

* fix

* fix

* WIP: start writing combined indexing get

* list/tuple/Variable

* getitem 80%

* add setitem

* add some unittest for setitem

* lazy import

* fix some setitem error

* fix advance indexing with decreasing axes; fix strided_slice input name

* combine int-tensor getitem is ok (without boolean support & broadcast); add getitem unittest for static

* add broadcast & parse bool tensor for __getitem

* [change getitem] _getitem_impl_ to _getitem_static, not deleting the former one

* refine new getitem; fix ut in variable/var_base

* add __getitem__ ut in dygraph

* re-dispatch getitem for Py/CPP; fix strided_slice decrease axes error in dygraph

* fix ut; support tensor in slice

* [change setitem] _setitem_impl_ to _setitem_static, not deleting the former one

* remove some UT (for some, temporarily)

* add IndexError to solve timeout problem in static-mode

* 1.temply forbideen all-False bool-indexput; 2.setitem_static will return new variable

* xpu uses old stratege

* rename dy2st setitem ut to avoid same-name problem

* dy2st for new combined index

* ut case for combine-index with dy2st

* open ut with all-false-bool setitem

* remove useless doc and _getitem_impl_

* change static res

* fix static xpu

* inplace pass support reshape2 and flatten_contiguous_range

* recover the modification to inplace_op_var_pass.cc