* change logic for optimize

* modifty

* optimize the backward speed of EinsumOp

* add cache optimizer for einsum op

* EinsumOp: fix new dygraph mode error

* fix bug

* change Cache->InnerCache

* fix code

* fix

* add nan inf utils for einsum op

* add as_extra

* Compatible with v2.3 EinsumOp

* remove dispensable

* Dynamic graph support to Automatic SParsity.

1. Added dynamic support to ASP module (paddle.fluid.contrib.sparsity).
2. Added ASP related unit-tests regards to above changes.
3. Put ASP module under paddle.static for now, waiting for APIs confirmation from Paddle.

* Modified documents of functions to have correct examples.

* Update in_dygraph_mode to paddle.in_dynamic_mode()

* Modified documents of functions and added comments

* Minor changes.

* Fix example errors in asp API.

* Code Change for Review

1. Added more examples in documents.
2. Chaged test_asp_pruning_static.

* Minor changes

* Update ASP function documents.

* Update ASP function documents.

* Reduce test case size of asp pruning due CI time limit.

* Update time limitation to some asp UTs.

* Fix sample code errors.

* Fix sample code errors.

* Fix sample code errors.

* Update time limitation to parts of ASP UTs.

* Update UTs to fit with CI.

* Reduce problem size in python/paddle/fluid/tests/unittests/asp/test_fleet_with_asp_dynamic.py

* Added paddle.asp

* Fixed type casting error of OpRole.Optimize in new dygraph mode.

* Made set_excluded_layers be compatible with 2.2

* Fix example code of calculate_density.

* Update code examples.

* Move paddle.asp to paddle.incubate.asp

* Fixed an example error of calculate_density

* fix_sharding3_offload

* fix_fp16dtype_bug

* upgrade to sdk2.5.1

* add pad3d_double_grad yaml and test case

* add squeeze and unsqueeze double grad

* add double grad config

* add pad_grad and pad_double_grad yaml

* add pad_double_grad in config

* add assign_out_ yaml

* fix final_state_assign

* fix inplace bug

* add inplace_check_blacklist for assign

* fix merge conflict

* support to onnx test

* add comments

* remove log

* remove log

* update paddle2onnx version

* add double grad yaml

* fix bugs when compile infrt

* support inplace in backward

* fix final_state_linear

* fix format of backward_inplace_map

* little change

* add subtract in yaml

* fix hook mem leak

* fix hook use_count

* little format change

* fix
Co-authored-by: NJiabinYang <360788950@qq.com>

* add is_train into the cache key

* fix unittest error

* add unittest

* remove import

* remove storage

* add glog include

* add glog include

* add glog include

* Reduce test case for test_tensordot

* Fix CI errors

* remove is_init_py of RandomGenerator, and use Global Generator if not OP seed

* fix comment

* use fp32 compute type for cublasGemmStridedBatchedEx with fp16 input/output

* add flags to control compute type

* default to false

* add unit test

* default to true

* add tril_indices cpu kernal

* modify tril_indice cpu op

* modify bug

* modify bug

* add tril_indices python api

* add tril_indices python api

* resolve conflict

* add tril_indices test

* modify details

* add tril_indices.cu

* pythonapi pass

* save tril_indices

* CPU tril_indices pass

* delete vlog

* modify test_tril_indices_op.py

* delete tril_indices_kernel.cc.swp

* delete tril_indice.cu

* modify code style

* add newline in creation.py

* modify creation.py linux newline

* delete annotation

* check code style

* check .py style add final_state??

* modify code style

* add gpu_tril_indices

* modify gpu_compiled_juage

* modify gpu judge

* code style

* add test example

* modify english document

modify english document

modify english document

modify document

modify document

* modify pram name

* modify pram name

* modify pram

* reduce test ex

merge dymf branch

* added md support inside (de)quantizes

* added missing file

* changed paddle enforce text

* another paddle enforce change

* same as before

* removed broken tests

* change the output format of C++ backward api

* fix merge conflict

* fix sparse api code auto-gen

* fix eager_gen bug

* fix bug of output is null

* fix bug of conv2d_grad_impl

* fix optional grad

* fix bug of eager-gen double_grad

* fix bug

* fix multiply_double_grad bug

* fix bug of higher order derivative

* fix bug of FillZeroForEmptyGradInput

* remove redundant vector in grad_node

* fix bug of test_deformable_conv_v1_op

* fix bug of test_deformable_conv_v1_op

* some refacotr

test=document_fix

* auto parallel support primitive op with data parallel

* add primitive change

* 5 loss 3D cylinder acc aligned

* add unitest

* support yolov5s static/int8

* fix eltwise_sub and div weight compute

* fix delete_fill_constant_pass

* update readme test=document_fix

* fix api docs;test=document_fix

* update logic.py;test=document_fix

* update docs;test=document_fix

* slice data in dist_loader & flag to scale grad

* bug fix

* update unittest

* enable static

* [Dy2Stat]Modify all jit.save path into tempfile

* [Dy2Stat]Modify all jit.save path into tempfile

Add Code Generation for operators,  op makers and argument mapping functions (#41772)