* switch autotune

* implement AutoTuneCache

* implement AutoTuneCache class

* add pybind api

* add dygraph test

* support static mode and eager mode and improve unittests

* rename the SwitchAutoTune Class and improve tests

* improve AutoTuneStatus and reduce the cost of tests

* eager math op, test=develop

* eager support lookahead, test=develop

* refine,test=develop

* refine doc, test=develop

* refine,test =develop

* refie, test=develop

* refie, test=develop

* refie, test=develop

* test_paddle_multiprocessing

* refine, test=develop

* refine, test=develop

* fix bug, test=develop

* refine, test=develop

* dataloader, test=develop

* refine, test=develop

* refine, test=develop

* refine, test=develop

* test_datasets timeout, test=develop

* refine, test=develop

* [Eager] Fix empty tensor Initializer bug with shape=[]

* [Eager] Fix empty tensor Initializer bug with shape=[]

* ignore two unittest

* fix unittest

* enable new executor by default

* enable stream safe allocator

* test=document_fix;test=coverage

* do not use scope in op kernel

* fit empty program for new executor

* fix communication depend

* fix test_sync_batch_norm

* skip unsupported place

* refine datatransfer

* fit for dirtributed program

* fix dependencpy

* fix some ut

* add stack yaml

* add stack yaml

* add stack yaml

* add no_need_buffer

* refine no_need_buffer declare

* remove original grad infershape

* revert stack op

* support inplace no_need_buffer

* fix

* use padle.add

* eager test var base

* refine, test=develop

* update

* fix bug

* update name

* update name

* fix test

* fix fleet bind

* update name

* update name

* fix test

* fix gpups wrapper

* remove Push/Pull/Load/Save with context in client and wrapper base class

* fix

* fix

* remove some interface

* fix

* remove

* code style

* recover

* fix

* remove code unused

* fix

* recover

* fix
Co-authored-by: Nesythan <esythan@126.com>

* [Eager]Fix tile API final_state bug

* fix backward bug

* [Dygraph] Support sparse tensor in refactored reducer

* add uts

* refactor

* update

* fix bugs

* add infershape and forward yaml

* add final_state call

* add base unittests

* add backward yaml and test

* fix without softmax test error

* add cross_entropy test

* [Refactor] refactored eager_gen.py PR #2

* [DoubleGrad PR #1] Decoupled code generation logics for Dygraph ForwardFunctions and GradNodes

* Fixed minor issue

* Adjusted logics of GenerateNodeCreationCodes and GenerateForwardDefinition

* Fixed issues

* Supported higher-order grad node generation

* [DoubleGrad PR #4] Supported higher-order GradNode generation

* [DoubleGrad #4] Bug Fixes to Double Grad Node Generation

* Fixed yaml typo

* Fixed yaml typo

* fixed minor issues

* [DoubleGrad PR #5] Enabled gradient computations for grad_tensors passed to paddle.grad()

* Fixed minor issue

* Fixed CI-Inference issue

* Fixed CI-inference issues

* [DoubleGrad PR #7] paddle.grad() to copy backward graph before backward run

* Fixed minor issues

* Fixed issue with backward graph construction logic

* Fixed implementation issues with backward graph reconstruction

* Fixed unittest issue

* Fixed issues

* [Eager]Polish enable/disable_legacy_dygraph logic

* Support _offset in eager mode

* Update framework.py

* Update framework.py
Co-authored-by: NAurelius84 <zhangliujie@baidu.com>

* DRY

* change nodes names

* add const prefix

* change asX to as_x in all files

* add dropout slice yaml

* remove useless code

* fix infer shape error

* skip infrt compile for dropout

* add concat_grad kernel

* fix error

* remove comment code

* fix outs nullptr error

* change to phi header

* add concat_grad declare for standalone_executor_test

* add concat_grad yaml

* add concat api

* fix test concat op error

* fix test concat op error

* fix eager gen opti bug

* polish code

* fix some bug

* fix some bugs;

* [Eager] support tensor uva, test=windows_ci

* Add headers to fix CI, test=windows_ci

* Expose _uva python interface, Fix windows ci issue

* add add_n infermeta

* forward run success

* add add_n grad yaml

* support scale final state

* fix inplace error

* pass arg directly

* pass arg directly for inplace api

* fix type

* Add yaml for flatten_contiguous_range OP

* update

* Fix typos
Co-authored-by: NShixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* add concat_grad kernel

* fix error

* remove comment code

* fix outs nullptr error

* change to phi header

* add concat_grad declare for standalone_executor_test

* do not mutabledata when init, test=develop

* refine, test=develop

* fix copy_, test=develop

* refine, test=develop

* add some infer meta

* fix bug

* fix bugs;

* fix bug and add set data type

* revert infer shape of lookup table

* recover test

* limit grid dim for index select

* mv LimitGridDim into gpu_launch_config.h

* fix conflicts

* fix conflicts

* fix code style

* set block to 256

* fix grid setting

* set dtype of block_dim to unsigned int

* [Eager]Enhance eager_trace_op logic to support Optimizer Op

* fix AsDispensable

* [Eager]Fix 17 unittest and open check_eager=True

* remove print

* fix unittests

* fix op_testa

* fix coverage CI failed

* fix ci

* add randperm and range yaml

* add eager test for randperm

* Update ResNet test cases

* [Eager] Support uva, adamax, fill_diagonal_, to_list and so on.

* Fix CI

* Updated CUDA defined statement

* Fix CI

* Update headers, Fix CI

* Remove useless setting

* Updated func name to Fix windows-CI

* Remove tensor uva related codes

* Remove uva related code

* recover original test

* Add yaml config

* Add yaml for flatten_contiguous_range_op

* Remove h_sigmoid yaml

* Fix CI errors

* Fix code format

* Fix flatten OP errors

* Fix conflicts

* Fix CI errors

* Remove flatten_contiguous_range OP

* Remove redundant code

* Fix typos

* [Eager]Fix eager no take effect problem

* add element_wise and fix greater_than