* add DistributedFusedLamb op

* polish code

* fix compile error

* compatible with pten changement

* fix rocm compile error

* improve converage

* update upstream/develop

* fix cast_with_ptr.h

* add FLAGS_distributed_lamb_divide_nranks_when_allreduce=1

* fix clip before allreduce

* add use_master_param_norm

* code polish

* fix bug

* fix ROCM ci

* merge legacy to fluid

* Remove legacy code

* Remove legacy code

* Remove DataType test

* Using Tensor directly instead of using EagerTensor

* support gradient_accumulation

* make test_imperative_lod_tensor_to_selected_rows longer

* make test_imperative_lod_tensor_to_selected_rows longer

* refine code

* Rename all EagerTensor to Tensor

* Rename some EagerTensor to Tensor

* rename EagerTensor to EagerVariable

* add more test

* Support copiable selected rows and merge develop

* move blas related files
* move lapack related files

* add cinn_instruction_run_op test code

* update several interfaces of CinnLaunchContext

* update several interfaces and add detail comments in CinnLaunchContext class

* to skip the bug of error message check

* fix ut test failed due to reliant interface updated

* trans diagonal kernel into pten

* fix by code review

* support dtype param for auto_cast

* add amp_dtype for tracer

* add unsupported bf16 list

* support bf16 amp for O2

* refine python interface for bfloat16

* refine code

* refine code

* refine unittest

* refine code

* refine code

* add bf16 o1

* refine code by comment

* add gradient accumulator

* add recompute

* Update IpuStrategy

* fix ci

* rerun ci

* fix_sharding_group

* fix_sharding_group

* new way of unit test , *test=kunlun

* new way of ut, *test=kunlun

* the mlir representation of pten, test=develop

* fixes an error, test=develop

* infrt registers pten kernels
Co-authored-by: NShixiaowei02 <39303645+Shixiaowei02@users.noreply.github.com>

* support inplace and intermediate in yaml

* add cmake for dygraph_api

* add T to Alloc and remove default value of DataType in DeviceContext::Alloc

* add dtype

* add tool: print kernel signaturs

* fix windows compile

* [MLU]add matmul and matmul_v2 op

* [MLU] fix data_type and del matmul

* [MLU] fix compile error

* [MLU] fix ci_check error

* [MLU]add sync stream ops and broadcast pytest

* [MLU]fix broadcast pytest to add data type

* merge legacy to fluid

* Remove legacy code

* Remove legacy code

* Remove DataType test

* Using Tensor directly instead of using EagerTensor

* support gradient_accumulation

* make test_imperative_lod_tensor_to_selected_rows longer

* make test_imperative_lod_tensor_to_selected_rows longer

* refine code

* Rename all EagerTensor to Tensor

* Rename some EagerTensor to Tensor

* rename EagerTensor to EagerVariable

* add more test

* fix different device gradient_accmulator bug

* merge develop

* remove useless tests

* [Eager] Support GradientHook before running seperate GradNode

* Fix CI issue

* Fix CI issue

* pten matmul cuda kernel support bf16

* avoid custom kernel deps on pten_function_api

* Revert "pten matmul cuda kernel support bf16"

This reverts commit 5d520845b9a189375677276efb673235ed8e5ee0.

* refine code

* fix compile

* fix test_split_api

* move bernoulli kernel to pten

* follow comments

* relocate code of interpretercore gc

* fix and add more tests

* remove unwanted changes

* check only concat and elementwise

* move check to a function

* add todo comment

* Revert "fix ptq fc attr name fuse_activation->activation_type"

This reverts commit ffd023353a5e9b0fd15e81b9e9f9fe1794035017.

* save the name lists of the input,internal and output variables of a subgraph as its attribute

* fix compile error

* move trunc to pten

* modify according to YuanRisheng's comment

* clean useless header in pten core

* fix compiled failed

* fix cmake target

* fix typo

* resolve conflict

* add softplus op for kunlun2. test=kunlun

* add softplus op for kunlun2. test=kunlun

* fix code style. test=kunlun

* fix code style. test=kunlun

* add more test cases. test=kunlun

* first commit

* teller fix

* bug fix

* enable for pool2d only

* fix global_pooling issue

* pooling_type

* fix test

* remove register of matmul_v2 kernel

* delete matmul_v2 grad register in fluid