* prepare only once

* support CUDAGraph for partial graph

* add ut

* fix ci

* fix ut again because of eager mode

* fix kunlun ci

* fix win ci

* add yaml

* fix infrt compile bugs

This reverts commit e680d581.

* Update random.py

* test=document_fix

* test=document_fix

* Update random.py

* support weight transpose

* add ut

* add template

* fix transpose error

* fix transpose_comment

* add api tests

* add skipif

* add doc

* adapt for 10 loss

* partitioner support optimizer

* update xpu cmake: xdnn 0527. test=kunlun

* update to xdnn 0531.

* update to xdnn 0531. test=kunlun

* update to xdnn 0601. test=kunlun

unittest parallel
Co-authored-by: Nzhangbo9674 <zhangbo54@baidu.com>

* Add pinned memory to HostMemoryStats

* Add macro for WrapStatAllocator

* Fix CI errors

* fix the bug of adamw which set the attribute in param group not working

* fix undefined variable

* fix api example typo

* add unittest

* fix unittest typo

* [Auto Parallel] Add the parallel tuner

* [Auto Parallel] Improve the parallel tuner and fix some bugs

* upodate cost model

* update import Resharder by dist op

* update cost model

* fix comp cost bug

* update cost model

* [Auto Parallel] Amend the dist attr for #processses=1

* update cost model and tuner

* update cost model and tuner

* update cost model and tuner

* update cluster

* update reshard

* [Auto Parallel] Add the estimation from the cost model

* [Auto Parallel] Reimplement the backup and restore functions

* [Auto Parallel] Fix the bugs of the parallel tuner

* [Auto Parallel] Update the engine api and dist context

* [Auto Parallel] Work around the high order grad problem

* [Auto Parallel] Add some miscellaneous improvements

* [Auto Parallel] Add a unittest for DistributedContext
Co-authored-by: Ncaozhou <caozhou@radi.ac.cn>

* add conv3d yaml

* add conv3d_grad, conv3d_double_grad

* add final_state_conv3d test case

* add conv3d double test case

* add depthwise_conv2d grad yaml

* add depthwise_conv2d double grad test case

* modify the order of args

* add depthwise_conv2d_grad_grad config

* remove attrs from base op

* fix typos

* remove brelu

* undo removing code related to matmul

* remove whitespaces

* undo changes in matmul

* remove empty line

* add double_grad and triple_grad inplace info in backward.yaml

* only generate inplace api in forward

* fix full zero

* fix full zero

* fix full zero

* fix full zero

* refine

* refine

* refine

* fix assign kernel copy impl

* fix test failed

* [IPU] support paddle.distributed.launch with IPUs

* add device_num to env_args_mapping

* update RandomCrop class code annotation; test=document_fix

* update adjust_brightness api in functional.py test=document_fix

* udpate uniform api in random.py

* update transforms.py

* polish append op using

* fix var error

* fix group norm impl

* fix arg_max and reduce_max

* add arg_max ut

* rrelu逻辑部分

* unregistered op kernel (unresolved)

* commit before merge

* 丰富测试用例

* 修复rrelu-sig的bug

* 修复cpu环境测试

* 修改拼写错误

* 修改code format

* 尝试优化测试用例timeout的问题

* 优化测试用例

* 移除seed, 优化随机函数

* update en doc for rrelu

* fix rrelu en docs, test=document_fix

* add paper link for en docs, test=document_fix

* udpate en doc

* add r,test=document_fix

* change einsum_v2 as default and add new flags: FLAG_einsum_opt=1|0

* make EInsumOP support bf16

* add unittest for BF16

* add condition for test_BF16

* fix bugs

* fix