* [CustomDevice] add process_group_xccl ut

* update

* move average_accumulates op to phi kernel

Add yaml and unittest for bmm op

* sync misc changes

* add authors
Co-authored-by: NZhaorui Chen <zhaoruic@graphcore.ai>

* up x

* Revert "up x"

This reverts commit f3fde458c6cc48613269a643cfe2acf689caccd3.

* add guarg for ipu
Co-authored-by: NZhaorui Chen <zhaoruic@graphcore.ai>

* fix bug of elementwise_add_grad, *test=kunlun

* fix bug, *test=kunlun

* rm pooling_t, *test=kunlun

* fix bug of ew_add_grad when inplace, *test=kunlun

* fix new dygraph record event for op

* update unit test

* Add a feed op before each input parameter var.

* Fix some issues about the unit test build_cinn_pass_test.

* back fl

* delete ssl cert

* .

* make warning

* .

* unittest paral degree

* solve unittest

* heter & multi cloud commm ready

* .

* .

* fl-ps v1.0

* .

* support N + N mode

* .

* .

* .

* .

* delete print

* .

* .

* .

* .

* fix bug

* .

* .

* fl-ps with coordinator ready

* merge dev

* update message parse only

* update fl client scheduler

* fix bug

* update multithreads sync

* fix ci errors

* update role_maker.py

* update role_maker.py

* fix ci error: windows py import error

* fix ci error: windows py import error

* fix windows ci pylib import error

* add dump fields & params

* try to fix windows import fleet error

* fix ps FLAGS error

* fix behavior of device_id=None in Tensor.cuda

* fix CI

* Set more attrs in ReplaceScaleLossGradOp

* Fix typos

* Fix CI errors

* Add UT

* add reverse yaml

* add complex for einsum grad kernel

* pass the ci

* transfer the svd infer into phi infermeta

* remove the svd.h

* modify svd api

* fix svd error by insert optional

* multi stream support handle lazy init.

* support eigen lazy init

* update

* fix ci problem

* Move slogdeterminant op to phi

* Add yaml and unit test for slogdeterminant

* paralle_test_daily

* test=paralle_test_daily

* test=paralle_test_daily

* test=parallel_test_daily

* test=paralle_test_daily

* test=paralle_test_daily

* test=paralle_test_daily

* test=paralle_test_daily

* test=paralle_test_daily

* test=paralle_test_daily

* test=paralle_test_daily

* test=paralle_test_daily

* test pre_test_bak

* test cfs

* test_cfs,test=paralle_test_daily

* test_cfs,test=paralle_test_daily

* fix nightly test name,test=paralle_test_daily

* fix nightly test name,test=paralle_test_daily

* test ci parallel speed

* refine parallel rule,test=paralle_test_daily

* update comp cost

* add dist default op cost

* add dist fill constant batch size like op cost

* add elewise op cost

* add fill_constant_batch_size_like op cost unittest

* add unittest and remove fill_constant_batch_size_like grad op cost

* add to cmakelist

* fix unittest bug

* (modified) fc support fp16

* __CUDA_ARCH__ version

* delete half

* delete half

* add half support

* add half support

* add half support

* 2022-04-28

* 2022-04-28_V2

* 2022-04-30

* 2022-04-30_V2

* 2022-05-01

* 2022-05-02

* 2022-05-02_V2

* 2022-05-05_V1

* 2022-05-06_V1

* 2022-05-07_V1

* Update loss.py

* 2022-05-07_V2

* 2022-05-13_V1

* Update test_soft_margin_loss.py

* Update loss.py

* Update loss.py

* 2022-05-16_V1

* 2022-05-19_V1

* 2022-05-20_V1

* Update test_soft_margin_loss.py

* 2022-06-01_V1

* 2022-06-05

* 2022-06-07

* 2022-06-07

* 2022-06-08

* 2022-06-08_V2

* 2022-06-17-code_style

* Modify python

* 2022-06-20

* for

* for CI;test=document_fix
Co-authored-by: NLigoml <39876205+Ligoml@users.noreply.github.com>