add allreduce and broadcast without test

* 添加rocm平台支持代码

* 修改一些问题

* 修改一些歧义并添加备注

* 修改代码格式

* 解决冲突后的代码修改

* 修改operators.cmake

* 修改格式

* 修正错误

* 统一接口

* 修改日期

* Add cholesky_op forward part. test=develop

* Complete cholesky_op forward part. test=develop

* Add cholesky_op backward part. test=develop

* Complete cholesky_op backward part. test=develop

* Refine cholesky_op error check and docs. test=develop

* Add grad_check unit test for cholesky_op. test=develop

* Fix sample code in cholesky doc. test=develop

* Refine some error messages of cholesky_op. test=develop

* Refine some error messages of cholesky_op. test=develop

* Remove unused input in cholesky_grad. test=develop

* Remove unused input in cholesky_grad. test=develop

* Fix stream for cusolverDnSetStream. test=develop

* Update PADDLE_ENFORCE_CUDA_SUCCESS from cholesky_op to adapt to latest code.
test=develop

* Add CUSOLVER ERROR in enforce.h
test=develop

* Fix the missing return value in cholesky. test=develop

cmake选项中添加了WITH_NCCL，显示指定是否编译NCCL的部分代码，WITH_NCCL默认打开，但如果WITH_GPU为OFF，则关闭WITH_NCCL

添加了PADDLE_WITH_NCCL定义

单机单卡能够关闭NCCL编译，多卡的话需要默认打开NCCL，如果关闭NCCL，则只能使用单卡
Co-authored-by: N石晓伟 <39303645+Shixiaowei02@users.noreply.github.com>

* Add the dynamic load of nvrtc, and support runtime compiling of CUDA kernel using nvrtc.
test=develop

* Call CUDA driver api to launch the kernel compiled by nvrtc.
test=develop

* Disable for mac and windows.
test=develop

* Refine the codes to support manually specified num_threads and workload_per_thread.
test=develop

* Refine the CUDA kernel to support large dims.
test=develop

* fix warpctc.dll not found issue, test=develop

* revert the linux platform change, test=develop

* delete warpctc_lib_path.h.in, test=develop

* add SetPySitePackagePath function

* fix warpctc.dylib not found issue on Mac, test=develop

* improve the paddle lib path setting logic, test=develop

* fix mac ci issue caused by test_warpctc_op unittest, test=develop

* tweak code, test=develop

Also fix a dependency error which may cause compile error

* remove unused FLAGS_warpctc_dir

test=develop

* remove FLAGS_warpctc_dir

test=develop

test=develop
This reverts commit c38c7c56.

* link the libwbaes.so into paddle

* polish detail, test=develop

* try fix mac_pr_ci error, test=develop

* add compile option, test=develop

* fix ci error, test=develop

* ignore failed to find mac lib, test=develop

* change cdn to bj, cdn can't get the latest version

* trigger ci, test=develop

* temporary delete win32 lib linking, test=develop

* change https to http, test=develop

* turn compile option on to off

* turn compile option off to on, test=develop

* try lib compiled by gcc4.8, test=develop

* update lib version, test=develop

* link other lib, test=develop

* add setup config

* delete false, test=develop

* delete no_soname, test=develop

* recover so name set

* fix, test=develop

* adjust make config, test=develop

* remove link to wbaes, test=develop

* remove useless define, test=develop

test=develop

* platform module

* Update profiler.h

* cudnn widndows

* "add comment"

* "windows support"

* "fix cmake error"

* Make paddle no longer depend on boost

* Update enforce.h

* Make CUPTI_LIB_PATH not passing by macro.
* Add missing header

* compile with nccl2

* add ncclGroup; it is necessary in nccl2

* add back libnccl-dev

* Add Seq2BatchFunctor, which will be used in WarpCTCOp.

* Implement WrapCTCFunctor and WrapCTCKernel.

* Add unittest of warpctc_op.

* Modify the check_output inferface in python unittest framework to allow check a subset of outputs.

* Use absolute offset lod in warpctc_op and related functors.

* Refine the comments of warpctc_op.

* The new python unittest supports checking a subset of the outputs, so revoke the previous change.

* Rename the transform from LoDTensor to Tensor with shape [max_sequence_length, num_sequences, sequence_width] to PaddingSequenceFunctor.

* Update to the newest codes.

* Rename the PaddingSequenceFunctor to PaddingLoDTensorFunctor and remove the computation of dimensions out of the functos.

* make enforce a target and dependent on nccl when gpu is enabled

* add some more dependency

* "add nccl enforce"

* Dev

* Update comment

* Add nccl test

* Follow comments