* add allreduce and broadcast without test (#31024)

add allreduce and broadcast without test

* Refactor HCCLCommContext to be compatible with Paddle (#31359)

Refactor HCCLCommContext to be compatible with Paddle (#31359)

* [NPU] add npu kernel for communication op (#31437)

* add allreduce and broadcast without test

* add c_broadcast_test case

* build c_comm_init and c_create_group operators

* make the whole thing compile

* add broadcast and init op test case but run failed

* make unit test compile

* fix broadcast test bug and change into hcom for ccl

* change c_comm_init and c_create_group ops accordingly

* make tests compile

* transfer code to 27

* compiled successfully in 28, but run failed

* test broadcast in 28, but failed

* make hcom primitives work

* change hccl data type for base.h

* fix broadcast bug

* make attributes work

* fix group name bug

* add allreduce but test failed

* allreduce bug for qiuliang

* allreduce finished

* add allgather and reducescatter

* merge all op code

* add allgather test

* finish run all ccl op test exclude send/recv

* all all op and test exclude send/recv

* send_v2_npu.cc recv_v2_npiu.cc compiled

* fix ccl core dump bug and test allgather, reducescatter, broadcast op

* fix allreduce bug just for test

* hcom send&recv test pass, without hcom_destroy

* for qiuliang test

* Ascend Send&Recv Test Pass

* all op (ex send/recv) ok

* fix bug

* merge all ccl op

* style merge to PaddlePaddle

* merge style

* new merge style

* merge style 2

* insert an empty at the end

* disable ctest for hcom to pass ci
Co-authored-by: Nvoid-main <voidmain1313113@gmail.com>
Co-authored-by: Nf2hkop <f2huestc@outlook.com>

* Add auto-increasing tag id for Hcom OPs (#31702)

* add c_reduce_sum op (#31793)

add c_reduce_sum op

* update Ascendrc hccl to 20.3 (#32126)

update Ascendrc hccl to 20.3 (#32126)

* fix merge code

* change cmake.txt1

* [NPU] Support npu kernel for c sync stream op (#31386)

* sync stream npu op

* add with_ascend_acl

* update c++ unittest

* compile all failed

* try to pre commit

* after pre commit

* merge&compile&test hccl successfully!

* fix code style

* fix code style

* fix bugs about hccl

* fix some bugs

* fix code style

* fix style

* fix style

* fix

* fixed

* merge develop
Co-authored-by: Nlw921014 <liuwei921014@yeah.net>
Co-authored-by: NVoid Main <voidmain1313113@gmail.com>
Co-authored-by: Nf2hkop <f2huestc@outlook.com>
Co-authored-by: Nxiayanming <41795079@qq.com>

* make hapi support amp, and add unittest

* make unittest only support GPU

* update parameters for amp in hapi.Model

* update hapi.Model.prepare interface, and update unittest

* fix test_model.py unittest bug

* add grad clear in dygraph

* use_fp16_guard defaults to True, which could avoid nan

* add input check, and add internal doc link to low level api

* update doc, and decrease the sample num of dataset to avoid timeout

* make hapi amp param  support str 'O1' or 'O2'

* resume calling , modify the code of the check part

* upgrade the usage of Fleet API, and disable 'pure_fp16' param

* fix warning and no grad

* add reducer

* refine envent for memorycopy

* add concat&split for allreduce

* apply concat & split for fuse tensor

* fix nccl dep

* fix the untest, compile problem and ddp initialize problem

* fix untest for mac & add some comments & solve the repeated param in sublayers

* fix untest for windows & fix document

* enhance logger callback for benchmark

* Fix doc format for callbacks, metrics and Model
* Fix code sample and doc

* add early stopping

* add doc for early stopping

* fix sample code bugs

* update infer of mode, update doc, add unittests to increase coverage rate

* fix sample code for early stopping

* update sample code and unittests

* reduce time cost of test_callbacks unittest

* fix model.py code style error

* update hapi save_inference_model output pathname

* update hapi save_inference_model output pathname

* use new 2.0-api paddle.static.io.load_inference_model

* add unittests to increase coverage rate

* add lr scheduler

* return pil image as default according backend

* fix lr scheduler

* fix code style

* update hapi predict interface

* fix code style

* fix docs

* fix docs

* fix docs

* update docs

* fix codes style

* fix unittest

* fix unittest

* fix coverage

* hapi/model step learning rate on batch end. test=develop

* step lr_scheduler on epoch end in hapi/model.py fit. test=develop

* fix model bugs, inputs can be InputSpec instance

* correct error message

* fix dynamic parallel for hapi

* fix code style

* remove input requirment in dygraph Model

* correct unittest

* upadte save inference model in dygraph without input

* fix unittets for test_model.py

* solve conflicts

* solve conflicts

* delete http.log

* fix test_model.py bug, correct initialization of MyModel

* fix unittests bugs

* set paddle manual seed for unittest

* fix Model bugs, because inputs can be list or dict when it is provided.

* add random seed for test_export_deploy_model

* delete redundant codes, because  calls

* Code optimization, error information optimization

* refine vision models

* reproduce summary api

* update model.save_inference_model

* update doc for _save_inference_model, delete useless class in unittests

* make users not be able to set model._inputs be None

* update usage of Model class in unittests

* fix bugs of _verify_spec

* fix bugs of _verify_spec

* add unittest to increase coverage rate

* delete http.log

* update doc for save, remove requirments and limitations for using

* update doc for class Model

Change ProgramTranslator.enable_declarative to ProgramTranslator.enable_to_static to meet API 2.0

* fix conv output_size has no default value bug
* fix summary bug

* add summary for hapi

* replace fluid.optimizer.set_dict with optimizer.set_state_dict

* replace fluid.optimizer.set_dict with optimizer.set_state_dict

* add coverage rate

* Increase coverage rate, fix code style

* Increase coverage rate, fix code style

* add fit to generate optimizer.state_dict() to save .pdopt to increase coverage rate

* delete http.log

* Move hapi form paddle/incubate to paddle

* Remove vision/datasets/utils.py and clean code

* Add sample code for conll05

* Print pull path when saving model

* Fix sample code after paramter_list of SGD is changed to parameters

* Fix bug in wmt16 datase

* fix sample code for hapi.model.save, test=document_fix

* test=document_fix

* update usage of 2.0 API, test=document_fix

* fix bugs, return dygraph back to users while using model.save in dygraph

* fix code style

* add InputSpec

* add unittest for tensorSpec and SimpleNet

* update save_inference_model for hapi

* update save_inference_model to support dygraph

* fix comments

* fix comments

* test=develop

* test, test=develop

* fix dim test, test=develop

* test, test=develop

* add test_export_deploy_model_dynamic

* fix unittest for hapi: save_inference_model

* fix code style

* accept review by guoshengCS

* fix coverage rate

* update doc for save_inference_model and copyright

* change test model back to LeNet() in test_export_deploy_model

* copy jit.save, use LeNet() to test export deploy model

* add return value for dygraph, and fix doc error

* corrected the doc writing

* Delete redundant import and correct import order in sample code.

* remove 'fluid' and add prepare() and fit() in sample code

* correct usage of API 2.0 in sample code

* fix sample code bugs

* fix code style bugs

* fix test_model.py bugs

* set for_inference=True

* correct usage for static.InputSpec

* update doc for model.save

* correct usage of API 2.0

* rename param name for model.save

* correct for_inference as training

* Move paddle/incubate/hapi/metrics to paddle/metric
* Add Precision, Recall and Auc metric

* Rename `Input` into `InputSpec`

* fix argument place of Input api

* Remove paddle.incubate.hapi.loss and reuse the paddle.nn.layer.loss in high level API

* Refine Model

1. Take the network (instance of Layer) as the input of Model.
2. Refine set_dict/load_dict of Layer.
3. Refine Input interface, so update code sample about Input

* add vision

* fix predict, test=develop

* add unittest for vision apis, test=develop

* fix typos

* add hapi models api, test=develop

* fix code format, test=develop

* fix typos, test=develop

* fix sample code import, test=develop

* fix sample codes, test=develop

* add decompress, test=develop

* rm darknet, test=develop

* rm debug code, test=develop

* Merge hapi into Paddle

Hapi is a high level API for training and inference.
The main modules include Model, Loss, Metrics, Dataset.
Also includes common modules and models in NLP and computer vision, such as BERT, ResNet.

These modules are developed by:
0YuanZhang0, guoshengCS heavengate, LielinJiang, qingqing01, xyzhou-puck huangjun12, wangxiao1021, zhangyang.