* support python39 and delete python35

* support python39 in Dockerfile.centos

* fix ubuntu18 bug

* update Dockerfile.ubuntu setuptools

* fix centos py39 errors

* fix centos py39 error2

* support save/load binary format tensor

* Fix error when create cudaplace

* Fix error when create cudaplace

* Fix error when create cudaplace

* get devive context from pool.

* move define of 'SerializeToStream' and 'DeserializeFromStream' to 'lod_tensor.cc' and 'selected_rows.cc'.

* support complex object

* improve coverage.

* improve coverage

* improve coverage.

* fix a bug.

* polish API

* save/load program

* paddle.save/load: layer

* deal with conflict

* if PY2, block test_paddle_save_load.TestSaveLoadLayer

* polish code.

* polish code

* edit unnittest

* The condition for object to be identified as state_dict becomes strict

* use 'core._cuda_synchronize'

* print the real name for Functions instead of the ArgSpec

class function method

* some API's name is not __module__ + __name__

so, we disard them temporarily.

* update the logging format for console

* ommit the top level of paddle package.

* these APIs have been removed.

test=document_fix

* Another Error occerd

* print_signatures.py 's stdout is redirect to spec file, so should not print any other info.

so sad.

* print the error msg to stderr

* disable the __init__ magic method

* update unittest for sampcd_processor.py

update unittest for sampcd_processor.py

* PR-CI-APPROVAL 's python interpreter name is not 'python3'.

it's a python3.9;
it does not have paddle installed yet.

此句在CI流水线竟然不可以用。报python3找不到
此句在CI流水线竟然不可以用。因为环境没有安装paddle

* testing only extract api from __all__

paddle module(the top module) does not have __add__
test=document_fix

* should import paddle here

* update the mechanism of extractiong and executing for the sample-codes test.

更新抽取代码和执行代码的逻辑
优化输出打印

* good code style

* remove ut from parallel_ut_rule caused by timeout

* remove timeout ut from parallel_ut_rule file

* move convert_model2dot_ernie to TWO_PARALLEL_JOB list

* fix Windows CI MP compile and environment install script

* clear Windows CI environment

* clear Windows CI environment

* clear Windows CI environment

* add retry on gcda_clean.py

* add exit code for paddle_coverage.sh

* fix format error

* fix format error

* [NPU] support GarbageCollector for npu (#31874)

* support GarbageCollector for npu

* fix typo

* fix gather_grad

* disable NPUDefaultStreamGarbageCollector on NPU

* [NPU] support npu for memcpy op (#31808)

* support npu for memcpy op

* add ut

* fix ut

* fix typo

* 【NPU】fix bug of using temp vector (#31963)

* fix bug when beta1_pow on cpu (#31995)

* [NPU] support npu profiler (#31684)

* support npu profiler

* add python api

* fix bugs

* add wrapper for incomplete type

* update profile proto

* record npu wait

* add xpu placeholder

* fix adam (#32016)

* [NPU] enable async copy and  add wait before sync operation (#31956)

* enable async copy and  add wait before sync operation

* remove unneccessary wait

* add FillNpuTensorWithConstant

* refine

* fix fill_constant

* make TensorFromVector/TensorToVector sync

* [NPU] Support dataloader on npu place. (#31867)

* [NPU] Wait on NPUPlace (#32086)

* [NPU] fix cast op (#32121)

* fix npu kernel of cast op to handle casting to same dtype

* add comments

* [NPU] support cann 20.3 (#32044)

* fix compile problem on cann 20.3

* fix ut

* fix test_mul

* fix check_finite_and_scale

* fix lookup_table_v2_grad

* fix cmake

* support print op

* [NPU] Support npu save load (#31893)

* support save load for NPU

* add save load npu unittest

* support np.array transform in NPU

* fix errors

* delete dygraph in unittest

* add Wait

* fix unittest

* fix review comment

* fix unittest problem

* fix little problem

* change aclrtSynchronizeDevice to aclrtSynchronizeStream for better performance (#32196)

* change aclrtSynchronizeDevice to aclrtSynchronizeStream for better performace

* refine code

* fix NPUDeviceContext in all c++ unittest (#32198)

* fix NPUDeviceContext in all c++ unittest

* refine log
Co-authored-by: Npangyoki <pangyoki@126.com>

* [NPU] Remove TensorFromVector and avoid sync copy in npu op kernel for better performance (#31994)

* enable async copy and  add wait before sync operation

* remove unneccessary wait

* add FillNpuTensorWithConstant

* refine

* fix fill_constant

* change TensorFromVector to FillNpuTensorWithConstant

* fix ignored api

* delete extra unittest

* fix little error

* fix update_loss_scaling_op_npu and check_finite_and_unscale_op_npu

* change TensorCopySync to TensorCopy

* delete useless Wait and add StreamWait

* fix npu_stream error

* fix check_finite_and_unscale_op_npu TensorCopy

* only save stream wait

* fix NPUDeviceContext in all c++ unittest

* delete wait
Co-authored-by: Nzhiqiu <chenqiuliang@baidu.com>

* delete useless unittest file (#32206)

* Fix op test (#32231)

* fix conditional block (#32243)

* fix adam bug again (#32246)

* fix compile

* fix ut

* fix ut
Co-authored-by: Nliym27 <33742067+liym27@users.noreply.github.com>
Co-authored-by: Npangyoki <pangyoki@126.com>

* get_api_md5 should prefer use the real name rather than the alias names

* case for ArgSpec style. update the unittests

test=document_fix

* Initial draft for SGD BG16 kernel.

* Unit tests for SGD with BF16 data type.

* Add VLOG message to SGD BF16 op CPU kernel.

* Enhance error messages and error types.

* Refactor SGD op kernels to leverage some common code.

* Make easier to add new kerne invoke code.

* Fix SGD op kernel for sparse grad.

* Unify quotes style.

* Fix error for ROCM compilation.

* Use specialized PADDLE_ENFORCE_xx functions.

* fix error for long args

* remove unneccessary code

* skip paddle.Tensor.<lambda>

* some file may not exists. such as version.py, it's generated by setup.py

* debug mode

* add unittests for sampcd_processor.py

* add test cases for sampcd_processor

* add test cases for sampcd_processor

* add testcases

* add test cases

* add testcases

* add testcases

* refactor, add testcases

* add import

* all files map to pool. dont split manually

* __all__ += another list

* add testcases

* add testcases

* handle个锤子啊

* this line should not removed

https://github.com/wadefelix/Paddle/commit/882e7f7c3be6c2415f58550f82be338b84f0c0ef#diff-cb0679475bf60202fd803ae05b9146989437c3f787d1502616be6c71c69d0fb1

* print -> logger

* regulate the logging infomation

* regulate the logging infomation

* logger to file

* logger

* threads or subprocesses number config

* follow the good code style

don't touch wlist.json

* run test_sampcd_processor.py, it's a unittest for sampcd_processor.py

* update unittest for sampcd_processor.py

test=document_fix

* test,test,notest,test=windows_ci

* test,notest,test=windows_ci

* test,notest,test=windows_ci

* test,notest,test=windows_ci

* remove test code

* delete some unnecessary logs

* fix format error

* turn on added ut check on windows

* added ut check on windows,notest,test=windows_ci

* debug,notest,test=windows_ci

* debug,notest,test=windows_ci

* fix bug,notest,test=windows_ci

* added ut check

* test for new ut add on windows

* test,notest,test=windows_ci

* fix bug,notest,test=windows_ci

* test

* test

* test

* test,notest,test=windows_ci

* test,notest,test=windows_ci

* check added ut on windows

* only fetch upstream develop

* modified according comment

* Update run_unittests.sh

* Update run_unittests.sh

* update develop whl package name

* distingush cpu and gpu name

* fix ref_gcc

* change whl name

* upgrade gcc 4.8 to 5.4 in ubuntu_dev

* update gcc4.8 to 5.4 in centos

* Upgrade pip from 18.0 to 20.0.1

* change 2.1.0_dev0 to 2.1.0.dev0 in gpu version

* use busybox run test on windows openblas

* fix error

* fix disable_quick and nightly lable issue

* add retry on windows openblas

* fix bug

* use one file to run cpu and gpu tests

* fix with grep warning

* fix syntax error

* change run_unittest to run_unittest_gpu

* Update run_unittests.sh

fix error

* support save/load single tensor

* compatibility modification according to unnittest

* Some python2.7 don't have 'copyreg' modules

* Handle a syntax error.

* Dealing with compatibility problems on Mac.

* Dealing with compatibility problems on Mac.

* edit unittest to improve coverage.

* Modify the code according to the review comments

* Reduce redundant code.

* support for static graph loading dygraph state_dict

* edit code according to CI

* edit unittest

* edit unnittest

* delete redundant file

* edit code according to Comments

* edit english doc

* edit english doc

* edit English DOC.

* get/set_tensor->get/set_value; return_numpy=False

* get/set_tensor->get/set_value; return_numpy=False

* edit unnittest

* edit unnittest

* polish code.

* graph engine demo

* upload unsaved changes

* fix dependency error

* fix shard_num problem

* py client

* remove lock and graph-type

* add load direct graph

* add load direct graph

* add load direct graph

* batch random_sample

* batch_sample_k

* fix num_nodes size

* batch brpc

* batch brpc

* add test

* add test

* add load_nodes; change add_node function

* change sample return type to pair

* resolve conflict

* resolved conflict

* resolved conflict

* separate server and client

* merge pair type

* fix

* resolved conflict

* fixed segment fault; high-level VLOG for load edges and load nodes

* random_sample return 0

* rm useless loop

* test:load edge

* fix ret -1

* test: rm sample

* rm sample

* random_sample return future

* random_sample return int

* test fake node

* fixed here

* memory leak

* remove test code

* fix return problem

* add common_graph_table

* random sample node &test & change data-structure from linkedList to vector

* add common_graph_table

* sample with srand

* add node_types

* optimize nodes sample

* recover test

* random sample

* destruct weighted sampler

* GraphEdgeBlob

* WeightedGraphEdgeBlob to GraphEdgeBlob

* WeightedGraphEdgeBlob to GraphEdgeBlob

* pybind sample nodes api

* pull nodes with step

* fixed pull_graph_list bug; add test for pull_graph_list by step

* add graph table;name

* add graph table;name

* add pybind

* add pybind

* add FeatureNode

* add FeatureNode

* add FeatureNode Serialize

* add FeatureNode Serialize

* get_feat_node

* avoid local rpc

* fix get_node_feat

* fix get_node_feat

* remove log

* get_node_feat return  py:bytes

* merge develop with graph_engine

* fix threadpool.h head

* fix

* fix typo

* resolve conflict

* fix conflict

* recover lost content

* fix pybind of FeatureNode

* recover cmake

* recover tools

* resolve conflict

* resolve linking problem

* code style

* change test_server port

* fix code problems

* remove shard_num config

* remove redundent threads

* optimize start server

* remove logs

* fix code problems by reviewers' suggestions
Co-authored-by: NHuang Zhengjie <270018958@qq.com>
Co-authored-by: NWeiyue Su <weiyue.su@gmail.com>
Co-authored-by: Nsuweiyue <suweiyue@baidu.com>
Co-authored-by: Nluobin06 <luobin06@baidu.com>
Co-authored-by: Nliweibin02 <liweibin02@baidu.com>

* [Parallel UT]improve Parallel UT level on Windows/Linux

* [Parallel UT]improve Parallel UT level on Windows/Linux

* [Parallel UT]Improve Parallel UT level on Windows/Linux

* [Parallel UT]Improve Parallel UT level on Windows/Linux

* fix CI

* Second optimization of retry method

* fix show_ut_retry_result repeat execuate

* fix wget with no proxy on windows

* modified import packages

* fix format error

* fix bug

* fix format error

* fix format error

* To help timeout_ut debug

* To help timeout_ut debug

* added show information

* add precision on mac

* added judge

* match file_ut.json on mac

* fix code format error

* fix code format error

* fix error caused by length of ut_lists exceeds the limit

* fix format error,notest,test=cpu

* fix code format error

* add windows judge on get_pr_ut

* add precision test for windows ci exclude check_added_ut

* fix error

* added PRECISION_TEST parameters

* fix format error