* refine ~npuOpRunner

* implement destructor and forbid copy

* use reference to avoid copy

* use const reference

* relax adam precision

* fix top_k

* optimize OP's compilation time, test=develop

* add more op and run ci test, test=develop

* CUDA Kernel register in cc file, test=develop

* fix macros, test=develop

* fix undefined symbol error, test=develop

* fix compilation error and undefined symbol, test=develop

* fix compilation error on Windows, test=develop

* fix compilation error on Windows, test=develop

Sigmoid: Out = Sigmoid(X)
SigmoidGrad: DX = DOut*(1-Out)*Out

[This Patch]
Out
DOut -> SigmoidGradGrad -> DOutNew
DDX                        DDOut

DDOut = (1-Out)*Out*DDX
DOutNew = (1-2*Out)*DOut*DDX

* added op cast functionality for fp32/bf16

* added newline

* added entries in static mode white list and unity build

* fixed failing tests

* changes after review

* added formatting

* upgraded tests file as reviewer suggested

* changes after review

* minor change

* modify complex template for elementwise ops

* modify mul, div grad struct

* add complex template for CudaShuffleDownSync CudaShuffleXorSync funcs and fix the bug when delete cuda<9000

* fix shuffle func args bug

* fix shuffle func args bug

* fix shuffle func args bug

* add the op def proto, test=develop

* add while.pbtxt

* revert elementwise and disable trt conv if strides > 1

* strides check

* remove useless var

* comments

* fix save/load with unexpected value
* fix save and user interface
* add save sparse sharding to selected rows

* add async save for sparse table
* add load profiler for sparse table
* add load info for sparse table

* fix hogwild_worker dev_ctx place bug; test=develop

* modify conj, real, imag OP to complex template

* replace with complex template to dot Op

* replace with complex template to Abs Op

* add support for complex64 and complex128

* graph engine demo

* upload unsaved changes

* fix dependency error

* fix shard_num problem

* py client

* remove lock and graph-type

* add load direct graph

* add load direct graph

* add load direct graph

* batch random_sample

* batch_sample_k

* fix num_nodes size

* batch brpc

* batch brpc

* add test

* add test

* add load_nodes; change add_node function

* change sample return type to pair

* resolve conflict

* resolved conflict

* resolved conflict

* separate server and client

* merge pair type

* fix

* resolved conflict

* fixed segment fault; high-level VLOG for load edges and load nodes

* random_sample return 0

* rm useless loop

* test:load edge

* fix ret -1

* test: rm sample

* rm sample

* random_sample return future

* random_sample return int

* test fake node

* fixed here

* memory leak

* remove test code

* fix return problem

* add common_graph_table

* random sample node &test & change data-structure from linkedList to vector

* add common_graph_table

* sample with srand

* add node_types

* optimize nodes sample

* recover test

* random sample

* destruct weighted sampler

* GraphEdgeBlob

* WeightedGraphEdgeBlob to GraphEdgeBlob

* WeightedGraphEdgeBlob to GraphEdgeBlob

* pybind sample nodes api

* pull nodes with step

* fixed pull_graph_list bug; add test for pull_graph_list by step

* add graph table;name

* add graph table;name

* add pybind

* add pybind

* add FeatureNode

* add FeatureNode

* add FeatureNode Serialize

* add FeatureNode Serialize

* get_feat_node

* avoid local rpc

* fix get_node_feat

* fix get_node_feat

* remove log

* get_node_feat return  py:bytes

* merge develop with graph_engine

* fix threadpool.h head

* fix

* fix typo

* resolve conflict

* fix conflict

* recover lost content

* fix pybind of FeatureNode

* recover cmake

* recover tools

* resolve conflict

* resolve linking problem

* code style

* change test_server port

* fix code problems

* remove shard_num config

* remove redundent threads

* optimize start server

* remove logs

* fix code problems by reviewers' suggestions

* move graph files into a folder

* code style change

* remove graph operations from base table

* optimize get_feat function of graph engine

* fix long long count problem

* remove redandunt graph files

* remove unused shell

* recover dropout_op_pass.h

* fix potential stack overflow when request number is too large & node add & node clear & node remove
Co-authored-by: NHuang Zhengjie <270018958@qq.com>
Co-authored-by: NWeiyue Su <weiyue.su@gmail.com>
Co-authored-by: Nsuweiyue <suweiyue@baidu.com>
Co-authored-by: Nluobin06 <luobin06@baidu.com>
Co-authored-by: Nliweibin02 <liweibin02@baidu.com>
Co-authored-by: Ntangwei12 <tangwei12@baidu.com>

* added support for most matmul cases

* added more functionality

* full functionality of matmul op, fp32 only

* added bf16 tests and functionality

* added formatting

* changes after review

* minor change

* added reviewers suggestions

* replace complex in set tensor from and to numpy

* replace complex template in cast op

* optimize softmax with cross entropy hard label

* label ignore_index cleaning

* cast indices and label if their type is not consistent

* fix bug

* add unittest

* fix gather op and add logsumexp op on kunlun

* update xpu depence

* update tests and fix elementwise_add

* add complex template file

* add numtraits for complex template

* add complex template type register

* modify specify template of complex

* modify specify template of complex

* modify specify template of complex

* modify specify template of complex

* make TensorCheckerVisitor support complex type

* fix operator= error

* add complex template

* add complex template type

* add complex template type to pyarray transform

* add complex template type to pyarray transform

* remove complex type for dlpack register

* set dlpack supprot complex type

* set dlpack supprot complex type

* set dlpack supprot complex type

* remove explict for complex constructor

* add complex unit test file

* CI skip inference test if only python files modified, test=develop

* fix compilation error on ROCM, test=develop

* fix cmake error on PR-CI-ROCM-Compile, test=develop

* [Rocm] fix test of random_crop_op

* [Rocm] fix test of random_crop_op

* [Rocm] fix test of random_crop_op & simple_rnn_op

* [Rocm] fix test of random_crop_op & simple_rnn_op & logsumexp

* [Rocm] fix test of random_crop_op & simple_rnn_op & logsumexp

* [Rocm] fix test of random_crop_op & simple_rnn_op & logsumexp

* [Rocm] fix test of random_crop_op & logsumexp

* Output indices of top_k npu op change to int64

* fix accuracy npu bug

* fix errors

* change cast method to FillNpuTensorWithConstant

* change cast method to FillNpuTensorWithConstant

* unit double

* unit double

* fix the paddle compare op for the broadcast

* fix compare op in for in the cuda device

* fix precision of mp

* fix bug of seed

* fix dp

* print group

* delete unused files.

* BugFix with ParseInputDataType from LodTensorArray

* BugFix with ParseInputDataType from LodTensorArray