1. 09 4月, 2021 1 次提交
    • L
      [NPU] cherry-pick basic NPU components/allocator/operator/executor supports from ascendrc (#32144) · ccf5709d
      Leo Chen 提交于
      * [feature] support npu allocator (#30840)
      
      [feature] support npu allocator
      
      * [feature] support npu operator (#30951)
      
      [feature] support npu operator
      
      * [feature] support npu allocator, part 2 (#30972)
      
      * support npu allocator
      
      * add npu device context
      
      * fix some compile problem
      
      * fix some compile problem
      
      * add npu info
      
      * compile ok
      
      * fix include dir
      
      * support naive_best_fit_allocator
      
      * run ut ok, bug failed to exit
      
      * call aclrtResetDevice before exit
      
      * fix aclFinilize
      
      * add system allocatot test
      
      * add selected_gpus in gtest
      
      * add tensor_test for npu
      
      * support npu op, initial commit
      
      * add npu stream
      
      * add elementwise_add_op
      
      * compile ok
      
      * fix typo
      
      * fix elementwise_add_op_npu_test
      
      * support op run
      
      * test can run but failed
      
      * change aclopExecuteV2 to aclopCompileAndExecute
      
      * support parsing ascend rank table file (#31000)
      
      support parsing ascend rank table file
      
      * Fix reshape on GE graph. (#31084)
      
      Fix reshape on GE graph
      
      * add npu kernel for elementwise_sub and elementwise_sub_grad (#30973)
      
      * add npu sub op
      
      * fix typo
      
      * rename test
      
      * fix bug
      
      * fix bug
      
      * add fp16 kernel
      
      * fix typo
      
      * support sub grad op
      
      * support elementwise_sub_grad op
      Co-authored-by: Nfrankwhzhang <frankwhzhang@126.com>
      
      * Fix compilation problem (#31100)
      
      Fix compilation problem (#31100)
      
      * fix compile
      
      * fix code stype
      
      * remove const_cast
      
      * support adding correct npu op in pybind.h (#31143)
      
      * support adding correct npu op in pybind.h
      
      * refine code
      
      * [NPU] Support executor with NPU (#31057)
      
      * [NPU] Support executor with NPU
      
      * Fix code according to reviews
      
      * Fix code
      
      * Add unittest for sub op npu
      
      * refactor npu device manager (#31154)
      
      refactor npu device manager (#31154)
      
      * fix selected npus
      
      * fix compile
      
      * fix reading flags from env
      
      * format
      Co-authored-by: Nxiayanming <41795079@qq.com>
      Co-authored-by: Ngongweibao <weibao.gong@gmail.com>
      Co-authored-by: Nfrankwhzhang <frankwhzhang@126.com>
      Co-authored-by: Nliym27 <33742067+liym27@users.noreply.github.com>
      ccf5709d
  2. 08 4月, 2021 1 次提交
  3. 07 4月, 2021 1 次提交
    • Z
      【NPU】Merge ascend GE&distributed code by 0208 from ascendrc (#31957) · 8c7c53b3
      zhang wenhui 提交于
      * Ascend rc (#30483)
      
      * Fix compilcation on CANN20.1 and older (#30494)
      
      Fix compilcation on CANN20.1 and older
      
      * Add distribution supported (#30578)
      
      Add distribution supported
      
      * Build praser for Hcom* operators (#30627)
      
      Build praser for Hcom* operators
      
      * Pass device_ids info from launch to trainer. (#30632)
      
      Pass device_ids info from launch to trainer
      
      * Add Hccl program group (#30642)
      
      Add Hccl program group
      
      * Add startup bash files of test_ascend_group. (#30645)
      
      Add startup bash files of test_ascend_group
      
      * cleanup (#30646)
      
      cleanup test_ascend_group.py
      
      * [Feature] Build parser to support distributed training (#30658)
      
      [Feature] Build parser to support distributed training
      
      * fix compilation on ascend-20.1 (#30722)
      
      fix compilation on ascend-20.1
      
      * Dev/fix ascend string (#30749)
      
      Dev/fix ascend string
      
      * code style (#30781)
      
      code style
      
      * Merge ascend_optimizer and ascend_parser. (#30776)
      
      Merge ascend_optimizer and ascend_parser.
      
      * Ascendrc add converted op : [range/equal/range/uniform_random/expand/squeeze], fix cast op bug  (#30797)
      
      Ascendrc add converted op : [range/equal/range/uniform_random/expand/squeeze], fix cast op bug
      
      * Add paddle ascend distribution training supported (#30796)
      
      Add paddle ascend distribution training supported
      
      * pass cxx_flags to gloo cmake (#30857)
      
      * Destroy session first. (#30954)
      
      Destroy session first.
      
      * merge
      
      * fix, test=develop
      
      * fix, test=develop
      
      * fix style, test=develop
      
      * fix, test=develop
      
      * fix
      
      * fix log fatal, test=develop
      
      * fix enforce style, test=develop
      
      * fix, test=develop
      
      * fix, test=develop
      
      * fix rccl, test=develop
      
      * fix test, test=develop
      
      * fix, test=develop
      
      * fix, test=develop
      
      * fix, test=develop
      
      * fix node_num, test=develop
      
      * fix ids str, test=develop
      
      * fix ids str, test=develop
      
      * fix ids str, test=develop
      
      * fix, test=develop
      
      * fix, test=develop
      
      * fix, test=develop
      
      * fix, test=develop
      
      * fix, test=develop
      
      * fix, test=develop
      
      * fix, test=develop
      
      * fix, test=develop
      
      * fix style code, test=develop
      
      * fix style code, test=develop
      
      * fix style code, test=develop
      
      * fix style code, test=develop
      Co-authored-by: Nhutuxian <hutuxian2011@sina.cn>
      Co-authored-by: Ngongweibao <weibao.gong@gmail.com>
      Co-authored-by: NVoid Main <voidmain1313113@gmail.com>
      Co-authored-by: NLeo Chen <chenqiuliang@baidu.com>
      Co-authored-by: Ndingsiyu <18369187719@163.com>
      Co-authored-by: NOleNet <olenet@126.com>
      8c7c53b3
  4. 02 4月, 2021 1 次提交
    • S
      graph engine (#31226) · 94736d60
      seemingwang 提交于
      * graph engine demo
      
      * upload unsaved changes
      
      * fix dependency error
      
      * fix shard_num problem
      
      * py client
      
      * remove lock and graph-type
      
      * add load direct graph
      
      * add load direct graph
      
      * add load direct graph
      
      * batch random_sample
      
      * batch_sample_k
      
      * fix num_nodes size
      
      * batch brpc
      
      * batch brpc
      
      * add test
      
      * add test
      
      * add load_nodes; change add_node function
      
      * change sample return type to pair
      
      * resolve conflict
      
      * resolved conflict
      
      * resolved conflict
      
      * separate server and client
      
      * merge pair type
      
      * fix
      
      * resolved conflict
      
      * fixed segment fault; high-level VLOG for load edges and load nodes
      
      * random_sample return 0
      
      * rm useless loop
      
      * test:load edge
      
      * fix ret -1
      
      * test: rm sample
      
      * rm sample
      
      * random_sample return future
      
      * random_sample return int
      
      * test fake node
      
      * fixed here
      
      * memory leak
      
      * remove test code
      
      * fix return problem
      
      * add common_graph_table
      
      * random sample node &test & change data-structure from linkedList to vector
      
      * add common_graph_table
      
      * sample with srand
      
      * add node_types
      
      * optimize nodes sample
      
      * recover test
      
      * random sample
      
      * destruct weighted sampler
      
      * GraphEdgeBlob
      
      * WeightedGraphEdgeBlob to GraphEdgeBlob
      
      * WeightedGraphEdgeBlob to GraphEdgeBlob
      
      * pybind sample nodes api
      
      * pull nodes with step
      
      * fixed pull_graph_list bug; add test for pull_graph_list by step
      
      * add graph table;name
      
      * add graph table;name
      
      * add pybind
      
      * add pybind
      
      * add FeatureNode
      
      * add FeatureNode
      
      * add FeatureNode Serialize
      
      * add FeatureNode Serialize
      
      * get_feat_node
      
      * avoid local rpc
      
      * fix get_node_feat
      
      * fix get_node_feat
      
      * remove log
      
      * get_node_feat return  py:bytes
      
      * merge develop with graph_engine
      
      * fix threadpool.h head
      
      * fix
      
      * fix typo
      
      * resolve conflict
      
      * fix conflict
      
      * recover lost content
      
      * fix pybind of FeatureNode
      
      * recover cmake
      
      * recover tools
      
      * resolve conflict
      
      * resolve linking problem
      
      * code style
      
      * change test_server port
      
      * fix code problems
      
      * remove shard_num config
      
      * remove redundent threads
      
      * optimize start server
      
      * remove logs
      
      * fix code problems by reviewers' suggestions
      Co-authored-by: NHuang Zhengjie <270018958@qq.com>
      Co-authored-by: NWeiyue Su <weiyue.su@gmail.com>
      Co-authored-by: Nsuweiyue <suweiyue@baidu.com>
      Co-authored-by: Nluobin06 <luobin06@baidu.com>
      Co-authored-by: Nliweibin02 <liweibin02@baidu.com>
      94736d60
  5. 01 4月, 2021 3 次提交
    • C
      add custom init grad for backward function (#31540) · 83b953f5
      chentianyu03 提交于
      * add custom init grad for backward function
      
      * add custom init grad for backward function
      
      * handle when the grad_tensor is none
      
      * handle when the grad_tensor is none
      
      * fix the args type error on windows platform
      
      * modify the args order and doc
      
      * format code
      
      * add grad_tensor to xpu
      
      * modify the grad_tensor type check
      
      * add paddle.backward api to support multi tensors gradient compute
      
      * add paddle.backward api to support multi tensors gradient compute
      
      * add paddle.atuograd module and backward api
      
      * change tensor.backward func args
      
      * modify tensor backward api
      
      * remove create_graph intputs args
      
      * add doc and examplex code for backward api
      
      * when have the same tensor, throw error
      
      * modify test Init func args
      
      * modify the execute.Init func args in test files
      
      * add paddle.autograd package in setup.py.in
      
      * modify error msg, remove _run_backward method in class Tensor
      
      * add test cases for backward api
      83b953f5
    • K
      new group (#31682) · 07741593
      kuizhiqing 提交于
      * new group
      
      * ci compatible fix
      
      * assert nccl
      07741593
    • C
      Refactor and simplify hook design & add Tensor.register_hook API (#31775) · dbeb3ea4
      Chen Weihang 提交于
      * refactor and simplify hook design
      
      * fix reducer add hook error
      
      * add Tensor.register_hook basic impl
      
      * refine prepare data impl
      
      * revert prepare data change
      
      * support register_hook for Tensor
      
      * add hook test in model
      
      * polish tests and doc example
      
      * fix double grad test failed
      
      * remove reduce hook func
      
      * fix set empty error
      
      * polish code by comments
      
      * change reduce_hook to mutable_hook
      
      * remove useless tmp_ins
      
      * fix shape code format error
      
      * fix shape code format error
      dbeb3ea4
  6. 31 3月, 2021 2 次提交
  7. 30 3月, 2021 2 次提交
  8. 29 3月, 2021 1 次提交
  9. 26 3月, 2021 2 次提交
  10. 23 3月, 2021 1 次提交
  11. 18 3月, 2021 1 次提交
  12. 15 3月, 2021 1 次提交
  13. 05 3月, 2021 1 次提交
  14. 04 3月, 2021 2 次提交
  15. 26 2月, 2021 1 次提交
  16. 22 2月, 2021 1 次提交
  17. 20 2月, 2021 2 次提交
  18. 19 2月, 2021 1 次提交
  19. 10 2月, 2021 1 次提交
    • C
      New custom operator extension mechanism (#30690) · f649442d
      Chen Weihang 提交于
      * initial commit: simple demo
      
      * polish copyright format
      
      * add grap op simple demo
      
      * adapt uncertain number of argument
      
      * change trait marco name
      
      * add place & dtype support for add kernel
      
      * add dispath and infershape func
      
      * poish code & add notes
      
      * add dynamic_loader dep for paddle_framework
      
      * add new custom op test dir
      
      * polish impl details
      
      * add unittest for new custom op
      
      * fix failed unittest
      
      * Costum op (#1)
      
      * fix compile error
      
      * wrap framework tensor with LoDTensor
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * add CustomTensor default constructor
      
      * add size() for CustomTensor
      
      * make size const for CustomTensor
      
      * refactor place related api to circle the concept
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * make place const
      
      * make Tensor copy
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * remove additional head of framework
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * add gpu test
      
      * merge latest cwh code in
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * Remove ShareData from user && Change CustomTensor to Tensor && Support more data type (#2)
      
      * fix compile error
      
      * wrap framework tensor with LoDTensor
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * add CustomTensor default constructor
      
      * add size() for CustomTensor
      
      * make size const for CustomTensor
      
      * refactor place related api to circle the concept
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * make place const
      
      * make Tensor copy
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * remove additional head of framework
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * add gpu test
      
      * merge latest cwh code in
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * hid share data from and to
      
      * rename CustomTensor to Tensor
      
      * refactor register design & add test
      
      * change op_funtion to op_meta_info
      
      * split op meta info into .h and .cc
      
      * move get methods into friend class
      
      * move OpMetaInfoHelper into framework space
      
      * move CustomTensorUtils into framework space
      
      * change pybind api name
      
      * move PD C API into op meta info
      
      * add register custom op api
      
      * remove inference cmake change
      
      * refactor copy to api && change Reshape to lowercase && support more dtype && add more test (#3)
      
      * fix compile error
      
      * wrap framework tensor with LoDTensor
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * add CustomTensor default constructor
      
      * add size() for CustomTensor
      
      * make size const for CustomTensor
      
      * refactor place related api to circle the concept
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * make place const
      
      * make Tensor copy
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * remove additional head of framework
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * add gpu test
      
      * merge latest cwh code in
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * hid share data from and to
      
      * rename CustomTensor to Tensor
      
      * support multi dtype
      
      * remove lod, make reshape lowercase, add copy test and refactor copy api
      
      * remove lod, make reshape lowercase, add copy test and refactor copy api
      
      * remove lod, make reshape lowercase, add copy test and refactor copy api
      
      * remove lod, make reshape lowercase, add copy test and refactor copy api
      
      * fix copy to error
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * polish detail & error message
      
      * polish test details
      
      * Add cast api && Change copy related api to copy_to && add more test (#4)
      
      * fix compile error
      
      * wrap framework tensor with LoDTensor
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * add CustomTensor default constructor
      
      * add size() for CustomTensor
      
      * make size const for CustomTensor
      
      * refactor place related api to circle the concept
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * fix compile error
      
      * make place const
      
      * make Tensor copy
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * debug CustomTensor core
      
      * remove additional head of framework
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * use back to shared ptr for custom tensor
      
      * add gpu test
      
      * merge latest cwh code in
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * adjust ut code of custom op
      
      * hid share data from and to
      
      * rename CustomTensor to Tensor
      
      * support multi dtype
      
      * remove lod, make reshape lowercase, add copy test and refactor copy api
      
      * remove lod, make reshape lowercase, add copy test and refactor copy api
      
      * remove lod, make reshape lowercase, add copy test and refactor copy api
      
      * remove lod, make reshape lowercase, add copy test and refactor copy api
      
      * fix copy to error
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add more test
      
      * add type cast
      
      * add cast and make copy to api
      
      * add cast and make copy to api
      
      * add cast and make copy to api
      
      * add cast and make copy to api
      
      * merge cwh code
      
      * merge cwh code
      
      * merge cwh code
      
      * merge cwh code
      
      * merge cwh code
      
      * add more error log
      
      * add more error log
      
      * polish code
      
      * used for test
      
      * remove test comment
      
      * remove test comment
      
      * fix uint8 type error
      
      * fix lost uint8 type error
      
      * add test for coverage
      
      * polish details by reviewer comments
      
      * add prefix for DISABLE_COPY_AND_ASSIGN
      Co-authored-by: NJiabin Yang <360788950@qq.com>
      f649442d
  20. 05 2月, 2021 1 次提交
  21. 04 2月, 2021 2 次提交
  22. 03 2月, 2021 2 次提交
  23. 01 2月, 2021 1 次提交
  24. 29 1月, 2021 1 次提交
  25. 25 1月, 2021 1 次提交
  26. 21 1月, 2021 1 次提交
  27. 20 1月, 2021 2 次提交
    • W
      use nvtx push pop in timeline (#30567) · 90773473
      wanghuancoder 提交于
      * delete empty line of pybing.cc, test=develop
      
      * use nvtx push pop in timeline, test=develop
      
      * change year, test=develop
      
      * add #ifdef PADDLE_WITH_CUDA, test=develop
      
      * add #ifndef WIN32, test=develop
      
      * is_pushed to is_pushed_, test=develop
      90773473
    • W
      add some RecordEvent, for dygraph timeline (#30299) · d1b25ed9
      wanghuancoder 提交于
      * add some RecordEvent, for dygraph timeline, test=develop
      
      * change GpuMemcpySync to memory::Copy, test=develop
      
      * fix compile problem, test=develop
      
      * fix compile problem, test=develop
      
      * fix, test=develop
      
      * fix, test=develop
      d1b25ed9
  28. 19 1月, 2021 2 次提交
  29. 18 1月, 2021 1 次提交