1. 11 2月, 2020 3 次提交
    • H
      Paddlebox about box_wrapper (#22497) · 1a7962be
      hutuxian 提交于
      Refine PaddleBox Framework, Main functions: 
      * Add MetricMsg util class, which can calculate metrics like AUC, bucket_error, COPC.
      * Replace FeedPass with new interface: BeginFeedPass & EndFeedPass
      * Refactor Pull/Push Sparse Function in box_wrapper.
      * Use CUDA Kernel to copy keys and copy feasign between tensor and boxps struct.
      * Cache copied keys in pull sparse in order to reuse it in push period.
      1a7962be
    • Y
      multi-loss optimization by adding a DownpourOpt worker (#22025) · 2235ee1a
      yaoxuefeng 提交于
      * update
      
      * update test=develop
      
      * update compile set test=develop
      
      * update compile set test=develop
      
      * update test=develop
      
      * update test=develop
      
      * update test=develop
      
      * update compile setting test=develop
      
      * update compile setting test=develop
      
      * update run demo test=develop
      
      * update test=develop
      
      * update test=develop
      
      * fix test=develop
      
      * update test=develop
      
      * update test=develop
      
      * update test=develop
      
      * update test=develop
      
      * update test=develop
      
      * update test=develop
      
      * update test=develop
      
      * update test=develop
      
      * update test=develop
      
      * update format test=develop
      
      * update format test=develop
      
      * update style test=develop
      
      * update style test=develop
      
      * change style test=develop
      
      * change style test=develop
      
      * change style test=develop
      
      * add dataset unittest test=develop
      
      * update test=develop
      
      * update for record test=develop
      
      * udpate style for record test=develop
      
      * update for record test=develop
      
      * update for record test=develop
      
      * update for record test=develop
      
      * fix format test=develop
      
      * update test=develop
      
      * update test=develop
      
      * update test=develop
      
      * update test=develop
      
      * update test=develop
      2235ee1a
    • W
      Compile without nccl deps. [1/2] (#22509) · a90fa540
      Wilber 提交于
      支持不依赖nccl进行编译。[1/2]
      
      多卡下,如果没有打开WITH_NCCL开关编译,多卡不能通信,则只能选择一张卡使用。
      Co-authored-by: N石晓伟 <39303645+Shixiaowei02@users.noreply.github.com>
      a90fa540
  2. 05 2月, 2020 1 次提交
  3. 02 2月, 2020 1 次提交
  4. 14 1月, 2020 1 次提交
  5. 20 12月, 2019 1 次提交
    • T
      add table id in cache shuffle (#21585) · c3cf42d0
      Thunderbrook 提交于
      * general table
      
      * add sparse table
      test=develop
      
      * no cvm
      test=develop
      
      * add no_cvm
      test=develop
      
      * add note
      test=develop
      
      * code style
      test=develop
      
      * code style
      test=develop
      
      * code style
      test=develop
      
      * code style
      test=develop
      
      * code style
      test=develop
      
      * add key of optimizer
      test=develop
      
      * solve pslib stop core
      test=develop
      
      * barrier
      test=develop
      
      * add notes
      test=develop
      
      * add table id in cache shuffle
      test=develop
      
      * table id
      test=develop
      
      * code style
      test=develop
      c3cf42d0
  6. 10 12月, 2019 1 次提交
  7. 25 11月, 2019 1 次提交
  8. 21 11月, 2019 1 次提交
    • T
      solve pslib core in stop worker (#21263) · 0d17c1b8
      Thunderbrook 提交于
      * general table
      
      * add sparse table
      test=develop
      
      * no cvm
      test=develop
      
      * add no_cvm
      test=develop
      
      * add note
      test=develop
      
      * code style
      test=develop
      
      * code style
      test=develop
      
      * code style
      test=develop
      
      * code style
      test=develop
      
      * code style
      test=develop
      
      * add key of optimizer
      test=develop
      
      * solve pslib stop core
      test=develop
      
      * barrier
      test=develop
      
      * add notes
      test=develop
      0d17c1b8
  9. 20 11月, 2019 1 次提交
    • T
      support general embedding params (#21217) · 349e82d6
      Thunderbrook 提交于
      * general table
      
      * add sparse table
      test=develop
      
      * no cvm
      test=develop
      
      * add no_cvm
      test=develop
      
      * add note
      test=develop
      
      * code style
      test=develop
      
      * code style
      test=develop
      
      * code style
      test=develop
      
      * code style
      test=develop
      
      * code style
      test=develop
      
      * add key of optimizer
      test=develop
      349e82d6
  10. 15 11月, 2019 2 次提交
  11. 25 10月, 2019 1 次提交
    • X
      fix several sparse table issuses (#20686) · 48669aa8
      xujiaqi01 提交于
      * no longer need to define all embedding layers (no one less) of all slots in each program. make trainer_param repeated in ps.proto.
      * add find_distributed_lookup_table_grads instead of hard code GRAD
      * support embedding stop gradient. push sparse has error before fix this.* 
      * fix fill sparse, skip slots which do not have embedding. each slot's embedding in a sparse table should be used in all training programs before fix this.
      * fix pull sparse, skip slots which do not have embedding.
      * fix collect feasign label info, skip slots which do not have embedding.
      * support when there are multi sparse tables in one or multi training programs, each program can pull/push its own related sparse tables instead of all sparse tables.
      * test=develop
      48669aa8
  12. 24 9月, 2019 1 次提交
    • X
      support change shuffle and train thread num (#19841) · cedc0477
      xujiaqi01 提交于
      * support change shuffle thread num
      * support change train thread num
      * fix receive shuffle data of each channel
      * data norm stop gradient
      * add check thread_tensor type and root_tensor type when merge metric
      * remove sleep in shuffle, add config
      * add config of pslib client to client communication
      * fix xbox str
      * add data norm op testcase
      * add flush in trainer finalize
      cedc0477
  13. 17 9月, 2019 1 次提交
  14. 31 8月, 2019 1 次提交
    • H
      Paddlebox Framework (#18982) · c756b5d2
      hutuxian 提交于
      * Support looking up embeddings from BoxPS.
      * Add a _pull_box_sparse op, for now this op is not exposed to users.
      * Add a BoxHelper class, providing 'BeginPass', 'EndPass', 'FeedPass' functions and so on.
      * Add 'BoxPSDataset' in python code.
      * Add a compile options WITH_BOX_PS and a MACRO PADDLE_WITH_BOX_PS.
      * Add UT.
      * More concrete information pls refer to: https://github.com/PaddlePaddle/Paddle/pull/18982
      c756b5d2
  15. 14 8月, 2019 1 次提交
  16. 11 8月, 2019 1 次提交
    • Y
      add save cache model api in fleet& add slots shuffle in dataset module & add... · 9150cf50
      yaoxuefeng 提交于
      add save cache model api in fleet& add slots shuffle in dataset module & add metric op to calculate ctr related metrics (#18871)
      
      * add ctr related metric layer test=develop
      
      * add save cache and slots shuffle test=develop
      
      * add save cache and slots shuffle test=develop
      
      * fix error
      
      * fix error
      
      * fix style for ci
      
      * fix for comments
      
      * change SlotsShuffle input to std::strinf for generality
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix stylr
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix style
      
      * fix style
      
      * change non-const reference to pointer
      
      * fix style
      
      * fix style
      
      * fix style test=develop
      
      * fix style  test=develop
      
      * add return ins num in ctr metric op
      
      * change dtype to float in metric_op.py
      
      * fix error test=develop
      
      * fix style test=develop
      
      * fix API spec
      
      * fix API spec
      
      * fix API spec test=develop
      
      * add UT test=develop
      9150cf50
  17. 29 7月, 2019 1 次提交
    • T
      add clear_model interface in fleetwrapper (#18815) · 52c1431e
      Thunderbrook 提交于
      * dump slot
      
      * test
      
      * proto
      
      * dump slot
      
      * test
      
      * proto
      
      * code style
      
      * code style
      
      * code style
      
      * style
      
      * add delete after unseen days
      
      * add unseen days
      
      * code style
      
      * conflict solve
      test=develop
      
      * add clear model
      
      * code style
      test=develop
      
      * code style
      test=develop
      52c1431e
  18. 25 7月, 2019 1 次提交
  19. 24 7月, 2019 1 次提交
    • T
      add slot to sparse table (#18686) · d8396281
      Thunderbrook 提交于
      The change includes 2 things:
      
      1. save delta model and shrink table are control by the same parameter before, now add delete_after_unseen_days to control shrink table.
      2. value in sparse table has no slot before, now add slot in sparse table, and add DownpureCtrAccessor to support the new meta.
      test=develop
      d8396281
  20. 23 7月, 2019 1 次提交
    • J
      support patch data, add load_one_table, fix bug (#18509) · d18aabb4
      jiaqi 提交于
      (1)support patch data (merge slots of instances of same line id, modify dense layer which
      changes its size)
      (2)add fleet load_one_table interface, support load from paddle model and load from pslib model
      (3)fix push sparse bug which cause push sparse cost more time(about 10% in my testcase)
      (4)when some slots are not in one of your network (join/update, etc.),data feed、collect label info、push/pull sparse will skip these slots, instead of throw error.
      (5)add more debug info in TrainFilesWithProfiler
      d18aabb4
  21. 15 5月, 2019 1 次提交
    • J
      add save/load model, shrink table, cvm, config file & fix pull dense bug (#17118) · 66d51206
      jiaqi 提交于
      * add save/load model, shrink table, cvm, config file & fix pull dense bug
      test=develop
      
      * fix global shuffle bug, fix pull dense bug, fix release memeory bug, fix shrink error
      add client flush, add get data size
      test=develop
      
      * fix global shuffle bug
      test=develop
      
      * fix global shuffle bug
      test=develop
      
      * fix code style
      test=develop
      
      * fix code style & modify pslib cmake
      test=develop
      
      * fix error of _role_maker
      test=develop
      
      * fix code style
      test=develop
      
      * fix code style
      test=develop
      
      * fix code style
      test=develop
      
      * fix code style
      test=develop
      
      * fix code style
      test=develop
      
      * fix windows compile error of fleet
      test=develop
      
      * fix global shuffle bug
      
      * add comment
      test=develop
      
      * update pslib.cmake
      test=develop
      
      * fix fill sparse bug
      test=develop
      
      * fix push sparse bug
      test=develop
      66d51206
  22. 22 4月, 2019 1 次提交
  23. 17 4月, 2019 2 次提交
  24. 16 4月, 2019 1 次提交
  25. 15 4月, 2019 1 次提交
  26. 04 4月, 2019 1 次提交
  27. 30 3月, 2019 3 次提交
  28. 29 3月, 2019 7 次提交