1. 21 3月, 2022 1 次提交
  2. 19 3月, 2022 1 次提交
    • P
      support inplace in dygraph eager_fluid state (#40400) · 8e612903
      pangyoki 提交于
      * [Eager] Support eager grad interface, draft version
      
      * Support eager grad interface with allow_unused and multi startup_op
      
      * Fix code format
      
      * Fix allow_unused case, return PyNone if tensor not initialize
      
      * Support output's stop_gradient related to create_graph
      
      * Support grad exception case in eager mode, fix coverage CI
      
      * Update ToPyObject, return PyNone if not initialize
      
      * AccumulationNode add FLAGS_retain_grad_for_all_tensor
      
      * Fix ci issue
      
      * Fix CI issue
      
      * fix, use core.eager.Tensor
      
      * Add func SetBufferSlotRankZeros for GradTensorHolder
      
      * Support retain_graph by using ClearTensorWrappers
      
      * Support retain_graph by using ClearTensorWrappers
      
      * Update retain_graph and no_grad_vars related test case
      
      * Update code gen logic for ClearTensorWrappers
      
      * Fix by override statement
      
      * fix override func args
      
      * Support retain_graph, update unit tests
      
      * Updated ClearTensorWrappers logic
      
      * fix grad python interface
      
      * Use deep copy and update unit tests
      
      * Polish code
      
      * Polish code
      
      * Fix CI issue, Deep copy only use when user set grad_tensors
      
      * Fix CI, use Backward instead RunBackward
      
      * Fix CI, Declare kernel explicitly in test file
      
      * Polish, remove vector of TensorWrapper
      
      * Refactor the logic of grad/backward, polish codes
      
      * Update code after merge upstream develop
      
      * Polish after merge upstream develop
      
      * Update to adapt new GradNodeBase superclass
      
      * Fix error introduced during conflict resolution
      
      * support inplace strategy in eager_fluid state
      
      * solve conflict
      
      * nothing
      
      * Update purify potential_startup_nodes logic
      
      * Fix errors
      
      * Polish code
      
      * Remove useless args for ToPyObject
      
      * Remove useless TensorWrappersSet
      
      * fix record conflict
      
      * Fix code-format, re-install pre-commit
      
      * fix tensor_wrapper bug
      
      * Fix pre-process logic for potential_startup_ops
      
      * Update unit tests, use eager mode
      
      * Fix conflicts
      
      * fix unittest timeout
      
      * little change
      Co-authored-by: NWeilong Wu <veyron_wu@163.com>
      8e612903
  3. 17 3月, 2022 1 次提交
  4. 15 3月, 2022 1 次提交
  5. 14 3月, 2022 1 次提交
  6. 11 3月, 2022 1 次提交
  7. 09 3月, 2022 3 次提交
  8. 07 3月, 2022 1 次提交
    • M
      cuBlasLt Epilogue To Fuse Linear + ReLU|GeLU (#39437) · 2a3d9eca
      Ming-Xu Huang 提交于
      * Added cuBlasLtHandle_t to device context.
      
      * Added fused_gemm_epilogue op.
      
      1. Added fused_gemm_epilogue op to leverage cuBlastLt Epilogue.
      2. Support fusion Act(X*Y + bias), X'dims >=2 and Y'dims shoule be 2.
      2. Act currently only be supported ReLU. (Will add GeLU in the future).
      
      * Added UT to fused_gemm_epilogue op.
      
      * Added LinearAct Pattern
      
      1. Added LinearAct into graph_pattern_detector.* to define (2.)'s
      pattern.
      2. LinearAct is used to detect act(element_add(matmul_v2(x, w), bias)).
      3. act currently only support ReLU (Will support GeLU in the future).
      
      * Added FuseGemmEpiloguePass
      
      1, Added FuseGemmEpiloguePass to handle nn.Linear + Act{ReLU}
      fusion (GeLU will be supported in the future).
      2. Only support matmul_v2 from nn.Linear.
      
      * Added pybind to BuildStrageter.fuse_gemm_epilogue_.
      
      * Added UT for fuse_gemm_epilogue_pass.
      
      * GeLU support and EpilogueSingleton
      
      1. Added GeLU support to fused_gemm_epilogue op.
      2. Added EpilogueSingleton to cache auxiliary pointer.
      3. Added related UTs.
      
      * Rename cublaslt_epilogue_opto gemm_epilogue_op.*.
      
      * Added both train and infer pattern to LinearAct.
      
      1. Added support of fwd graph with grap_ops linking to LinearAct.
      2. Added related changes to fuse_gemm_epilogue_pass for above
      modification.
      
      * Changed CUDA requirement from 11.4 to 11.6 for fuse_gemm_epilogue_pass.
      
      * Added identity activation support to gemm_epilogue_op.
      
      * Added Linear Fusion (matmul_v2 + ele_add)
      
      1. Added matmul_v2 + ele_add pattern to LinearActPattern.
      2. Added matmul_v2 + ele_add support to fuse_gemm_epilogue_pass.
      
      * Rename gemm_epilogue_op.* to fused_gemm_epilogue_op.*
      
      * Add fused_gemm_epilogue_grad op.
      
      1. Added fused_gemm_epilogue_grad to support backward epilogue fusion.
      
      * Add UTs to fused_gemm_epilogue_grad_op.
      
      * Change attribute name in fused_gemm_epilogue_grad_op for clearing.
      
      * Allow DX and DBias be dispensable to fused_gemm_epilogue_grad op.
      
      * Added ElementwiseAdd+Matmul+Act graph pattern detection.
      
      * Fuse backward of Linear( Act(x))
      
      1. Added backward fusion pass to Linear( Act(x)).
      2. Added backward fusion pass to Linear(x).
      
      * Added UTs to backward fusion of Linear(Act(x)).
      
      * Complete document of arguments to fused_gemm_epilogue_op.
      
      * Made arguments of some functions pass by reference.
      
      * Modify code with review comments.
      
      1. Made arguments of some function pass by reference.
      2. Removed redundant code.
      3. Followed Google code style to change code.
      
      * Made 'const' code style be consistent
      
      * Fixed random seed of python UTs.
      
      * Set Compiling constrains to cuBlasLt
      
      1. Require CUDA 11.6+
      2. Remove fuse_gemm_epilogue related tests when CUDA < 11.6.
      
      * Code Reivew from Paddle
      
      1. Changed arguments name is_first_gemm to without_x_gradient for
      clearing.
      2. Applied PADDLE_THROW in fused_gemm_epilogue_op.
      
      * Remove EpilogueSingleton
      
      1. Applied ReserveSpace to replace Epilogue for passing auxiliary
      pointers between FWD and BWD.
      
      * Fix a logical error and enhance UTs.
      
      1. Added act op count checking in UTs.
      2. Fix issue to fuse backward or ReLU(Linear(X)).
      3. TODO: solve GELU fusion issues.
      
      * Fix Linear and GeLU fusion issues.
      
      1. Modified graph_detech_pattern to fit with both linear wiht gelu or
      relu.
      2. Modified data range in Uts to allow negative values.
      
      * Removed fused_gemm_epilogue_op.h.
      
      * Rename namespace pten to phi.
      
      * Rename name of arguments in fused_gemm_epilogue_op
      
      1. bias -> Bias.
      2. out -> Out.
      3. reserve_space -> ReserveSpace.
      
      * Change EpiloguePassActivationCache as local variable.
      
      1. Removed singleton in EpiloguePassActivationCache.
      2. Made EpiloguePassActivationCache as an argument to each pass
      functions.
      2a3d9eca
  9. 02 3月, 2022 2 次提交
    • Z
      new fleet_desc builder (#39948) · 1c4e3e5d
      ziyoujiyi 提交于
      * delete gloo connect retry
      
      * the_one_ps dirs reconstruct
      
      * .
      
      * .
      
      * create the_one_ps dirs
      
      * create the_one_ps dirs
      
      * create the_one_ps dirs
      
      * create the_one_ps dirs
      
      * create the_one_ps dirs
      
      * create the_one_ps dirs
      
      * the one ps dirs modify
      
      * the one ps dirs modify
      
      * the one ps dirs modify
      
      * the one ps dirs modify
      
      * refactor ps optimize
      
      * refactor ps optimize
      
      * refactor ps optimize
      
      * .
      
      * .
      
      * .
      
      * .
      
      * .
      
      * .
      
      * refactor theoneps
      
      * the_one_ps
      
      * add ps pass unittest
      
      * add ps pass unittest
      
      * ps unitest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * ps unittest frame
      
      * add cpu_async_ps_mode test
      
      * add cpu_async_ps_mode test
      
      * add cpu_async_ps_mode test
      
      * ps unittest ready
      
      * ps unittest ready
      
      * solve dist_pass init conflict
      
      * solve import CommContext error
      
      * unittest ok
      
      * implement AllocateFrom
      
      * solve setup.py.in conflict
      
      * solve conflict
      
      * solve conflict
      
      * solve conflict
      
      * .
      
      * .
      
      * cpu-async-ps minimize test ok & gpu minimize test ok
      
      * add heter 2stage unittest
      
      * add heter 2stage unittest
      
      * add heter 2stage unittest
      
      * sync/geo test ok & fix heter_worker program ok
      
      * .
      
      * new fleet desc generator
      
      * new fleet_desc builder
      
      * new fleet_desc builder
      
      * .
      
      * .
      
      * correct ps.proto compile
      
      * .
      Co-authored-by: Nzkh2016 <zhangkaihuo@baidu.com>
      1c4e3e5d
    • W
      [Eager] open eager when WITH_PYTHON (#39979) · 9af72957
      wanghuancoder 提交于
      * open eager when WITH_PYTHON, test=develop
      
      * refine, test=develop
      
      * refine, test=develop
      
      * add DWITH_PYTHON for gen_fluid_lib, test=develop
      9af72957
  10. 01 3月, 2022 1 次提交
  11. 28 2月, 2022 1 次提交
  12. 23 2月, 2022 1 次提交
  13. 22 2月, 2022 1 次提交
  14. 21 2月, 2022 1 次提交
  15. 19 2月, 2022 1 次提交
    • S
      Add the DistributedFusedLamb optimizer (#39148) · 5df3cd61
      sneaxiy 提交于
      * add DistributedFusedLamb op
      
      * polish code
      
      * fix compile error
      
      * compatible with pten changement
      
      * fix rocm compile error
      
      * improve converage
      
      * update upstream/develop
      
      * fix cast_with_ptr.h
      
      * add FLAGS_distributed_lamb_divide_nranks_when_allreduce=1
      
      * fix clip before allreduce
      
      * add use_master_param_norm
      
      * code polish
      
      * fix bug
      
      * fix ROCM ci
      5df3cd61
  16. 09 2月, 2022 1 次提交
    • J
      Replace EagerTensor with Tensor (#39376) · 945a3ce9
      Jiabin Yang 提交于
      * merge legacy to fluid
      
      * Remove legacy code
      
      * Remove legacy code
      
      * Remove DataType test
      
      * Using Tensor directly instead of using EagerTensor
      
      * support gradient_accumulation
      
      * make test_imperative_lod_tensor_to_selected_rows longer
      
      * make test_imperative_lod_tensor_to_selected_rows longer
      945a3ce9
  17. 28 1月, 2022 1 次提交
  18. 25 1月, 2022 1 次提交
  19. 24 1月, 2022 1 次提交
    • Z
      Refactored python-level trace_op to call through _C_ops instead of... · c3796061
      Zhanlue Yang 提交于
      Refactored python-level trace_op to call through _C_ops instead of Tracer::TraceOp, under eager_mode (#38338)
      
      * Replaced core.ops with _C_ops
      
      * Refactored python-level trace_op to call through _C_ops instead of Tracer::TraceOp, under eager_mode
      
      * Modified trace_op interface
      
      * Refactored trace_op logic for eager mode
      
      * Added Eager Dygraph support for OpTest
      
      * Fixed ci issues
      
      * Fixed CI failures
      
      * Fixed Coverage CI Issues
      
      * Fixed XPU CI Issues
      c3796061
  20. 21 1月, 2022 1 次提交
  21. 19 1月, 2022 1 次提交
    • J
      ipu python interface p1 (#38096) · 0837a2cc
      jianghaicheng 提交于
      * ipu_commit_tests p1
      
      * resolve comments
      
      * resolve comments
      
      * resolve comments
      
      * resolve comments
      
      * resolve comments
      
      * resolve comments
      
      * resolve comments
      
      * update lint and ipustrategy introduction
      
      * update ipu_config
      
      * update __init__ of static
      
      * update doc
      
      * update doc 2
      
      * update doc 3
      
      * update doc 4
      
      * update doc 5
      
      * update doc 5
      
      * update doc 6
      
      * update lint
      
      * update lint 2
      
      * update ipustrategy
      
      * add IpuStrategy to all
      
      * update ipustrategy
      
      * update ipu_shard_guard
      
      * update ipu_shard_guard 2
      Co-authored-by: Nyaozhixin <522190855@qq.com>
      0837a2cc
  22. 14 1月, 2022 2 次提交
  23. 11 1月, 2022 1 次提交
  24. 10 1月, 2022 1 次提交
  25. 31 12月, 2021 1 次提交
  26. 23 12月, 2021 1 次提交
  27. 21 12月, 2021 1 次提交
  28. 20 12月, 2021 1 次提交
  29. 08 12月, 2021 1 次提交
  30. 07 12月, 2021 1 次提交
    • Y
      [Auto para] Relaunch with auto mapping function (#37326) · 506e79d1
      Yulong Ao 提交于
      * [Auto Parallel]  Add the unified cluster representation
      
      * [Auto Parallel] Add the graph class for physical mapping
      
      * [Auto Parallel] Add the simple physical mapper
      
      * Set the timeout of the mapper
      
      * Merge the upstream develop unittests cmake files
      
      * Fix a bug of the process group
      
      * Remove mapper unittest from platforms which is not GPU
      
      * Move the instantiation of process group after resharding
      
      * Add the local id for devices
      
      * Update the rank mapping format
      
      * [Auto Parallel] Relaunch with the rank mapping file
      
      * Remove the unnecessary json file
      
      * Avoid entering get_device_proc_info for auto mapping
      
      * Correct the mapper unit test
      
      * Add some comments
      
      * Remove the related files about mapping
      
      * Update the unittest for auto mapping
      
      * Remove unused rank_mapping unittest
      
      * Improve the unittest coverage
      
      * Improve the unittest coverage
      
      * Improve the unittest of relaunch
      
      * Fix the unittest problem in CI
      
      * Improve the unittest of relaunch
      
      * Remove unnecessary statements
      
      * Update the unittest cmakefile
      
      * Correct the cmakefile of auto parallel unittests
      
      * Modify codes based on the new elastic change
      
      * Use the GPUs exclusively in the unittest
      
      * Correct the cmakefile
      
      * Set the timeout of the unittest
      506e79d1
  31. 02 12月, 2021 1 次提交
  32. 30 11月, 2021 1 次提交
    • Y
      [Auto Parallel] Do the physical mapping between the process graph and the cluster graph (#37094) · b0dff05d
      Yulong Ao 提交于
      * [Auto Parallel]  Add the unified cluster representation
      
      * [Auto Parallel] Add the graph class for physical mapping
      
      * [Auto Parallel] Add the simple physical mapper
      
      * Set the timeout of the mapper
      
      * Merge the upstream develop unittests cmake files
      
      * Fix a bug of the process group
      
      * Remove mapper unittest from platforms which is not GPU
      
      * Move the instantiation of process group after resharding
      
      * Add the local id for devices
      
      * Update the rank mapping format
      
      * Add some comments
      
      * Remove the related files about mapping
      
      * Update the unittest for auto mapping
      
      * Remove unused rank_mapping unittest
      
      * Improve the unittest coverage
      
      * Improve the unittest coverage
      b0dff05d
  33. 27 11月, 2021 1 次提交
    • Y
      [Auto Parallel] Add the graph class for the process and cluster (#37482) · 48faf638
      Yulong Ao 提交于
      * [Auto Parallel]  Add the unified cluster representation
      
      * [Auto Parallel] Add the graph class for physical mapping
      
      * [Auto Parallel] Add the simple physical mapper
      
      * Set the timeout of the mapper
      
      * Merge the upstream develop unittests cmake files
      
      * Fix a bug of the process group
      
      * Remove mapper unittest from platforms which is not GPU
      
      * Move the instantiation of process group after resharding
      
      * Add the local id for devices
      
      * Update the rank mapping format
      
      * Add some comments
      
      * Remove the related files about mapping
      
      * Remove unused rank_mapping unittest
      
      * Improve the unittest coverage
      48faf638
  34. 26 11月, 2021 1 次提交
  35. 25 11月, 2021 2 次提交