• F
    [cherry-pick] Adapt BKCL comm for XPUPS (#42266) · 0c68ae0c
    Fan Zhang 提交于
    * XPUPS Adaptation (#40991)
    
    * Adapt XPUPS - 1st version - 3.24
    
    * Adapt XPUPS - update XPU PushSparse -  2nd version - 3.24
    
    * Adapt XPUPS - add XPU PullSparseOp - 3nd version - 3.25
    
    * refactor heter comm kernel
    
    * update. test=develop
    
    * Adapt XPUPS - modify by compilation - 4th version - 3.27
    
    * update calc_shard_offset. test=develop
    
    * update xpu kernel. test=develop
    
    * update args of calc_shard_offset
    
    * update. test=develop
    
    * remove customGradMerger
    
    * update. test=develop
    
    * heter_comm update
    
    * heter_comm update
    
    * update calc_shard_offset. test=develop
    
    * heter_comm update
    
    * update args of calc_shard_offset
    
    * update. test=develop
    
    * remove customGradMerger
    
    * update. test=develop
    
    * fix. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update optimizer kernel
    
    * Adapt XPUPS - use WITH_XPU_KP and modify wrapper kernel function - 5th version - 3.30
    
    * update. test=develop
    
    * update pslib.cmake
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * Adapt XPUPS - modify by kp compilation  - 6th version - 3.30
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update optimizer kernel
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * used by minxu
    
    * update heter_comm_inl
    
    * fix. test=develop
    
    * Adapt XPUPS - modify by kp compilation  - 7th version - 3.30
    
    * fix. test=develop
    
    * add optimizer kernel. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * 3.31 update
    
    * Adapt XPUPS - update kp compilation path  - 8th version - 3.31
    
    * add optimizer kernel. test=develop
    
    * fix kunlun not support size_t. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix kunlun not support size_t. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * update heter_comm_kernel.kps 3.31
    
    * fix. test=develop
    
    * fix. test=develop
    
    * update heter_comm_kernel.kps 3.31
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * update heter_comm.h 3.31
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * update hashtable. test=develop
    
    * update. test=develop
    
    * Adapt XPUPS - update by kp compilation  - 9th version - 4.1
    
    * update hashtable. test=develop
    
    * fix. test=develop
    
    * update hashtable 4.1
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * Adapt XPUPS - update by kp compilation  - 10th version - 4.1
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * update. test=develop
    
    * modify by compilation 4.1
    
    * update. test=develop
    
    * update. test=develop
    
    * fix. test=develop
    
    * modify by compilation 4.1
    
    * update. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * modify by compilation 4.1
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * modify by compilation 4.1 19:30
    
    * fix. test=develop
    
    * update ps_gpu_wrapper.kps 4.1
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * Adapt XPUPS - update by kp compilation  - 11th version - 4.1
    
    * fix. test=develop
    
    * Adapt XPUPS - update by kp compilation  - 12nd version - 4.2
    
    * fix. test=develop
    
    * fix. test=develop
    
    * modify by compilation 4.2
    
    * 4.2 update
    
    * fix. test=develop
    
    * template init. test=develop
    
    * update 4.6
    
    * fix. test=develop
    
    * template init. test=develop
    
    * 4.6 modify by compilation
    
    * hashtable template init. test=develop
    
    * hashtable template init. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=devlop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=devlop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * Adapt XPUPS - update by kp compilation  - 13nd version - 4.7
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * 4.11 update
    
    * fix. test=develop
    
    * fix. test=develop
    
    * 4.11 update
    
    * update by pre-commit
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * 4.12 update
    
    * fix. test=develop
    
    * Adapt XPUPS - update by kp compilation  - 14th version - 4.13
    
    * 4.13 update
    
    * 4.14 update
    
    * 4.14 update
    
    * 4.14 update
    
    * 4.14 modify by merged latest compilation
    
    * retry CI 4.14
    
    * 4.15 pass static check
    
    * 4.15 modify by gpups CI
    
    * 3.16 update by gpups CI - modify ps_gpu_wrapper.h
    
    * 4.16 update
    
    * 4.16 pass xpu compile
    
    * 4.16 retry CI
    
    * 4.16 update
    Co-authored-by: Nzmxdream <zhangminxu01@baidu.com>
    
    * modify ps_gpu_wrapper.cc
    
    * update
    
    * Adapt BKCL comm for XPUPS (#42168)
    
    * Adapt XPUPS - 1st version - 3.24
    
    * Adapt XPUPS - update XPU PushSparse -  2nd version - 3.24
    
    * Adapt XPUPS - add XPU PullSparseOp - 3nd version - 3.25
    
    * refactor heter comm kernel
    
    * update. test=develop
    
    * Adapt XPUPS - modify by compilation - 4th version - 3.27
    
    * update calc_shard_offset. test=develop
    
    * update xpu kernel. test=develop
    
    * update args of calc_shard_offset
    
    * update. test=develop
    
    * remove customGradMerger
    
    * update. test=develop
    
    * heter_comm update
    
    * heter_comm update
    
    * update calc_shard_offset. test=develop
    
    * heter_comm update
    
    * update args of calc_shard_offset
    
    * update. test=develop
    
    * remove customGradMerger
    
    * update. test=develop
    
    * fix. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update optimizer kernel
    
    * Adapt XPUPS - use WITH_XPU_KP and modify wrapper kernel function - 5th version - 3.30
    
    * update. test=develop
    
    * update pslib.cmake
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * Adapt XPUPS - modify by kp compilation  - 6th version - 3.30
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update optimizer kernel
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * update. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * used by minxu
    
    * update heter_comm_inl
    
    * fix. test=develop
    
    * Adapt XPUPS - modify by kp compilation  - 7th version - 3.30
    
    * fix. test=develop
    
    * add optimizer kernel. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * 3.31 update
    
    * Adapt XPUPS - update kp compilation path  - 8th version - 3.31
    
    * add optimizer kernel. test=develop
    
    * fix kunlun not support size_t. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix kunlun not support size_t. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * update heter_comm_kernel.kps 3.31
    
    * fix. test=develop
    
    * fix. test=develop
    
    * update heter_comm_kernel.kps 3.31
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * update heter_comm.h 3.31
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * update hashtable. test=develop
    
    * update. test=develop
    
    * Adapt XPUPS - update by kp compilation  - 9th version - 4.1
    
    * update hashtable. test=develop
    
    * fix. test=develop
    
    * update hashtable 4.1
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * Adapt XPUPS - update by kp compilation  - 10th version - 4.1
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * update. test=develop
    
    * modify by compilation 4.1
    
    * update. test=develop
    
    * update. test=develop
    
    * fix. test=develop
    
    * modify by compilation 4.1
    
    * update. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * modify by compilation 4.1
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * modify by compilation 4.1 19:30
    
    * fix. test=develop
    
    * update ps_gpu_wrapper.kps 4.1
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * Adapt XPUPS - update by kp compilation  - 11th version - 4.1
    
    * fix. test=develop
    
    * Adapt XPUPS - update by kp compilation  - 12nd version - 4.2
    
    * fix. test=develop
    
    * fix. test=develop
    
    * modify by compilation 4.2
    
    * 4.2 update
    
    * fix. test=develop
    
    * template init. test=develop
    
    * update 4.6
    
    * fix. test=develop
    
    * template init. test=develop
    
    * 4.6 modify by compilation
    
    * hashtable template init. test=develop
    
    * hashtable template init. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=devlop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=devlop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * Adapt XPUPS - update by kp compilation  - 13nd version - 4.7
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * 4.11 update
    
    * fix. test=develop
    
    * fix. test=develop
    
    * 4.11 update
    
    * update by pre-commit
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * fix. test=develop
    
    * 4.12 update
    
    * fix. test=develop
    
    * Adapt XPUPS - update by kp compilation  - 14th version - 4.13
    
    * 4.13 update
    
    * 4.14 update
    
    * 4.14 update
    
    * 4.14 update
    
    * 4.14 modify by merged latest compilation
    
    * retry CI 4.14
    
    * 4.15 pass static check
    
    * 4.15 modify by gpups CI
    
    * 3.16 update by gpups CI - modify ps_gpu_wrapper.h
    
    * 4.16 update
    
    * 4.16 pass xpu compile
    
    * 4.16 retry CI
    
    * 4.16 update
    
    * Adapt XPUPS - adapt BKCL comm for XPUPS - 4.24
    
    * update by compilation
    
    * Adapt XPUPS - register PSGPUTrainer for XPUPS - 4.25
    
    * update device_worker_factory
    Co-authored-by: Nzmxdream <zhangminxu01@baidu.com>
    
    * update
    
    * update CMakeLists
    Co-authored-by: Nzmxdream <zhangminxu01@baidu.com>
    0c68ae0c
device_worker.h 26.6 KB