体验新版 GitCode,发现更多精彩内容 >>
Optimizers =================== DeepSpeed offers high-performance implementations of ``Adam`` optimizer on CPU; ``FusedAdam``, ``FusedLamb``, ``OnebitAdam``, ``OnebitLamb`` optimizers on GPU. Adam (CPU) ---------------------------- .. autoclass:: deepspeed.ops.adam.DeepSpeedCPUAdam FusedAdam (GPU) ---------------------------- .. autoclass:: deepspeed.ops.adam.FusedAdam FusedLamb (GPU) ---------------------------- .. autoclass:: deepspeed.ops.lamb.FusedLamb OneBitAdam (GPU) ---------------------------- .. autoclass:: deepspeed.runtime.fp16.onebit.adam.OnebitAdam OnebitLamb (GPU) ---------------------------- .. autoclass:: deepspeed.runtime.fp16.onebit.lamb.OnebitLamb