!409 rename sparse optimizer op and delete adamweightdecaydynamiclr

Merge pull request !409 from wangnan39/delete_adamweightdecaydynamiclr_and_rename_cpu_sparse_optimizer_op

!409 rename sparse optimizer op and delete adamweightdecaydynamiclr
Merge pull request !409 from wangnan39/delete_adamweightdecaydynamiclr_and_rename_cpu_sparse_optimizer_op
0ac525b5 · mindspore-ci-bot · Gitee · 327d7957 · 43aab742 · 0ac525b5
4 changed file
--- a/docs/source_en/operator_list.md
+++ b/docs/source_en/operator_list.md
@@ -64,7 +64,6 @@
 | [mindspore.nn.LazyAdam](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LazyAdam)                                            | Supported | Doing | Doing |optim/lazyadam
 | [mindspore.nn.Adam](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Adam)                                                    | Supported |Doing | Doing |optim/adam
 | [mindspore.nn.AdamWeightDecay](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.AdamWeightDecay)                              |  Supported | Supported | Doing |optim/adam
-| [mindspore.nn.AdamWeightDecayDynamicLR](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.AdamWeightDecayDynamicLR)            |  Supported | Supported | Doing |optim/adam
 | [mindspore.nn.Lamb](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Lamb)                                                    |  Supported | Supported | Doing |optim/lamb
 | [mindspore.nn.LARS](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LARS)                                                    |Supported |Doing | Doing |optim/lars
 | [mindspore.nn.Momentum](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Momentum)                                            |  Supported |  Supported |   Doing |optim/momentum
@@ -131,9 +130,10 @@
 | [mindspore.ops.operations.SparseApplyAdagrad](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdagrad)                 |    Supported  | Doing    |   Doing | nn_ops
 | [mindspore.ops.operations.SparseApplyAdagradV2](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdagradV2)                 |    Supported  | Doing    |   Doing | nn_ops
 | [mindspore.ops.operations.SparseApplyProximalAdagrad](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyProximalAdagrad)                 |    Supported  | Doing    |   Doing | nn_ops
+| [mindspore.ops.operations.FusedSparseProximalAdagrad](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FusedSparseProximalAdagrad)                 |    Doing  | Doing    |   Supported | nn_ops
 | [mindspore.ops.operations.ApplyProximalAdagrad](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyProximalAdagrad)                 |    Supported  | Doing    |   Doing | nn_ops
-| [mindspore.ops.operations.SparseApplyLazyAdam](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyLazyAdam)                  |  Doing  | Doing    |   Supported | nn_ops
-| [mindspore.ops.operations.SparseApplyAdam](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdam)                                 |  Doing  | Doing    |   Supported | nn_ops
+| [mindspore.ops.operations.FusedSparseLazyAdam](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FusedSparseLazyAdam)                  |  Doing  | Doing    |   Supported | nn_ops
+| [mindspore.ops.operations.FusedSparseAdam](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FusedSparseAdam)                                 |  Doing  | Doing    |   Supported | nn_ops
 | [mindspore.ops.operations.SmoothL1Loss](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SmoothL1Loss)                   |  Supported  | Doing  | Doing | nn_ops
 | [mindspore.ops.operations.SGD](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SGD)                                     |  Supported | Doing  | Doing | nn_ops
 | [mindspore.ops.operations.LayerNorm](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LayerNorm)                         |    Supported  | Supported  | Doing | nn_ops
@@ -271,6 +271,7 @@
 | [mindspore.ops.operations.GatherNd](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GatherNd)                           |  Supported | Doing  | Doing | array_ops
 | [mindspore.ops.operations.ApplyFtrl](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyFtrl)                           |  Supported | Supported  | Doing | array_ops
 | [mindspore.ops.operations.SparseApplyFtrl](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyFtrl)                           |  Supported | Doing  | Doing | array_ops
+| [mindspore.ops.operations.FusedSparseFtrl](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FusedSparseFtrl)                           |  Doing | Doing  | Supported | array_ops
 | [mindspore.ops.operations.SparseApplyFtrlV2](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyFtrlV2)                           |  Supported | Doing  | Doing | array_ops
 | [mindspore.ops.operations.ScatterNdUpdate](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterNdUpdate)             |  Supported | Doing  | Doing | array_ops
 | [mindspore.ops.operations.ScatterUpdate](https://www.mindspore.cn/api/en/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterUpdate)             |  Supported | Doing  | Doing | array_ops

--- a/docs/source_zh_cn/operator_list.md
+++ b/docs/source_zh_cn/operator_list.md
@@ -64,7 +64,6 @@
 | [mindspore.nn.LazyAdam](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LazyAdam)                                            |  Supported |Doing | Doing |optim/lazyadam
 | [mindspore.nn.Adam](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Adam)                                                    |  Supported |Doing | Doing |optim/adam
 | [mindspore.nn.AdamWeightDecay](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.AdamWeightDecay)                              |  Supported | Supported | Doing |optim/adam
-| [mindspore.nn.AdamWeightDecayDynamicLR](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.AdamWeightDecayDynamicLR)            |  Supported | Supported | Doing |optim/adam
 | [mindspore.nn.Lamb](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Lamb)                                                    |  Supported | Supported | Doing |optim/lamb
 | [mindspore.nn.LARS](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.LARS)                                                    |Supported |Doing | Doing |optim/lars
 | [mindspore.nn.Momentum](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.nn.html#mindspore.nn.Momentum)                                            |  Supported |  Supported |   Doing |optim/momentum
@@ -131,9 +130,10 @@
 | [mindspore.ops.operations.SparseApplyAdagrad](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdagrad)                 |    Supported  | Doing    |   Doing | nn_ops
 | [mindspore.ops.operations.SparseApplyAdagradV2](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdagradV2)                 |    Supported  | Doing    |   Doing | nn_ops
 | [mindspore.ops.operations.SparseApplyProximalAdagrad](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyProximalAdagrad)                 |    Supported  | Doing    |   Doing | nn_ops
+| [mindspore.ops.operations.FusedSparseProximalAdagrad](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FusedSparseProximalAdagrad)                 |    Doing  | Doing    |   Supported | nn_ops
 | [mindspore.ops.operations.ApplyProximalAdagrad](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyProximalAdagrad)                 |    Supported  | Doing    |   Doing | nn_ops
-| [mindspore.ops.operations.SparseApplyLazyAdam](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyLazyAdam)     |  Doing |  Doing  |  Supported | nn_ops
-| [mindspore.ops.operations.SparseApplyAdam](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyAdam)             |  Doing |  Doing  |  Supported | nn_ops
+| [mindspore.ops.operations.FusedSparseLazyAdam](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FusedSparseLazyAdam)     |  Doing |  Doing  |  Supported | nn_ops
+| [mindspore.ops.operations.FusedSparseAdam](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FusedSparseAdam)             |  Doing |  Doing  |  Supported | nn_ops
 | [mindspore.ops.operations.SmoothL1Loss](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SmoothL1Loss)                   |  Supported  | Doing  | Doing | nn_ops
 | [mindspore.ops.operations.SGD](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SGD)                                     |  Supported  | Doing  | Doing | nn_ops
 | [mindspore.ops.operations.LayerNorm](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.LayerNorm)                         |    Supported  | Supported  | Doing | nn_ops
@@ -271,6 +271,7 @@
 | [mindspore.ops.operations.GatherNd](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.GatherNd)                           |  Supported | Doing  | Doing | array_ops
 | [mindspore.ops.operations.ApplyFtrl](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ApplyFtrl)                           |  Supported | Supported  | Doing | array_ops
 | [mindspore.ops.operations.SparseApplyFtrl](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyFtrl)                           |  Supported | Doing  | Doing | array_ops
+| [mindspore.ops.operations.FusedSparseFtrl](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.FusedSparseFtrl)                           |  Doing | Doing  | Supported | array_ops
 | [mindspore.ops.operations.SparseApplyFtrlV2](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.SparseApplyFtrlV2)                           |  Supported | Doing  | Doing | array_ops
 | [mindspore.ops.operations.ScatterNdUpdate](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterNdUpdate)             |  Supported | Doing  | Doing | array_ops
 | [mindspore.ops.operations.ScatterUpdate](https://www.mindspore.cn/api/zh-CN/master/api/python/mindspore/mindspore.ops.operations.html#mindspore.ops.operations.ScatterUpdate)             |  Supported | Doing  | Doing | array_ops

--- a/tutorials/source_en/advanced_use/host_device_training.md
+++ b/tutorials/source_en/advanced_use/host_device_training.md
@@ -107,11 +107,11 @@ Search `EmbeddingLookup` in `test_deep0.log`, the following can be found:

 showing the running time of `EmbeddingLookup` on the host.

-Search `SparseApplyFtrl` and `SparseApplyLazyAdam` in `test_deep0.log`, the following can be found:
+Search `FusedSparseFtrl` and `FusedSparseLazyAdam` in `test_deep0.log`, the following can be found:

 ```
-[INFO] DEVICE(109904,python3.7):2020-06-27-12:42:35.422.963 [mindspore/ccsrc/device/cpu/cpu_kernel_runtime.cc:324] Run] cpu kernel: Default/optimizer_w-FTRL/SparseApplyFtrl-op299 costs 54492 us.
-[INFO] DEVICE(109904,python3.7):2020-06-27-12:42:35.565.953 [mindspore/ccsrc/device/cpu/cpu_kernel_runtime.cc:324] Run] cpu kernel: Default/optimizer_d-LazyAdam/SparseApplyLazyAdam-op300 costs 142865 us.
+[INFO] DEVICE(109904,python3.7):2020-06-27-12:42:35.422.963 [mindspore/ccsrc/device/cpu/cpu_kernel_runtime.cc:324] Run] cpu kernel: Default/optimizer_w-FTRL/FusedSparseFtrl-op299 costs 54492 us.
+[INFO] DEVICE(109904,python3.7):2020-06-27-12:42:35.565.953 [mindspore/ccsrc/device/cpu/cpu_kernel_runtime.cc:324] Run] cpu kernel: Default/optimizer_d-LazyAdam/FusedSparseLazyAdam-op300 costs 142865 us.
 ```

 showing the running time of two optimizers on the host.
--- a/tutorials/source_zh_cn/advanced_use/host_device_training.md
+++ b/tutorials/source_zh_cn/advanced_use/host_device_training.md
@@ -100,11 +100,11 @@ epoch: 1 step: 10, wide_loss is 0.566089, deep_loss is 0.6884129
 ```

 表示`EmbeddingLookup`在主机端的执行时间。
-继续在`test_deep0.log`搜索关键字`SparseApplyFtrl`和`SparseApplyLazyAdam`，可找到如下信息：
+继续在`test_deep0.log`搜索关键字`FusedSparseFtrl`和`FusedSparseLazyAdam`，可找到如下信息：

 ```
-[INFO] DEVICE(109904,python3.7):2020-06-27-12:42:35.422.963 [mindspore/ccsrc/device/cpu/cpu_kernel_runtime.cc:324] Run] cpu kernel: Default/optimizer_w-FTRL/SparseApplyFtrl-op299 costs 54492 us.
-[INFO] DEVICE(109904,python3.7):2020-06-27-12:42:35.565.953 [mindspore/ccsrc/device/cpu/cpu_kernel_runtime.cc:324] Run] cpu kernel: Default/optimizer_d-LazyAdam/SparseApplyLazyAdam-op300 costs 142865 us.
+[INFO] DEVICE(109904,python3.7):2020-06-27-12:42:35.422.963 [mindspore/ccsrc/device/cpu/cpu_kernel_runtime.cc:324] Run] cpu kernel: Default/optimizer_w-FTRL/FusedSparseFtrl-op299 costs 54492 us.
+[INFO] DEVICE(109904,python3.7):2020-06-27-12:42:35.565.953 [mindspore/ccsrc/device/cpu/cpu_kernel_runtime.cc:324] Run] cpu kernel: Default/optimizer_d-LazyAdam/FusedSparseLazyAdam-op300 costs 142865 us.
 ```

 表示两个优化器在主机端的执行时间。