未验证 提交 b47b5521 编写于 作者: D Double_V 提交者: GitHub

Merge pull request #184 from tink2123/add_cosine_decay

add cosine decay
...@@ -41,3 +41,7 @@ Optimizer: ...@@ -41,3 +41,7 @@ Optimizer:
base_lr: 0.0005 base_lr: 0.0005
beta1: 0.9 beta1: 0.9
beta2: 0.999 beta2: 0.999
decay:
function: cosine_decay
step_each_epoch: 20
total_epoch: 1000
...@@ -46,3 +46,18 @@ ...@@ -46,3 +46,18 @@
| img_set_dir | 数据集路径 | ./train_data | \ | | img_set_dir | 数据集路径 | ./train_data | \ |
| label_file_path | 数据标签路径 | ./train_data/rec_gt_train.txt| \ | | label_file_path | 数据标签路径 | ./train_data/rec_gt_train.txt| \ |
| infer_img | 预测图像文件夹路径 | ./infer_img | \| | infer_img | 预测图像文件夹路径 | ./infer_img | \|
## 配置文件 Optimizer 系列参数介绍
`rec_icdar15_train.yml` 为例
| 字段 | 用途 | 默认值 | 备注 |
| :---------------------: | :---------------------: | :--------------: | :--------------------: |
| function | 选择优化器 | pocr.optimizer,AdamDecay | 目前只支持Adam方式 |
| base_lr | 设置初始学习率 | 0.0005 | \ |
| beta1 | 设置一阶矩估计的指数衰减率 | 0.9 | \ |
| beta2 | 设置二阶矩估计的指数衰减率 | 0.999 | \ |
| decay | 是否使用decay | \ | \ |
| function(decay) | 设置decay方式 | cosine_decay | 目前只支持cosin_decay |
| step_each_epoch | 每个epoch包含多少次迭代 | 20 | 计算方式:total_image_num / (batch_size_per_card * card_size) |
| total_epoch | 总共迭代多少个epoch | 1000 | 与Global.epoch_num 一致 |
...@@ -46,3 +46,18 @@ Take `rec_chinese_reader.yml` as an example: ...@@ -46,3 +46,18 @@ Take `rec_chinese_reader.yml` as an example:
| img_set_dir | Image folder path | ./train_data | \ | | img_set_dir | Image folder path | ./train_data | \ |
| label_file_path | Groundtruth file path | ./train_data/rec_gt_train.txt| \ | | label_file_path | Groundtruth file path | ./train_data/rec_gt_train.txt| \ |
| infer_img | Result folder path | ./infer_img | \| | infer_img | Result folder path | ./infer_img | \|
## Introduction to Optimizer parameters of Configuration file
Take `rec_icdar15_train.yml` as an example:
| Parameter | Use | Default | None |
| :---------------------: | :---------------------: | :--------------: | :--------------------: |
| function | Select Optimizer function | pocr.optimizer,AdamDecay | Only support Adam |
| base_lr | Set the base lr | 0.0005 | \ |
| beta1 | Set the exponential decay rate for the 1st moment estimates | 0.9 | \ |
| beta2 | Set the exponential decay rate for the 2nd moment estimates | 0.999 | \ |
| decay | Whether to use decay | \ | \ |
| function(decay) | Set the decay function | cosine_decay | Only support cosine_decay |
| step_each_epoch | The number of steps in an epoch. | 20 | Calculation :total_image_num / (batch_size_per_card * card_size) |
| total_epoch | The number of epochs | 1000 | Consistent with Global.epoch_num |
...@@ -15,6 +15,9 @@ from __future__ import absolute_import ...@@ -15,6 +15,9 @@ from __future__ import absolute_import
from __future__ import division from __future__ import division
from __future__ import print_function from __future__ import print_function
import paddle.fluid as fluid import paddle.fluid as fluid
from ppocr.utils.utility import initial_logger
logger = initial_logger()
def AdamDecay(params, parameter_list=None): def AdamDecay(params, parameter_list=None):
...@@ -28,6 +31,18 @@ def AdamDecay(params, parameter_list=None): ...@@ -28,6 +31,18 @@ def AdamDecay(params, parameter_list=None):
base_lr = params['base_lr'] base_lr = params['base_lr']
beta1 = params['beta1'] beta1 = params['beta1']
beta2 = params['beta2'] beta2 = params['beta2']
if 'decay' in params:
params = params['decay']
decay_mode = params['function']
step_each_epoch = params['step_each_epoch']
total_epoch = params['total_epoch']
if decay_mode == "cosine_decay":
base_lr = fluid.layers.cosine_decay(
learning_rate=base_lr,
step_each_epoch=step_each_epoch,
epochs=total_epoch)
else:
logger.info("Only support Cosine decay currently")
optimizer = fluid.optimizer.Adam( optimizer = fluid.optimizer.Adam(
learning_rate=base_lr, learning_rate=base_lr,
beta1=beta1, beta1=beta1,
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册