对paddlehub中strategy.py中的几个问题
Created by: mrhan36
注:以下问题均为在paddlehub中实施迁移学习,进行图像分类时遇到的问题,主要是针对strategy.py的理解上的困惑,谢谢!
#问题1:以下scheduler部分,代码的功能是设置对末3层(块)进行finetune,这样理解对吗? #问题2:如果设置scheduler[gradual_unfreeze][blocks]=0,则不会实施多层finetune,discriminative也会无效,这样理解对吗? #问题3:如果设置scheduler[gradual_unfreeze][blocks]=3,但scheduler[disciminative][blocks]=0,则会对末3层进行finetune,但discriminative无效(各层学习率相同),这样理解对吗? scheduler = { "discriminative": { "blocks": 3, "factor": 2.6 },
"gradual_unfreeze": { "blocks":3, "params_layer": None, } }
#问题4:以下两行代码希望设置L2正则化,请问应该设置regularization[L2]=1e-3,还是应该设置regularization[weight_decay]=1e-3 ?
regularization = {"L2": 1e-3}
regularization = {"weight_decay": 1e-3}
#问题5:如果不设置clip参数,默认各层均不会有batchnorm,这样理解对吗? #问题6:设置clip后,会对每层normalize还是只对unfreezing层进行normalize? #问题7:是不是globalnorm更适合用于迁移学习,norm适合非迁移学习? clip = {"GlobalNorm": 1.0} strategy = hub.CombinedStrategy( optimizer_name="adam", # "sgd","Adagrad","adamax","decayedadagrad","ftrl","larsmomentum","momentum","decayedadagrad","rmsprop" learning_rate=1e-4, scheduler = scheduler, regularization = regularization, clip = clip )
config = hub.RunConfig(
log_interval=10,
use_cuda=True,
num_epoch=30,
checkpoint_dir=ckpt_dirckpt_dir,
batch_size=32,
eval_interval=50,
strategy=strategy
)
问题比较多,麻烦您了,谢谢!
参考资料: https://github.com/PaddlePaddle/PaddleHub/wiki/PaddleHub-API:-Strategy https://github.com/PaddlePaddle/PaddleHub/blob/a2cda1381ace47a8053280bc8f9dda043e25bd27/paddlehub/finetune/strategy.py