gru_unit的初始化方式
Created by: nizihan
请问如果没有指定initializer的话,gru_unit的初始化方式是Xavier吗?相比于使用默认值,我显示的指定initializer为Xavier的时候,实验效果会有提升。请问这是什么原因?
使用默认值:
clip = fluid.clip.GradientClipByNorm(clip_norm=2.0)
clip_attr = fluid.ParamAttr(gradient_clip=clip)
h, _, _ = fluid.layers.gru_unit( input=decoder_inputs, hidden = hidden_mem, param_attr = clip_attr, bias_attr = clip_attr, size=args.decoder_size * 3)
显示的指定initializer为Xavier:
gru_clip = fluid.clip.GradientClipByNorm(clip_norm=2.0)
gru_attr = fluid.ParamAttr( gradient_clip = gru_clip, initializer = fluid.initializer.Xavier())
h, _, _ = fluid.layers.gru_unit( input=decoder_inputs, hidden = hidden_mem, param_attr = gru_attr, bias_attr = clip_attr, size=args.decoder_size * 3)