用paddle实现ranknet的几点疑问
Created by: yuyy0823
用paddle实现ranknet,有一些不太懂的地方,请相关同学指教下。 1、ranknet中左右两个网络结构共享参数,是否一定要通过设置param_attr和bias_attr中的name来实现,如果不设置这两个参数的名字的话,实际训练过程中是训练了两套参数吗? 2、测试时发现,将网络中的参数通过param_attr和bias_attr设置name后(如下图),调用fluid.io.save_inference_model接口分别保存左右两个模型时,发现参数文件的md5sum是一样的,但是模型文件的md5sum值不一样,这是什么原因导致的呢?
def half_ranknet_2(input):
"""
rank_net
"""
def wide_layer():
"""
wide layer
"""
out = fluid.layers.fc(input=input, size=64, name='fc0',
param_attr=fluid.param_attr.ParamAttr(name="wide_w"),
bias_attr=fluid.param_attr.ParamAttr(name="wide_b"))
return out
def deep_layer():
"""
deep layer
"""
hidden1 = fluid.layers.fc(input=input, size=256, act='relu', name='fc1',
param_attr=fluid.param_attr.ParamAttr(name="deep_w1"),
bias_attr=fluid.param_attr.ParamAttr(name="deep_b1"))
hidden2 = fluid.layers.fc(input=hidden1, size=128, act='relu', name='fc2',
param_attr=fluid.param_attr.ParamAttr(name="deep_w2"),
bias_attr=fluid.param_attr.ParamAttr(name="deep_b2"))
#hidden3 = fluid.layers.fc(input=hidden2, size=128, act='relu', name='fc3')
out = fluid.layers.fc(input=hidden2, size=64, act='relu', name='fc3',
param_attr=fluid.param_attr.ParamAttr(name="deep_w3"),
bias_attr=fluid.param_attr.ParamAttr(name="deep_b3"))
return out
deep_out = deep_layer()
wide_out = wide_layer()
concat1 = fluid.layers.concat(input=[deep_out, wide_out], axis=1)
hidden3 = fluid.layers.fc(input=concat1, size=64, act='relu', name='fc4',
param_attr=fluid.param_attr.ParamAttr(name="deep_w4"),
bias_attr=fluid.param_attr.ParamAttr(name="deep_b4"))
hidden4 = fluid.layers.fc(input=hidden3, size=32, act='relu', name='fc5',
param_attr=fluid.param_attr.ParamAttr(name="deep_w5"),
bias_attr=fluid.param_attr.ParamAttr(name="deep_b5"))
pre = fluid.layers.fc(input=hidden4, size=1, name='out',
param_attr=fluid.param_attr.ParamAttr(name="out_w"),
bias_attr=fluid.param_attr.ParamAttr(name="out_b"))
pre_scale = fluid.layers.scale(pre, 1.0)
return pre_scale
3、如果将2的网络参数的param_attr和bias_attr设置去掉,再调用fluid.io.save_inference_model接口分别保存左右两个模型时,发现先保存左模型再保存右模型,和先保存右模型再保存左模型时,参数文件的名称不一样,这又是为什么呢? 先保存左模型,再保存右模型
fluid.io.save_inference_model(model_save_dir + "_left", [feats2.name], [predict2], exe, train_prog)
fluid.io.save_inference_model(model_save_dir + "_right", [feats1.name], [predict1], exe, train_prog)
得到的参数文件如下(左右两个参数文件夹下的文件一致):
fc0.b_0 fc0.w_0 fc1.b_0 fc1.w_0 fc2.b_0 fc2.w_0 fc3.b_0 fc3.w_0 fc4.b_0 fc4.w_0 fc5.b_0 fc5.w_0 __model__ out.b_0 out.w_0
先保存右模型,再保存左模型,得到的参数文件如下:
fc0.b_1 fc0.w_1 fc1.b_1 fc1.w_1 fc2.b_1 fc2.w_1 fc3.b_1 fc3.w_1 fc4.b_1 fc4.w_1 fc5.b_1 fc5.w_1 __model__ out.b_1 out.w_1