关于batch_norm layer的参数自命名
Created by: hiahiahu
因为要共享参数,对batch_norm layer,配置如下: Layer(name=name + 'batch_norm1', type='batch_norm', active_type="relu", use_global_stats=True, bias=Bias(parameter_name = parse_parameter(name) +'BN1.bias', initial_std=0.02, is_static=static, learning_rate=lr), inputs=Input(name + 'conv1', parameter_name = parse_parameter(name) +'BN1.w', initial_std=0.02, is_static=static, learning_rate=lr, image=Image(channels=size, img_size=1)), )
训练日志中 I1219 22:16:30.714959 18400 TrainerInternal.cpp:207] title_block_2_t3_BN1.w avg_abs_val=0.0155404 max_val=0.0588142 avg_abs_grad=0.103073 max_grad=0.509655 I1219 22:16:30.715075 18400 TrainerInternal.cpp:207] _title_block_2_t3_batch_norm1.w1 avg_abs_val=0.00608151 max_val=0.0277185 avg_abs_grad=0 max_grad=0 I1219 22:16:30.715212 18400 TrainerInternal.cpp:207] _title_block_2_t3_batch_norm1.w2 avg_abs_val=3.72902e-05 max_val=0.00016382 avg_abs_grad=0 max_grad=0 I1219 22:16:30.715330 18400 TrainerInternal.cpp:207] title_block_2_t3_BN1.bias avg_abs_val=0.017024 max_val=0.0620825 avg_abs_grad=0.0651094 max_grad=0.299386
其中两个 _title 开头的参数 怎么自己设置命名? 好像没有lstm里面的para_prefix 这么个参数? 看config_parse 这两个layer似乎是在这里添加进去的 use_gpu = bool(int(g_command_config_args.get("use_gpu", 0))) is_shared = True if not use_gpu else False for i in xrange(2): inputs.append(Input(inputs[0].input_layer_name, initial_std=0.0, initial_mean=0.0, is_static=True, is_shared=is_shared, ))
这里设置的 is_static, 为什么日志文件中的average_val和max_val不是0 ? 求解答!辛苦~