batch norm 预测时指定 use_global_stats=True,出现 CUDNN_STATUS_NOT_SUPPORTED 错误
Created by: lcy-seso
文本分类任务,想使用 cnn + batch norm,配置的片段如下:
Layer(name=name + 'context1',
type="mixed",
bias=False,
inputs=ContextProjection(input_name,
context_start=0,
context_length=window_size,
trainable_padding=False))
Layer(name=name + 'conv0',
type="mixed",
size=size,
active_type="linear",
bias=Bias(initial_std=1e-1,
initial_mean=0,
is_static=static,
learning_rate=lr),
inputs=[FullMatrixProjection(name + "context1",
initial_std=2e-2,
is_static=static,
learning_rate=lr)])
Layer(name=name + 'batch_norm0',
type='batch_norm',
active_type="relu",
use_global_stats=True,
bias=Bias(initial_mean=0.1, initial_std=0,
is_static=static, learning_rate=lr),
inputs=Input(name + 'conv0',
initial_mean=1.0,
initial_std=0.0,
is_static=static,
learning_rate=lr,
image=Image(channels=size, img_size=1)), )
使用 gpu 训练可以正常训练。 测试时,希望保持 use_global_stats=True,使用训练时存储下来的 mean 和 std 的working average , 但是报如下错误: