load_persistables 重新加载模型后训练,报错MomentumOp的维度不一致
Created by: xiangyubo
- 版本、环境信息: 1)PaddlePaddle版本:1.2 4)系统环境:python 2.7
- 训练信息 1)单机/单卡 我训练的过程中用 save_persistables 保存的模型,然后程序停止了,我第二次运行程序的时候,我用 load_persistables 方式加载模型,然后训练的时候报错。说优化器MomentumOp的维度不一致
EnforceNotMetTraceback (most recent call last)<ipython-input-1-0317d6de6533> in <module>()
649 total_batch_count += 1
650 t1 = time.time()
--> 651 loss = exe.run(train_program, feed=feeder.feed(data), fetch_list=train_fetch_list)
652 period = time.time() - t1
653 loss = np.mean(np.array(loss))
/opt/conda/envs/py27-paddle1.2.0/lib/python2.7/site-packages/paddle/fluid/executor.pyc in run(self, program, feed, fetch_list, feed_var_name, fetch_var_name, scope, return_numpy, use_program_cache)
470
471 self._feed_data(program, feed, feed_var_name, scope)
--> 472 self.executor.run(program.desc, scope, 0, True, True)
473 outs = self._fetch_data(fetch_list, fetch_var_name, scope)
474 if return_numpy:
EnforceNotMet: Enforce failed. Expected param_dim == ctx->GetInputDim("Velocity"), but received param_dim:32 != ctx->GetInputDim("Velocity"):21, 1.
Param and Velocity of MomentumOp should have the same dimension. at [/paddle/paddle/fluid/operators/optimizers/momentum_op.h:64]
PaddlePaddle Call Stacks:
0 0x7f70fa297826p paddle::platform::EnforceNotMet::EnforceNotMet(std::__exception_ptr::exception_ptr, char const*, int) + 486
1 0x7f70fb30ade9p paddle::operators::MomentumOp::InferShape(paddle::framework::InferShapeContext*) const + 1945