optimizer minimize loss时报错,无具体错误信息
Created by: hxk11111
版本、环境信息: 1)PaddlePaddle版本:1.6.2 2)python:3.6.3
训练信息 1)单机单卡
total_loss = self.alpha * losses[0] + self.beta * losses[1] + self.gamma * losses[2]
logging.info("Shape of total_loss: {}".format(total_loss.shape))
inference_program = fluid.default_main_program().clone(for_test=True)
optimizer = fluid.optimizer.Adam(learning_rate=self.learning_rate)
optimizer.minimize(total_loss)
在minimize total loss时报错
I0106 22:17:22.031168 64214 op_desc.cc:685] CompileTime infer shape on sum
I0106 22:17:22.031216 64214 op_desc.cc:685] CompileTime infer shape on expand_grad
I0106 22:17:22.031261 64214 op_desc.cc:685] CompileTime infer shape on unsqueeze2_grad
I0106 22:17:22.031307 64214 op_desc.cc:685] CompileTime infer shape on gather_nd_grad
I0106 22:17:22.031350 64214 op_desc.cc:685] CompileTime infer shape on cast
Traceback (most recent call last):
File "models/train.py", line 158, in <module>
train()
File "models/train.py", line 56, in train
accuracies, inference_program = gnn_model.train_net()
File "/root/paddle/GNN-Table-Parsing/models/gnn_model.py", line 349, in train_net
optimizer.minimize(total_loss)
File "<decorator-gen-36>", line 2, in minimize
File "/opt/conda/lib/python3.6/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
return wrapped_func(*args, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/paddle/fluid/dygraph/base.py", line 78, in __impl__
return func(*args, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/paddle/fluid/optimizer.py", line 678, in minimize
no_grad_set=no_grad_set)
File "/opt/conda/lib/python3.6/site-packages/paddle/fluid/optimizer.py", line 551, in backward
no_grad_set, callbacks)
File "/opt/conda/lib/python3.6/site-packages/paddle/fluid/backward.py", line 1085, in append_backward
_append_backward_vars_(root_block, fwd_op_num, grad_to_var, grad_info_map)
File "/opt/conda/lib/python3.6/site-packages/paddle/fluid/backward.py", line 891, in _append_backward_vars_
op_desc.infer_shape(block.desc)
paddle.fluid.core_avx.EnforceNotMet:
--------------------------------------------
C++ Call Stacks (More useful to developers):
--------------------------------------------
0 std::string paddle::platform::GetTraceBackString<char const*>(char const*&&, char const*, int)
1 paddle::platform::EnforceNotMet::EnforceNotMet(std::__exception_ptr::exception_ptr, char const*, int)
2 paddle::operators::CastOpInferShape::operator()(paddle::framework::InferShapeContext*) const
3 std::_Function_handler<void (paddle::framework::InferShapeContext*), paddle::framework::details::OpInfoFiller<paddle::operators::CastOpInferShape, (paddle::framework::details::OpInfoFillType)4>::operator()(char const*, paddle::framework::OpInfo*) const::{lambda(paddle::framework::InferShapeContext*)#1}>::_M_invoke(std::_Any_data const&, paddle::framework::InferShapeContext*)
4 paddle::framework::OpDesc::InferShape(paddle::framework::BlockDesc const&) const
----------------------
Error Message Summary:
----------------------
Error: The input of cast op must be set at (/paddle/paddle/fluid/operators/cast_op.cc:44)