优化器 运行出现未知错误,不清楚错误原因
Created by: biubiuyi
paddle1.7.2,cpu,单机训练 在使用sgd和adam是都出现相同错误,不知道问题在哪。 和这个问题类似:https://ai.baidu.com/forum/topic/show/957673,同样用到了layers.gather_nd 函数。 完整报错信息如下: Traceback (most recent call last): File "/Users/casfive-public/projects/NERCheck/614EvolveModel/train.py", line 372, in opt.minimize(loss) File "</Users/casfive-public/anaconda3/lib/python3.6/site-packages/decorator.py:decorator-gen-49>", line 2, in minimize File "/Users/casfive-public/anaconda3/lib/python3.6/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in impl return wrapped_func(*args, **kwargs) File "/Users/casfive-public/anaconda3/lib/python3.6/site-packages/paddle/fluid/dygraph/base.py", line 100, in impl return func(*args, **kwargs) File "/Users/casfive-public/anaconda3/lib/python3.6/site-packages/paddle/fluid/optimizer.py", line 793, in minimize no_grad_set=no_grad_set) File "/Users/casfive-public/anaconda3/lib/python3.6/site-packages/paddle/fluid/optimizer.py", line 642, in backward act_no_grad_set, callbacks) File "/Users/casfive-public/anaconda3/lib/python3.6/site-packages/paddle/fluid/backward.py", line 1264, in append_backward grad_info_map) File "/Users/casfive-public/anaconda3/lib/python3.6/site-packages/paddle/fluid/backward.py", line 969, in append_backward_vars op_desc.infer_shape(block.desc) paddle.fluid.core_avx.EnforceNotMet:
C++ Call Stacks (More useful to developers):
0 std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > paddle::platform::GetTraceBackString<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&>(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&&&, char const*, int) 1 paddle::operators::CastOp::InferShape(paddle::framework::InferShapeContext*) const 2 paddle::framework::OpDesc::InferShape(paddle::framework::BlockDesc const&) const 3 void pybind11::cpp_function::initialize<pybind11::cpp_function::cpp_function<void, paddle::framework::OpDesc, paddle::framework::BlockDesc const&, pybind11::name, pybind11::is_method, pybind11::sibling>(void (paddle::framework::OpDesc::)(paddle::framework::BlockDesc const&) const, pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&)::'lambda'(paddle::framework::OpDesc const, paddle::framework::BlockDesc const&), void, paddle::framework::OpDesc const*, paddle::framework::BlockDesc const&, pybind11::name, pybind11::is_method, pybind11::sibling>(void&&, paddle::framework::OpDesc ()(paddle::framework::BlockDesc const&), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&)::'lambda'(pybind11::detail::function_call&)::operator()(pybind11::detail::function_call&) const 4 pybind11::cpp_function::dispatcher(_object, _object*, _object*)
Error Message Summary:
NotFoundError: The input(X) of cast op must be set [Hint: Expected context->HasInput("X") == true, but received context->HasInput("X"):0 != true:1.] at (/home/teamcity/work/ef54dc8a5b211854/paddle/fluid/operators/cast_op.cc:66)
为使您的问题得到快速解决,在建立Issues前,请您先通过如下方式搜索是否有相似问题:【搜索issue关键字】【使用labels筛选】【官方文档】
如果您没有查询到相似问题,为快速解决您的提问,建立issue时请提供如下细节信息:
- 标题:简洁、精准概括您的问题,例如“Insufficient Memory xxx" ”
- 版本、环境信息: 1)PaddlePaddle版本:请提供您的PaddlePaddle版本号,例如1.1或CommitID 2)CPU:预测若用CPU,请提供CPU型号,MKL/OpenBlas/MKLDNN/等数学库使用情况 3)GPU:预测若用GPU,请提供GPU型号、CUDA和CUDNN版本号 4)系统环境:请您描述系统类型、版本,例如Mac OS 10.14,Python版本 注:您可以通过执行summary_env.py获取以上信息。
- 训练信息 1)单机/多机,单卡/多卡 2)显存信息 3)Operator信息
- 复现信息:如为报错,请给出复现环境、复现步骤
- 问题描述:请详细描述您的问题,同步贴出报错信息、日志、可复现的代码片段
Thank you for contributing to PaddlePaddle. Before submitting the issue, you could search issue in the github in case that there was a similar issue submitted or resolved before. If there is no solution,please make sure that this is a training issue including the following details: System information -PaddlePaddle version (eg.1.1)or CommitID -CPU: including CPUMKL/OpenBlas/MKLDNN version -GPU: including CUDA/CUDNN version -OS Platform (eg.Mac OS 10.14) -Other imformation: Distriuted training/informantion of operator/ Graphics card storage Note: You can get most of the information by running summary_env.py. To Reproduce Steps to reproduce the behavior Describe your current behavior Code to reproduce the issue Other info / logs