Skip to content

  • 体验新版
    • 正在加载...
  • 登录
  • PaddlePaddle
  • PaddleDetection
  • Issue
  • #990

P
PaddleDetection
  • 项目概览

PaddlePaddle / PaddleDetection
大约 2 年 前同步成功

通知 708
Star 11112
Fork 2696
  • 代码
    • 文件
    • 提交
    • 分支
    • Tags
    • 贡献者
    • 分支图
    • Diff
  • Issue 184
    • 列表
    • 看板
    • 标记
    • 里程碑
  • 合并请求 40
  • Wiki 0
    • Wiki
  • 分析
    • 仓库
    • DevOps
  • 项目成员
  • Pages
P
PaddleDetection
  • 项目概览
    • 项目概览
    • 详情
    • 发布
  • 仓库
    • 仓库
    • 文件
    • 提交
    • 分支
    • 标签
    • 贡献者
    • 分支图
    • 比较
  • Issue 184
    • Issue 184
    • 列表
    • 看板
    • 标记
    • 里程碑
  • 合并请求 40
    • 合并请求 40
  • Pages
  • 分析
    • 分析
    • 仓库分析
    • DevOps
  • Wiki 0
    • Wiki
  • 成员
    • 成员
  • 收起侧边栏
  • 动态
  • 分支图
  • 创建新Issue
  • 提交
  • Issue看板
已关闭
开放中
Opened 6月 28, 2020 by saxon_zh@saxon_zhGuest

operator < generate_proposals > error: The index of gather_op should not be empty when the index's rank is 1

Created by: flishwang

使用cascade_rcnn_r200训练的模型,进行推理时,如果test_proposals的参数min_size设为非0,在个别图片上会报错误。

请问这个bug应该在这里反馈,还是在PaddlePaddle的issues里反馈?

具体报错的日志如下:

2020-06-28 09:55:50,943-INFO: loading roidb 2012_debug 2020-06-28 09:57:29,972-INFO: load roidb from scope 2012_debug 2020-06-28 09:57:29,972-INFO: finish loading roidbs, total num = 500 2020-06-28 09:57:29,972-INFO: set max batches to 0 2020-06-28 09:57:29,973-INFO: places would be ommited when DataLoader is not iterable W0628 09:57:30.245823 4165090 device_context.cc:252] Please NOTE: device: 4, CUDA Capability: 70, Driver API Version: 10.1, Runtime API Version: 10.0 W0628 09:57:31.638833 4165090 device_context.cc:260] device: 4, cuDNN Version: 7.6. 2020-06-28 10:00:02,147-INFO: Test iter 0 2020-06-28 10:02:24,172-INFO: Test iter 100 2020-06-28 10:04:39,244-INFO: Test iter 200 /home/bwang/anaconda3/envs/paddle/lib/python3.7/site-packages/paddle/fluid/executor.py:1070: UserWarning: The following exception is not an EOF exception. "The following exception is not an EOF exception.") Traceback (most recent call last): File "tools/eval.py", line 243, in main() File "tools/eval.py", line 180, in main sub_eval_prog, sub_keys, sub_values, resolution) File "/home/bwang/projects/sniper-paddle/ppdet/utils/eval_utils.py", line 134, in eval_run return_merged=False) File "/home/bwang/anaconda3/envs/paddle/lib/python3.7/site-packages/paddle/fluid/executor.py", line 1071, in run six.reraise(*sys.exc_info()) File "/home/bwang/anaconda3/envs/paddle/lib/python3.7/site-packages/six.py", line 703, in reraise raise value File "/home/bwang/anaconda3/envs/paddle/lib/python3.7/site-packages/paddle/fluid/executor.py", line 1066, in run return_merged=return_merged) File "/home/bwang/anaconda3/envs/paddle/lib/python3.7/site-packages/paddle/fluid/executor.py", line 1167, in _run_impl return_merged=return_merged) File "/home/bwang/anaconda3/envs/paddle/lib/python3.7/site-packages/paddle/fluid/executor.py", line 879, in _run_parallel tensors = exe.run(fetch_var_names, return_merged)._move_to_list() paddle.fluid.core_avx.EnforceNotMet:


C++ Call Stacks (More useful to developers):

0 std::string paddle::platform::GetTraceBackString<std::string const&>(std::string const&, char const*, int) 1 paddle::platform::EnforceNotMet::EnforceNotMet(std::string const&, char const*, int) 2 void paddle::operators::GPUGather<float, int>(paddle::platform::DeviceContext const&, paddle::framework::Tensor const&, paddle::framework::Tensor const&, paddle::framework::Tensor*) 3 paddle::operators::CUDAGenerateProposalsKernel<paddle::platform::CUDADeviceContext, float>::Compute(paddle::framework::ExecutionContext const&) const 4 std::_Function_handler<void (paddle::framework::ExecutionContext const&), paddle::framework::OpKernelRegistrarFunctor<paddle::platform::CUDAPlace, false, 0ul, paddle::operators::CUDAGenerateProposalsKernel<paddle::platform::CUDADeviceContext, float> >::operator()(char const*, char const*, int) const::{lambda(paddle::framework::ExecutionContext const&)#1}>::_M_invoke(std::_Any_data const&, paddle::framework::ExecutionContext const&) 5 paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&, paddle::framework::RuntimeContext*) const 6 paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&) const 7 paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, paddle::platform::Place const&) 8 paddle::framework::details::ComputationOpHandle::RunImpl() 9 paddle::framework::details::FastThreadedSSAGraphExecutor::RunOpSync(paddle::framework::details::OpHandleBase*) 10 paddle::framework::details::FastThreadedSSAGraphExecutor::RunOp(paddle::framework::details::OpHandleBase*, std::shared_ptr<paddle::framework::BlockingQueue > const&, unsigned long*) 11 std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result, std::__future_base::_Result_base::_Deleter>, void> >::_M_invoke(std::_Any_data const&) 12 std::__future_base::_State_base::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>&, bool&) 13 ThreadPool::ThreadPool(unsigned long)::{lambda()#1}::operator()() const


Python Call Stacks (More useful to users):

File "/home/bwang/anaconda3/envs/paddle/lib/python3.7/site-packages/paddle/fluid/framework.py", line 2610, in append_op attrs=kwargs.get("attrs", None)) File "/home/bwang/anaconda3/envs/paddle/lib/python3.7/site-packages/paddle/fluid/layer_helper.py", line 43, in append_op return self.main_program.current_block().append_op(*args, **kwargs) File "/home/bwang/anaconda3/envs/paddle/lib/python3.7/site-packages/paddle/fluid/layers/detection.py", line 2844, in generate_proposals 'RpnRoisLod': rpn_rois_lod File "/home/bwang/projects/sniper-paddle/ppdet/core/workspace.py", line 150, in partial_apply return op(*args, **kwargs_) File "/home/bwang/projects/sniper-paddle/ppdet/modeling/anchor_heads/rpn_head.py", line 438, in _get_single_proposals variances=self.anchor_var) File "/home/bwang/projects/sniper-paddle/ppdet/modeling/anchor_heads/rpn_head.py", line 462, in get_proposals fpn_feat, im_info, lvl, mode) File "/home/bwang/projects/sniper-paddle/ppdet/modeling/architectures/cascade_rcnn_cls_aware.py", line 197, in build_multi_scale body_feats, im_info, mode="test") File "/home/bwang/projects/sniper-paddle/ppdet/modeling/architectures/cascade_rcnn_cls_aware.py", line 330, in eval return self.build_multi_scale(feed_vars) File "tools/eval.py", line 105, in main fetches = model.eval(feed_vars, multi_scale_test) File "tools/eval.py", line 243, in main()


Error Message Summary:

Error: The index of gather_op should not be empty when the index's rank is 1. [Hint: Expected index.dims()[0] > 0, but received index.dims()[0]:0 <= 0:0.] at (/paddle/paddle/fluid/operators/gather.cu.h:82) [operator < generate_proposals > error] terminate called without an active exception W0628 10:05:48.434573 4165181 init.cc:216] Warning: PaddlePaddle catches a failure signal, it may not work properly W0628 10:05:48.434607 4165181 init.cc:218] You could check whether you killed PaddlePaddle thread/process accidentally or report the case to PaddlePaddle W0628 10:05:48.434615 4165181 init.cc:221] The detail failure signal is:

W0628 10:05:48.434626 4165181 init.cc:224] *** Aborted at 1593309948 (unix time) try "date -d @1593309948" if you are using GNU date *** W0628 10:05:48.439401 4165181 init.cc:224] PC: @ 0x0 (unknown) W0628 10:05:48.560981 4165181 init.cc:224] *** SIGABRT (@0x2716003f8de2) received by PID 4165090 (TID 0x7fe905fb8700) from PID 4165090; stack trace: *** W0628 10:05:48.565740 4165181 init.cc:224] @ 0x7fe96d5c4390 (unknown) W0628 10:05:48.570439 4165181 init.cc:224] @ 0x7fe96d21e428 gsignal W0628 10:05:48.575121 4165181 init.cc:224] @ 0x7fe96d22002a abort W0628 10:05:48.909902 4165181 init.cc:224] @ 0x7fe8bdcd084a __gnu_cxx::__verbose_terminate_handler() W0628 10:05:48.912384 4165181 init.cc:224] @ 0x7fe8bdccef47 __cxxabiv1::__terminate() W0628 10:05:48.920951 4165181 init.cc:224] @ 0x7fe8bdccef7d std::terminate() W0628 10:05:48.923445 4165181 init.cc:224] @ 0x7fe8bdccec5a __gxx_personality_v0 W0628 10:05:48.927270 4165181 init.cc:224] @ 0x7fe96a790b97 _Unwind_ForcedUnwind_Phase2 W0628 10:05:48.931037 4165181 init.cc:224] @ 0x7fe96a790e7d _Unwind_ForcedUnwind W0628 10:05:48.934270 4165181 init.cc:224] @ 0x7fe96d5c3070 __GI___pthread_unwind W0628 10:05:48.937424 4165181 init.cc:224] @ 0x7fe96d5bb845 __pthread_exit W0628 10:05:49.322953 4165181 init.cc:224] @ 0x55eadd5ea059 PyThread_exit_thread W0628 10:05:49.323395 4165181 init.cc:224] @ 0x55eadd46fc10 PyEval_RestoreThread.cold.799 W0628 10:05:49.345948 4165181 init.cc:224] @ 0x7fe88283f5c9 pybind11::gil_scoped_release::~gil_scoped_release()

指派人
分配到
无
里程碑
无
分配里程碑
工时统计
无
截止日期
无
标识: paddlepaddle/PaddleDetection#990
渝ICP备2023009037号

京公网安备11010502055752号

网络110报警服务 Powered by GitLab CE v13.7
开源知识
Git 入门 Pro Git 电子书 在线学 Git
Markdown 基础入门 IT 技术知识开源图谱
帮助
使用手册 反馈建议 博客
《GitCode 隐私声明》 《GitCode 服务条款》 关于GitCode
Powered by GitLab CE v13.7