配置正常,训练报错
Created by: Irvingao
2020-05-19 12:30:19,695-INFO: 2000 samples in file /home/aistudio/PaddleDetection-release-0.2/dataset/voc/test.txt 2020-05-19 12:30:19,696-INFO: places would be ommited when DataLoader is not iterable W0519 12:30:20.471262 1536 device_context.cc:237] Please NOTE: device: 0, CUDA Capability: 70, Driver API Version: 9.2, Runtime API Version: 9.0 W0519 12:30:20.474319 1536 device_context.cc:245] device: 0, cuDNN Version: 7.3. 2020-05-19 12:30:21,901-INFO: Found /home/aistudio/.cache/paddle/weights/DarkNet53_pretrained 2020-05-19 12:30:21,901-INFO: Loading parameters from /home/aistudio/.cache/paddle/weights/DarkNet53_pretrained... 2020-05-19 12:30:21,901-WARNING: /home/aistudio/.cache/paddle/weights/DarkNet53_pretrained.pdparams not found, try to load model file saved with [ save_params, save_persistables, save_vars ] 2020-05-19 12:30:21,901-WARNING: /home/aistudio/.cache/paddle/weights/DarkNet53_pretrained.pdparams not found, try to load model file saved with [ save_params, save_persistables, save_vars ] 2020-05-19 12:30:23,542-INFO: 18000 samples in file /home/aistudio/PaddleDetection-release-0.2/dataset/voc/trainval.txt 2020-05-19 12:30:29,800-INFO: places would be ommited when DataLoader is not iterable I0519 12:30:30.583034 1536 parallel_executor.cc:440] The Program will be executed on CUDA using ParallelExecutor, 1 cards are used, so 1 programs are executed in parallel. I0519 12:30:30.684587 1536 build_strategy.cc:365] SeqOnlyAllReduceOps:0, num_trainers:1 I0519 12:30:30.801499 1536 parallel_executor.cc:307] Inplace strategy is enabled, when build_strategy.enable_inplace = True I0519 12:30:30.835758 1536 parallel_executor.cc:375] Garbage collection strategy is enabled, when FLAGS_eager_delete_tensor_gb = 0 /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/executor.py:782: UserWarning: The following exception is not an EOF exception. "The following exception is not an EOF exception.") Traceback (most recent call last): File "tools/train.py", line 323, in main() File "tools/train.py", line 233, in main outs = exe.run(compiled_train_prog, fetch_list=train_values) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/executor.py", line 783, in run six.reraise(*sys.exc_info()) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/six.py", line 693, in reraise raise value File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/executor.py", line 778, in run use_program_cache=use_program_cache) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/executor.py", line 843, in _run_impl return_numpy=return_numpy) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/executor.py", line 677, in _run_parallel tensors = exe.run(fetch_var_names)._move_to_list() paddle.fluid.core_avx.EnforceNotMet:
C++ Call Stacks (More useful to developers):
0 std::string paddle::platform::GetTraceBackString<std::string const&>(std::string const&, char const*, int) 1 paddle::platform::EnforceNotMet::EnforceNotMet(std::string const&, char const*, int) 2 paddle::operators::ReadOp::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&) const 3 paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, paddle::platform::Place const&) 4 paddle::framework::details::ComputationOpHandle::RunImpl() 5 paddle::framework::details::FastThreadedSSAGraphExecutor::RunOpSync(paddle::framework::details::OpHandleBase*) 6 paddle::framework::details::FastThreadedSSAGraphExecutor::RunOp(paddle::framework::details::OpHandleBase*, std::shared_ptr<paddle::framework::BlockingQueue > const&, unsigned long*) 7 std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result, std::__future_base::_Result_base::_Deleter>, void> >::_M_invoke(std::_Any_data const&) 8 std::__future_base::_State_base::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>&, bool&) 9 ThreadPool::ThreadPool(unsigned long)::{lambda()#1}::operator()() const
Python Call Stacks (More useful to users):
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/framework.py", line 2525, in append_op attrs=kwargs.get("attrs", None)) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/reader.py", line 733, in _init_non_iterable outputs={'Out': self._feed_list}) File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/reader.py", line 646, in init self._init_non_iterable() File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/reader.py", line 280, in from_generator iterable, return_list) File "/home/aistudio/PaddleDetection-release-0.2/ppdet/modeling/architectures/yolov3.py", line 152, in build_inputs iterable=iterable) if use_dataloader else None File "tools/train.py", line 115, in main feed_vars, train_loader = model.build_inputs(**inputs_def) File "tools/train.py", line 323, in main()
Error Message Summary:
Error: The fed Variable gt_bbox should have dimensions = 3, shape = [-1, 3, 4], but received fed shape [8, 50, 4] [Hint: Expected DimensionIsCompatibleWith(shapes[i], in_dims) == true, but received DimensionIsCompatibleWith(shapes[i], in_dims):0 != true:1.] at (/paddle/paddle/fluid/operators/reader/read_op.cc:133) [operator < read > error]