Created by: Sparrow0817

----------- Configuration Arguments ----------- batch_size: 8 class_num: 80 data_dir: dataset/coco dataset: coco2017 debug: False draw_thresh: 0.5 image_name: 000000000139.jpg image_path: ./dataset/coco/val2017 input_size: 608 label_smooth: True learning_rate: 0.001 max_iter: 500200 model_save_dir: checkpoints nms_posk: 100 nms_thresh: 0.45 nms_topk: 400 no_mixup_iter: 40000 pretrain: weights/darknet53 random_shape: True snapshot_iter: 2000 start_iter: 0 syncbn: True use_gpu: True use_multiprocess: True valid_thresh: 0.005 weights:

loading annotations into memory... Done (t=0.31s) creating index... index created! Load in 80 categories. Traceback (most recent call last): File "infer.py", line 80, in infer() File "infer.py", line 63, in infer feed=feeder.feed(data), File "/home/lwd/.local/lib/python3.6/site-packages/paddle/fluid/data_feeder.py", line 246, in feed ret_dict[each_name] = each_converter.done() File "/home/lwd/.local/lib/python3.6/site-packages/paddle/fluid/data_feeder.py", line 95, in done t.set(arr, self.place) paddle.fluid.core.EnforceNotMet: Enforce failed. Expected allocating <= available, but received allocating:5403593101 > available:5300944640. Insufficient GPU memory to allocation. at [/paddle/paddle/fluid/platform/gpu_info.cc:262] PaddlePaddle Call Stacks: 0 0x7eff92634228p void paddle::platform::EnforceNotMet::Initstd::string(std::string, char const*, int) + 360 1 0x7eff92634577p paddle::platform::EnforceNotMet::EnforceNotMet(std::string const&, char const*, int) + 87 2 0x7eff9437c706p paddle::platform::GpuMaxChunkSize() + 630 3 0x7eff943510a2p 4 0x7effd8c62827p 5 0x7eff9435074dp paddle::memory::legacy::GetGPUBuddyAllocator(int) + 109 6 0x7eff94351573p void* paddle::memory::legacy::Allocpaddle::platform::CUDAPlace(paddle::platform::CUDAPlace const&, unsigned long) + 35 7 0x7eff943519b5p paddle::memory::allocation::LegacyAllocator::AllocateImpl(unsigned long, paddle::memory::allocation::Allocator::Attr) + 389 8 0x7eff94376c6bp paddle::memory::allocation::Allocator::Allocate(unsigned long, paddle::memory::allocation::Allocator::Attr) + 27 9 0x7eff943455b3p paddle::memory::allocation::AllocatorFacade::Alloc(boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&, unsigned long, paddle::memory::allocation::Allocator::Attr) + 435 10 0x7eff943456d1p paddle::memory::allocation::AllocatorFacade::AllocShared(boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&, unsigned long, paddle::memory::allocation::Allocator::Attr) + 33 11 0x7eff93f835a0p paddle::memory::AllocShared(boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&, unsigned long, paddle::memory::allocation::Allocator::Attr) + 48 12 0x7eff9431796ap paddle::framework::Tensor::mutable_data(boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_>, paddle::framework::proto::VarType_Type, paddle::memory::allocation::Allocator::Attr, unsigned long) + 154 13 0x7eff926447efp float* paddle::framework::Tensor::mutable_data(boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_>, paddle::memory::allocation::Allocator::Attr, unsigned long) + 79 14 0x7eff92644cb6p 15 0x7eff9268c457p 16 0x7eff926667cep 17 0x565d5cp _PyCFunction_FastCallDict + 860 18 0x503073p 19 0x506859p _PyEval_EvalFrameDefault + 1097 20 0x502209p 21 0x502f3dp 22 0x506859p _PyEval_EvalFrameDefault + 1097 23 0x502209p 24 0x502f3dp 25 0x506859p _PyEval_EvalFrameDefault + 1097 26 0x502209p 27 0x502f3dp 28 0x506859p _PyEval_EvalFrameDefault + 1097 29 0x504c28p 30 0x506393p PyEval_EvalCode + 35 31 0x634d52p 32 0x634e0ap PyRun_FileExFlags + 154 33 0x6385c8p PyRun_SimpleFileExFlags + 392 34 0x63915ap Py_Main + 1402 35 0x4a6f10p main + 224 36 0x7effd8e93b97p __libc_start_main + 231 37 0x5afa0ap _start + 42

+-----------------------------------------------------------------------------+ | NVIDIA-SMI 418.56 Driver Version: 418.56 CUDA Version: 10.1 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 GeForce RTX 2060 Off | 00000000:01:00.0 On | N/A | | N/A 47C P8 11W / N/A | 579MiB / 5896MiB | 2% Default | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | 0 1164 G /usr/lib/xorg/Xorg 18MiB | | 0 1198 G /usr/bin/gnome-shell 51MiB | | 0 1472 G /usr/lib/xorg/Xorg 214MiB | | 0 1601 G /usr/bin/gnome-shell 189MiB | | 0 2153 G ...quest-channel-token=4841798388314319105 103MiB | +-----------------------------------------------------------------------------+