Created by: sicccs

通过其他数据，在运行到一半的时候出现以下错误 2020-06-24 12:16:31 [INFO] [TRAIN] Epoch 5 finished, loss=0.058296, lr=0.000584 . 2020-06-24 12:16:31 [INFO] Start to evaluating(total_samples=1500, total_steps=188)... 0%| | 0/188 [00:00<?, ?it/s]share_vars_from is set, scope is ignored. 0%| | 0/188 [00:00<?, ?it/s] Traceback (most recent call last): File "train.py", line 169, in train(args) File "train.py", line 164, in train use_vdl=args.use_vdl) File "/data2/sxc/PaddleSeg/contrib/HumanSeg/models/humanseg.py", line 580, in train epoch_id=i + 1) File "/data2/sxc/PaddleSeg/contrib/HumanSeg/models/humanseg.py", line 663, in evaluate conf_mat.calculate(pred=pred, label=labels, ignore=mask) File "/data2/sxc/PaddleSeg/contrib/HumanSeg/utils/metrics.py", line 47, in calculate shape=(self.num_classes, self.num_classes)) File "/work/anaconda3/envs/paddle_pnet_0623/lib/python3.7/site-packages/scipy/sparse/compressed.py", line 57, in init other = self.class(coo_matrix(arg1, shape=shape)) File "/work/anaconda3/envs/paddle_pnet_0623/lib/python3.7/site-packages/scipy/sparse/coo.py", line 198, in init self._check() File "/work/anaconda3/envs/paddle_pnet_0623/lib/python3.7/site-packages/scipy/sparse/coo.py", line 285, in _check raise ValueError('row index exceeds matrix dimensions') ValueError: row index exceeds matrix dimensions terminate called without an active exception W0624 12:16:31.998463 9330 init.cc:216] Warning: PaddlePaddle catches a failure signal, it may not work properly W0624 12:16:31.998479 9330 init.cc:218] You could check whether you killed PaddlePaddle thread/process accidentally or report the case to PaddlePaddle W0624 12:16:31.998484 9330 init.cc:221] The detail failure signal is:

W0624 12:16:31.998488 9330 init.cc:224] *** Aborted at 1592972191 (unix time) try "date -d @1592972191" if you are using GNU date *** W0624 12:16:32.000007 9330 init.cc:224] PC: @ 0x0 (unknown) W0624 12:16:32.000090 9330 init.cc:224] *** SIGABRT (@0x3e8000023c1) received by PID 9153 (TID 0x7f2a717fe700) from PID 9153; stack trace: *** W0624 12:16:32.001487 9330 init.cc:224] @ 0x7f2b24c9f390 (unknown) W0624 12:16:32.002831 9330 init.cc:224] @ 0x7f2b248f9428 gsignal W0624 12:16:32.004184 9330 init.cc:224] @ 0x7f2b248fb02a abort W0624 12:16:32.005360 9330 init.cc:224] @ 0x7f2b0c23584a __gnu_cxx::__verbose_terminate_handler() W0624 12:16:32.006330 9330 init.cc:224] @ 0x7f2b0c233f47 __cxxabiv1::__terminate() W0624 12:16:32.007462 9330 init.cc:224] @ 0x7f2b0c233f7d std::terminate() W0624 12:16:32.008489 9330 init.cc:224] @ 0x7f2b0c233c5a __gxx_personality_v0 W0624 12:16:32.009673 9330 init.cc:224] @ 0x7f2b20c91b97 _Unwind_ForcedUnwind_Phase2 W0624 12:16:32.010874 9330 init.cc:224] @ 0x7f2b20c91e7d _Unwind_ForcedUnwind W0624 12:16:32.012226 9330 init.cc:224] @ 0x7f2b24c9e070 __GI___pthread_unwind W0624 12:16:32.013592 9330 init.cc:224] @ 0x7f2b24c96845 __pthread_exit W0624 12:16:32.013970 9330 init.cc:224] @ 0x55a4a83fb2d9 PyThread_exit_thread W0624 12:16:32.014071 9330 init.cc:224] @ 0x55a4a827fc52 PyEval_RestoreThread.cold.799 W0624 12:16:32.015002 9330 init.cc:224] @ 0x7f2b097e1f03 (unknown) W0624 12:16:32.015401 9330 init.cc:224] @ 0x55a4a837cd36 _PyMethodDef_RawFastCallKeywords W0624 12:16:32.015786 9330 init.cc:224] @ 0x55a4a837cdb1 _PyCFunction_FastCallKeywords W0624 12:16:32.016170 9330 init.cc:224] @ 0x55a4a83e8a85 _PyEval_EvalFrameDefault W0624 12:16:32.016532 9330 init.cc:224] @ 0x55a4a832c2b9 _PyEval_EvalCodeWithName W0624 12:16:32.016863 9330 init.cc:224] @ 0x55a4a837c435 _PyFunction_FastCallKeywords W0624 12:16:32.017242 9330 init.cc:224] @ 0x55a4a83e3be6 _PyEval_EvalFrameDefault W0624 12:16:32.017593 9330 init.cc:224] @ 0x55a4a832c2b9 _PyEval_EvalCodeWithName W0624 12:16:32.017946 9330 init.cc:224] @ 0x55a4a832d3e5 _PyFunction_FastCallDict W0624 12:16:32.018290 9330 init.cc:224] @ 0x55a4a834bb93 _PyObject_Call_Prepend W0624 12:16:32.018463 9330 init.cc:224] @ 0x55a4a838316a slot_tp_call W0624 12:16:32.018818 9330 init.cc:224] @ 0x55a4a838400b _PyObject_FastCallKeywords W0624 12:16:32.019198 9330 init.cc:224] @ 0x55a4a83e8186 _PyEval_EvalFrameDefault W0624 12:16:32.019548 9330 init.cc:224] @ 0x55a4a832c2b9 _PyEval_EvalCodeWithName W0624 12:16:32.019901 9330 init.cc:224] @ 0x55a4a832d3e5 _PyFunction_FastCallDict W0624 12:16:32.020251 9330 init.cc:224] @ 0x55a4a834bb93 _PyObject_Call_Prepend W0624 12:16:32.020423 9330 init.cc:224] @ 0x55a4a838316a slot_tp_call W0624 12:16:32.020789 9330 init.cc:224] @ 0x55a4a838400b _PyObject_FastCallKeywords W0624 12:16:32.021173 9330 init.cc:224] @ 0x55a4a83e8186 _PyEval_EvalFrameDefault 已放弃 (核心已转储)