paddle1.7多进程问题
Created by: shuoyin
我使用python2.7.13+paddle-gpu-1.7+pgl 1.0 测试代码如下
from multiprocessing import Pool
import pgl
def f(x):
return x*x
p = Pool(1)
x = [1,2,3,4,5,6]
y = p.map(f, x)
print y
p.terminate()
运行这段程序,会报这个错误
W0507 19:54:19.298550 17687 init.cc:209] Warning: PaddlePaddle catches a failure signal, it may not work properly
W0507 19:54:19.298624 17687 init.cc:211] You could check whether you killed PaddlePaddle thread/process accidentally or report the case to PaddlePaddle
W0507 19:54:19.298629 17687 init.cc:214] The detail failure signal is:
W0507 19:54:19.298636 17687 init.cc:217] *** Aborted at 1588852459 (unix time) try "date -d @1588852459" if you are using GNU date ***
W0507 19:54:19.300464 17687 init.cc:217] PC: @ 0x0 (unknown)
W0507 19:54:19.300927 17687 init.cc:217] *** SIGTERM (@0x20000004477) received by PID 17687 (TID 0x7ff960b48700) from PID 17527; stack trace: ***
W0507 19:54:19.302687 17687 init.cc:217] @ 0x7ff9602ff160 (unknown)
W0507 19:54:19.304461 17687 init.cc:217] @ 0x7ff960589860 (unknown)
W0507 19:54:19.306123 17687 init.cc:217] @ 0x7ff96061961c PyEval_EvalFrameEx
W0507 19:54:19.307770 17687 init.cc:217] @ 0x7ff96061c0bd PyEval_EvalCodeEx
W0507 19:54:19.309356 17687 init.cc:217] @ 0x7ff960592eb0 function_call
W0507 19:54:19.311004 17687 init.cc:217] @ 0x7ff960560df3 PyObject_Call
W0507 19:54:19.312568 17687 init.cc:217] @ 0x7ff96056f9cd instancemethod_call
W0507 19:54:19.314218 17687 init.cc:217] @ 0x7ff960560df3 PyObject_Call
W0507 19:54:19.315840 17687 init.cc:217] @ 0x7ff9605cebaf slot_tp_init
W0507 19:54:19.317483 17687 init.cc:217] @ 0x7ff9605cb46f type_call
W0507 19:54:19.319144 17687 init.cc:217] @ 0x7ff960560df3 PyObject_Call
W0507 19:54:19.320789 17687 init.cc:217] @ 0x7ff9606164a6 PyEval_EvalFrameEx
W0507 19:54:19.322435 17687 init.cc:217] @ 0x7ff960619460 PyEval_EvalFrameEx
W0507 19:54:19.324079 17687 init.cc:217] @ 0x7ff960619460 PyEval_EvalFrameEx
W0507 19:54:19.325723 17687 init.cc:217] @ 0x7ff96061c0bd PyEval_EvalCodeEx
W0507 19:54:19.327309 17687 init.cc:217] @ 0x7ff960592eb0 function_call
W0507 19:54:19.328956 17687 init.cc:217] @ 0x7ff960560df3 PyObject_Call
W0507 19:54:19.330516 17687 init.cc:217] @ 0x7ff96056f9cd instancemethod_call
W0507 19:54:19.332163 17687 init.cc:217] @ 0x7ff960560df3 PyObject_Call
W0507 19:54:19.333753 17687 init.cc:217] @ 0x7ff9605cebaf slot_tp_init
W0507 19:54:19.335361 17687 init.cc:217] @ 0x7ff9605cb46f type_call
W0507 19:54:19.337008 17687 init.cc:217] @ 0x7ff960560df3 PyObject_Call
W0507 19:54:19.338646 17687 init.cc:217] @ 0x7ff9606164a6 PyEval_EvalFrameEx
W0507 19:54:19.340299 17687 init.cc:217] @ 0x7ff96061c0bd PyEval_EvalCodeEx
W0507 19:54:19.341948 17687 init.cc:217] @ 0x7ff960619345 PyEval_EvalFrameEx
W0507 19:54:19.343593 17687 init.cc:217] @ 0x7ff96061c0bd PyEval_EvalCodeEx
W0507 19:54:19.345240 17687 init.cc:217] @ 0x7ff96061c1f2 PyEval_EvalCode
W0507 19:54:19.346880 17687 init.cc:217] @ 0x7ff960644f42 PyRun_FileExFlags
W0507 19:54:19.348536 17687 init.cc:217] @ 0x7ff9606462d9 PyRun_SimpleFileExFlags
W0507 19:54:19.350172 17687 init.cc:217] @ 0x7ff96065c00d Py_Main
W0507 19:54:19.401824 17687 init.cc:217] @ 0x7ff95f859bd5 __libc_start_main
W0507 19:54:19.420583 17687 init.cc:217] @ 0x4007a1 (unknown)
后来发现把p.terminate()
换成p.close()
就可以正常跑了
但是,运行命令
python preprocessing/dump_graph.py -i data.txt -o work_dir --encoding utf-8 -l 40 --vocab_file vocab/vocab.txt
发现会报错,即使把p.terminate()
换成p.close()
也不行,只有把进程数开的比较小才行(我换成了两个进程)
请问这种情况怎么解决