develop failed for language_model when use_program_cache is True
Created by: Xreki
When I train language_model model with the following command:
python train.py --data_path data/simple-examples/data/ --model_type small --rnn_model padding --use_gpu TrueI get the following error:
2019-06-04 03:07:03,296 - lm - INFO - Running with args : Namespace(data_path='data/simple-examples/data/', enable_ce=False, log_path=None, model_type='small', para_init=False, rnn_model='padding', use_gpu=True)
2019-06-04 03:07:03,296-INFO: Running with args : Namespace(data_path='data/simple-examples/data/', enable_ce=False, log_path=None, model_type='small', para_init=False, rnn_model='padding', use_gpu=True)
W0604 03:07:04.698928 389 device_context.cc:261] Please NOTE: device: 0, CUDA Capability: 70, Driver API Version: 10.1, Runtime API Version: 9.2
W0604 03:07:04.704860 389 device_context.cc:269] device: 0, cuDNN Version: 7.0.
begin to load data
vocab word num 10000
finished load data
epoch id 0
ppl 232 857.33685 1.0
ppl 464 630.8254 1.0
ppl 696 511.6698 1.0
ppl 928 438.74655 1.0
ppl 1160 394.31873 1.0
ppl 1392 354.0691 1.0
ppl 1624 326.4417 1.0
ppl 1856 305.87878 1.0
ppl 2088 286.34476 1.0
ppl 2320 270.66284 1.0
train ppl 270.62207
Traceback (most recent call last):
File "train.py", line 314, in <module>
train()
File "train.py", line 299, in train
valid_ppl = eval(valid_data)
File "train.py", line 223, in eval
use_program_cache=True)
File "/usr/local/lib/python2.7/dist-packages/paddle/fluid/executor.py", line 650, in run
use_program_cache=use_program_cache)
File "/usr/local/lib/python2.7/dist-packages/paddle/fluid/executor.py", line 750, in _run
exe.run_cached_prepared_ctx(ctx, scope, False, False, False)
paddle.fluid.core.EnforceNotMet: 0x1d4edfe0 Cannot find 0x1e4597e0 as kid scope at [/paddle/paddle/fluid/framework/scope.cc:124]
PaddlePaddle Call Stacks:
0 0x7fb5aaedb2dap void paddle::platform::EnforceNotMet::Init<char const*>(char const*, char const*, int) + 506
1 0x7fb5aaedbfe5p paddle::platform::EnforceNotMet::EnforceNotMet(std::__exception_ptr::exception_ptr, char const*, int) + 165
2 0x7fb5ad03dff3p paddle::framework::Scope::DeleteScope(paddle::framework::Scope*) const + 1219
3 0x7fb5ab28fd10p paddle::operators::StepScopes::StepScopes(paddle::platform::DeviceContext const&, paddle::framework::Scope const&, std::vector<paddle::framework::Scope*, std::allocator<paddle::framework::Scope*> >*, bool, unsigned long, bool) + 192
4 0x7fb5ab297c47p paddle::operators::RecurrentOp::RunImpl(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&) const + 551
5 0x7fb5acfd292ep paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&) + 318
6 0x7fb5ab0b3736p paddle::framework::Executor::RunPreparedContext(paddle::framework::ExecutorPrepareContext*, paddle::framework::Scope*, bool, bool, bool) + 374
7 0x7fb5aaeaf730p
8 0x7fb5aaf0e4e4p
9 0x4c5326p PyEval_EvalFrameEx + 37958
10 0x4b9b66p PyEval_EvalCodeEx + 774
11 0x4c1f56p PyEval_EvalFrameEx + 24694
12 0x4b9b66p PyEval_EvalCodeEx + 774
13 0x4c17c6p PyEval_EvalFrameEx + 22758
14 0x4b9b66p PyEval_EvalCodeEx + 774
15 0x4c1f56p PyEval_EvalFrameEx + 24694
16 0x4b9b66p PyEval_EvalCodeEx + 774
17 0x4c1f56p PyEval_EvalFrameEx + 24694
18 0x4b9b66p PyEval_EvalCodeEx + 774
19 0x4eb69fp
20 0x4e58f2p PyRun_FileExFlags + 130
21 0x4e41a6p PyRun_SimpleFileExFlags + 390
22 0x4938cep Py_Main + 1358
23 0x7fb671eee830p __libc_start_main + 240
24 0x493299p _start + 41It seems related to the recent modification for executor when setting use_program_cache=True. I found use_program_cache is not set for unitests here. I try to set use_program_cache=True for all models in the unittests, but I got another error which I am not sure about it. See #17807 .
+ make test 'ARGS=-R test_rnn_encoder_decoder -V'
Running tests...
UpdateCTestConfiguration from :/paddle/build_paddle/build_docker/DartConfiguration.tcl
UpdateCTestConfiguration from :/paddle/build_paddle/build_docker/DartConfiguration.tcl
Test project /paddle/build_paddle/build_docker
Constructing a list of tests
Done constructing a list of tests
Checking test dependency graph...
Checking test dependency graph end
test 574
Start 574: test_rnn_encoder_decoder
574: Test command: /usr/bin/cmake "-E" "env" "FLAGS_init_allocated_mem=true" "FLAGS_cudnn_deterministic=true" "FLAGS_cpu_deterministic=true" "FLAGS_limit_of_tmp_allocation=4294967296" "PYTHONPATH=/paddle/build_paddle/build_docker/python" "/usr/bin/python2.7" "-u" "test_rnn_encoder_decoder.py"
574: Test timeout computed to be: 600
574:
574:
574: pass_id=0 batch=0 avg_cost=[10.308955]
574: E
574: W0604 02:48:52.363976 41659 device_context.cc:261] Please NOTE: device: 0, CUDA Capability: 70, Driver API Version: 10.1, Runtime API Version: 9.2
574: W0604 02:48:52.369340 41659 device_context.cc:269] device: 0, cuDNN Version: 7.0.
574: pass_id=0 batch=0 avg_cost=[10.309135]
574: E
574: ======================================================================
574: ERROR: test_cpu (__main__.TestRnnEncoderDecoder)
574: ----------------------------------------------------------------------
574: Traceback (most recent call last):
574: File "test_rnn_encoder_decoder.py", line 273, in test_cpu
574: main(use_cuda=False)
574: File "test_rnn_encoder_decoder.py", line 262, in main
574: train(use_cuda, save_dirname, use_program_cache)
574: File "test_rnn_encoder_decoder.py", line 185, in train
574: use_program_cache=use_program_cache)
574: File "/paddle/build_paddle/build_docker/python/paddle/fluid/executor.py", line 650, in run
574: use_program_cache=use_program_cache)
574: File "/paddle/build_paddle/build_docker/python/paddle/fluid/executor.py", line 750, in _run
574: exe.run_cached_prepared_ctx(ctx, scope, False, False, False)
574: EnforceNotMet: Invoke operator cross_entropy2 error.
574: Python Callstacks:
574: File "/paddle/build_paddle/build_docker/python/paddle/fluid/framework.py", line 1703, in append_op
574: attrs=kwargs.get("attrs", None))
574: File "/paddle/build_paddle/build_docker/python/paddle/fluid/layer_helper.py", line 43, in append_op
574: return self.main_program.current_block().append_op(*args, **kwargs)
574: File "/paddle/build_paddle/build_docker/python/paddle/fluid/layers/nn.py", line 1562, in cross_entropy2
574: attrs={'ignore_index': ignore_index})
574: File "/paddle/build_paddle/build_docker/python/paddle/fluid/layers/nn.py", line 1537, in cross_entropy
574: return cross_entropy2(input, label, ignore_index)
574: File "test_rnn_encoder_decoder.py", line 151, in seq_to_seq_net
574: cost = fluid.layers.cross_entropy(input=prediction, label=label)
574: File "test_rnn_encoder_decoder.py", line 158, in train
574: [avg_cost, prediction] = seq_to_seq_net()
574: File "test_rnn_encoder_decoder.py", line 262, in main
574: train(use_cuda, save_dirname, use_program_cache)
574: File "test_rnn_encoder_decoder.py", line 273, in test_cpu
574: main(use_cuda=False)
574: File "/usr/lib/python2.7/unittest/case.py", line 329, in run
574: testMethod()
574: File "/usr/lib/python2.7/unittest/case.py", line 393, in __call__
574: return self.run(*args, **kwds)
574: File "/usr/lib/python2.7/unittest/suite.py", line 108, in run
574: test(result)
574: File "/usr/lib/python2.7/unittest/suite.py", line 70, in __call__
574: return self.run(*args, **kwds)
574: File "/usr/lib/python2.7/unittest/suite.py", line 108, in run
574: test(result)
574: File "/usr/lib/python2.7/unittest/suite.py", line 70, in __call__
574: return self.run(*args, **kwds)
574: File "/usr/lib/python2.7/unittest/runner.py", line 151, in run
574: test(result)
574: File "/usr/lib/python2.7/unittest/main.py", line 232, in runTests
574: self.result = testRunner.run(self.test)
574: File "/usr/lib/python2.7/unittest/main.py", line 95, in __init__
574: self.runTests()
574: File "test_rnn_encoder_decoder.py", line 286, in <module>
574: unittest.main()
574: C++ Callstacks:
574: Enforce failed. Expected framework::slice_ddim(x_dims, 0, rank - 1) == framework::slice_ddim(label_dims, 0, rank - 1), but received framework::slice_ddim(x_dims, 0, rank - 1):209 != framework::slice_ddim(label_dims, 0, rank - 1):163.
574: Input(X) and Input(Label) shall have the same shape except the last dimension. at [/paddle/paddle/fluid/operators/cross_entropy_op.cc:45]
574: PaddlePaddle Call Stacks:
574: 0 0x7fec1ab6323dp void paddle::platform::EnforceNotMet::Init<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, char const*, int) + 509
574: 1 0x7fec1ab63f2dp paddle::platform::EnforceNotMet::EnforceNotMet(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char const*, int) + 125
574: 2 0x7fec1b2e932ep paddle::operators::CrossEntropyOpBase::InferShape(paddle::framework::InferShapeContext*) const + 2110
574: 3 0x7fec1b2eb4e2p paddle::operators::CrossEntropyOp2::InferShape(paddle::framework::InferShapeContext*) const + 50
574: 4 0x7fec1cc5f951p paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&, paddle::framework::RuntimeContext*) const + 897
574: 5 0x7fec1cc603e7p paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&) const + 487
574: 6 0x7fec1cc5992ep paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&) + 318
574: 7 0x7fec1ad3a736p paddle::framework::Executor::RunPreparedContext(paddle::framework::ExecutorPrepareContext*, paddle::framework::Scope*, bool, bool, bool) + 374
574: 8 0x7fec1ab36730p
574: 9 0x7fec1ab954e4p
574: 10 0x4c5326p PyEval_EvalFrameEx + 37958
574: 11 0x4b9b66p PyEval_EvalCodeEx + 774
574: 12 0x4c1f56p PyEval_EvalFrameEx + 24694
574: 13 0x4b9b66p PyEval_EvalCodeEx + 774
574: 14 0x4c17c6p PyEval_EvalFrameEx + 22758
574: 15 0x4b9b66p PyEval_EvalCodeEx + 774
574: 16 0x4c17c6p PyEval_EvalFrameEx + 22758
574: 17 0x4b9b66p PyEval_EvalCodeEx + 774
574: 18 0x4c1f56p PyEval_EvalFrameEx + 24694
574: 19 0x4b9b66p PyEval_EvalCodeEx + 774
574: 20 0x4c1f56p PyEval_EvalFrameEx + 24694
574: 21 0x4b9b66p PyEval_EvalCodeEx + 774
574: 22 0x4d57a3p
574: 23 0x4a587ep PyObject_Call + 62
574: 24 0x4be51ep PyEval_EvalFrameEx + 9790
574: 25 0x4b9b66p PyEval_EvalCodeEx + 774
574: 26 0x4d5669p
574: 27 0x4eef5ep
574: 28 0x4a587ep PyObject_Call + 62
574: 29 0x548fc3p
574: 30 0x4c166dp PyEval_EvalFrameEx + 22413
574: 31 0x4b9b66p PyEval_EvalCodeEx + 774
574: 32 0x4d57a3p
574: 33 0x4a587ep PyObject_Call + 62
574: 34 0x4be51ep PyEval_EvalFrameEx + 9790
574: 35 0x4b9b66p PyEval_EvalCodeEx + 774
574: 36 0x4d5669p
574: 37 0x4eef5ep
574: 38 0x4a587ep PyObject_Call + 62
574: 39 0x548fc3p
574: 40 0x4c166dp PyEval_EvalFrameEx + 22413
574: 41 0x4b9b66p PyEval_EvalCodeEx + 774
574: 42 0x4d57a3p
574: 43 0x4a587ep PyObject_Call + 62
574: 44 0x4be51ep PyEval_EvalFrameEx + 9790
574: 45 0x4b9b66p PyEval_EvalCodeEx + 774
574: 46 0x4d5669p
574: 47 0x4eef5ep
574: 48 0x4a587ep PyObject_Call + 62
574: 49 0x548fc3p
574: 50 0x4c166dp PyEval_EvalFrameEx + 22413
574: 51 0x4c141fp PyEval_EvalFrameEx + 21823
574: 52 0x4c141fp PyEval_EvalFrameEx + 21823
574: 53 0x4b9b66p PyEval_EvalCodeEx + 774
574: 54 0x4d5669p
574: 55 0x4eef5ep
574: 56 0x4eeb66p
574: 57 0x4aaafbp
574: 58 0x4c166dp PyEval_EvalFrameEx + 22413
574: 59 0x4b9b66p PyEval_EvalCodeEx + 774
574: 60 0x4eb69fp
574: 61 0x4e58f2p PyRun_FileExFlags + 130
574: 62 0x4e41a6p PyRun_SimpleFileExFlags + 390
574: 63 0x4938cep Py_Main + 1358
574: 64 0x7fece1b96830p __libc_start_main + 240
574: 65 0x493299p _start + 41
574:
574:
574: ======================================================================
574: ERROR: test_cuda (__main__.TestRnnEncoderDecoder)
574: ----------------------------------------------------------------------
574: Traceback (most recent call last):
574: File "test_rnn_encoder_decoder.py", line 269, in test_cuda
574: main(use_cuda=True)
574: File "test_rnn_encoder_decoder.py", line 262, in main
574: train(use_cuda, save_dirname, use_program_cache)
574: File "test_rnn_encoder_decoder.py", line 185, in train
574: use_program_cache=use_program_cache)
574: File "/paddle/build_paddle/build_docker/python/paddle/fluid/executor.py", line 650, in run
574: use_program_cache=use_program_cache)
574: File "/paddle/build_paddle/build_docker/python/paddle/fluid/executor.py", line 750, in _run
574: exe.run_cached_prepared_ctx(ctx, scope, False, False, False)
574: EnforceNotMet: Enforce failed. Expected in_dim == x_dim, but received in_dim:1, 32 != x_dim:2, 32.
574: Input tensors must have same shape at [/paddle/paddle/fluid/operators/sum_op.cc:71]
574: PaddlePaddle Call Stacks:
574: 0 0x7fec1ab6323dp void paddle::platform::EnforceNotMet::Init<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, char const*, int) + 509
574: 1 0x7fec1ab63f2dp paddle::platform::EnforceNotMet::EnforceNotMet(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char const*, int) + 125
574: 2 0x7fec1b3bbb25p paddle::operators::SumOp::InferShape(paddle::framework::InferShapeContext*) const + 2149
574: 3 0x7fec1cc5f951p paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&, paddle::framework::RuntimeContext*) const + 897
574: 4 0x7fec1cc603e7p paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&) const + 487
574: 5 0x7fec1cc5992ep paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&) + 318
574: 6 0x7fec1ad3a736p paddle::framework::Executor::RunPreparedContext(paddle::framework::ExecutorPrepareContext*, paddle::framework::Scope*, bool, bool, bool) + 374
574: 7 0x7fec1c76820fp paddle::operators::WhileGradOp::RunImpl(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&) const + 1775
574: 8 0x7fec1cc5992ep paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, boost::variant<paddle::platform::CUDAPlace, paddle::platform::CPUPlace, paddle::platform::CUDAPinnedPlace, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_, boost::detail::variant::void_> const&) + 318
574: 9 0x7fec1ad3a736p paddle::framework::Executor::RunPreparedContext(paddle::framework::ExecutorPrepareContext*, paddle::framework::Scope*, bool, bool, bool) + 374
574: 10 0x7fec1ab36730p
574: 11 0x7fec1ab954e4p
574: 12 0x4c5326p PyEval_EvalFrameEx + 37958
574: 13 0x4b9b66p PyEval_EvalCodeEx + 774
574: 14 0x4c1f56p PyEval_EvalFrameEx + 24694
574: 15 0x4b9b66p PyEval_EvalCodeEx + 774
574: 16 0x4c17c6p PyEval_EvalFrameEx + 22758
574: 17 0x4b9b66p PyEval_EvalCodeEx + 774
574: 18 0x4c17c6p PyEval_EvalFrameEx + 22758
574: 19 0x4b9b66p PyEval_EvalCodeEx + 774
574: 20 0x4c1f56p PyEval_EvalFrameEx + 24694
574: 21 0x4b9b66p PyEval_EvalCodeEx + 774
574: 22 0x4c1f56p PyEval_EvalFrameEx + 24694
574: 23 0x4b9b66p PyEval_EvalCodeEx + 774
574: 24 0x4d57a3p
574: 25 0x4a587ep PyObject_Call + 62
574: 26 0x4be51ep PyEval_EvalFrameEx + 9790
574: 27 0x4b9b66p PyEval_EvalCodeEx + 774
574: 28 0x4d5669p
574: 29 0x4eef5ep
574: 30 0x4a587ep PyObject_Call + 62
574: 31 0x548fc3p
574: 32 0x4c166dp PyEval_EvalFrameEx + 22413
574: 33 0x4b9b66p PyEval_EvalCodeEx + 774
574: 34 0x4d57a3p
574: 35 0x4a587ep PyObject_Call + 62
574: 36 0x4be51ep PyEval_EvalFrameEx + 9790
574: 37 0x4b9b66p PyEval_EvalCodeEx + 774
574: 38 0x4d5669p
574: 39 0x4eef5ep
574: 40 0x4a587ep PyObject_Call + 62
574: 41 0x548fc3p
574: 42 0x4c166dp PyEval_EvalFrameEx + 22413
574: 43 0x4b9b66p PyEval_EvalCodeEx + 774
574: 44 0x4d57a3p
574: 45 0x4a587ep PyObject_Call + 62
574: 46 0x4be51ep PyEval_EvalFrameEx + 9790
574: 47 0x4b9b66p PyEval_EvalCodeEx + 774
574: 48 0x4d5669p
574: 49 0x4eef5ep
574: 50 0x4a587ep PyObject_Call + 62
574: 51 0x548fc3p
574: 52 0x4c166dp PyEval_EvalFrameEx + 22413
574: 53 0x4c141fp PyEval_EvalFrameEx + 21823
574: 54 0x4c141fp PyEval_EvalFrameEx + 21823
574: 55 0x4b9b66p PyEval_EvalCodeEx + 774
574: 56 0x4d5669p
574: 57 0x4eef5ep
574: 58 0x4eeb66p
574: 59 0x4aaafbp
574: 60 0x4c166dp PyEval_EvalFrameEx + 22413
574: 61 0x4b9b66p PyEval_EvalCodeEx + 774
574: 62 0x4eb69fp
574: 63 0x4e58f2p PyRun_FileExFlags + 130
574: 64 0x4e41a6p PyRun_SimpleFileExFlags + 390
574: 65 0x4938cep Py_Main + 1358
574: 66 0x7fece1b96830p __libc_start_main + 240
574: 67 0x493299p _start + 41
574:
574:
574: ----------------------------------------------------------------------
574: Ran 2 tests in 9.057s
574:
574: FAILED (errors=2)
1/1 Test #574: test_rnn_encoder_decoder .........***Failed 13.86 sec
0% tests passed, 1 tests failed out of 1
Total Test time (real) = 13.90 sec
The following tests FAILED:
574 - test_rnn_encoder_decoder (Failed)
Errors while running CTest
Makefile:72: recipe for target 'test' failed
make: *** [test] Error 8