test_conv2d_bf16_mkldnn_op fails on 5117 machine
Created by: luotao1
test_conv2d_bf16_mkldnn_op fails on 5117 machine (Nightly Job)
- error log
[19:20:48] : [Step 1/2] The following tests FAILED:
[19:20:48] : [Step 1/2] 1179 - test_conv2d_bf16_mkldnn_op (Failed)
[19:20:48] : [Step 1/2] At least one test failed with exit code => 0
[19:20:48] : [Step 1/2] At least one test failed with exit code => 0
[19:20:48] : [Step 1/2] 1/1 Test #1179: test_conv2d_bf16_mkldnn_op .......***Failed 1.98 sec
[19:20:48] : [Step 1/2] W0926 19:20:48.445756 51794 operator.cc:205] conv2d raises an exception dnnl::error, could not create a primitive descriptor iterator
[19:20:48] : [Step 1/2] W0926 19:20:48.461982 51794 operator.cc:205] conv2d raises an exception dnnl::error, could not create a primitive descriptor iterator
[19:20:48] : [Step 1/2] W0926 19:20:48.662735 51794 operator.cc:205] conv2d raises an exception dnnl::error, could not create a primitive descriptor iterator
[19:20:48] : [Step 1/2] W0926 19:20:48.681520 51794 operator.cc:205] conv2d raises an exception dnnl::error, could not create a primitive descriptor iterator
[19:20:48] : [Step 1/2] W0926 19:20:48.695686 51794 operator.cc:205] conv2d raises an exception dnnl::error, could not create a primitive descriptor iterator
[19:20:48] : [Step 1/2] W0926 19:20:48.721101 51794 operator.cc:205] conv2d raises an exception dnnl::error, could not create a primitive descriptor iterator
[19:20:48] : [Step 1/2] W0926 19:20:48.740229 51794 operator.cc:205] conv2d raises an exception dnnl::error, could not create a primitive descriptor iterator
[19:20:48] : [Step 1/2] test_conv2d_bf16_mkldnn_op failed
[19:20:48] : [Step 1/2] ...E...E.......E...E...E...E...E
[19:20:48] : [Step 1/2] ======================================================================
[19:20:48] : [Step 1/2] ERROR: test_check_output (test_conv2d_bf16_mkldnn_op.TestConv2d)
[19:20:48] : [Step 1/2] ----------------------------------------------------------------------
[19:20:48] : [Step 1/2] Traceback (most recent call last):
[19:20:48] : [Step 1/2] File "/paddle/build/python/paddle/fluid/tests/unittests/mkldnn/test_conv2d_bf16_mkldnn_op.py", line 105, in test_check_output
[19:20:48] : [Step 1/2] self.check_output_with_place(core.CPUPlace())
[19:20:48] : [Step 1/2] File "/paddle/build/python/paddle/fluid/tests/unittests/op_test.py", line 995, in check_output_with_place
[19:20:48] : [Step 1/2] outs, fetch_list = self._calc_output(place, no_check_set=no_check_set)
[19:20:48] : [Step 1/2] File "/paddle/build/python/paddle/fluid/tests/unittests/op_test.py", line 619, in _calc_output
[19:20:48] : [Step 1/2] return_numpy=False)
[19:20:48] : [Step 1/2] File "/paddle/build/python/paddle/fluid/executor.py", line 1101, in run
[19:20:48] : [Step 1/2] six.reraise(*sys.exc_info())
[19:20:48] : [Step 1/2] File "/paddle/build/python/paddle/fluid/executor.py", line 1099, in run
[19:20:48] : [Step 1/2] return_merged=return_merged)
[19:20:48] : [Step 1/2] File "/paddle/build/python/paddle/fluid/executor.py", line 1223, in _run_impl
[19:20:48] : [Step 1/2] use_program_cache=use_program_cache)
[19:20:48] : [Step 1/2] File "/paddle/build/python/paddle/fluid/executor.py", line 1308, in _run_program
[19:20:48] : [Step 1/2] fetch_var_name)
[19:20:48] : [Step 1/2] RuntimeError: could not create a primitive descriptor iterator
[19:20:48] : [Step 1/2]
[19:20:48] : [Step 1/2] ======================================================================
[19:20:48] : [Step 1/2] ERROR: test_check_output (test_conv2d_bf16_mkldnn_op.TestConv2dBf16Op)
[19:20:48] : [Step 1/2] ----------------------------------------------------------------------
- machine information in log
[20:24:47][Step 1/2] ****************************************
[20:24:47][Step 1/2] + bash /paddle/tools/get_cpu_info.sh
[20:24:47][Step 1/2] ********** Hardware Information **********
[20:24:47][Step 1/2] CPU Name : Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz
[20:24:47][Step 1/2] CPU Family : 6
[20:24:47][Step 1/2] Socket Number : 16
[20:24:47][Step 1/2] Cores Per Socket : 1
[20:24:47][Step 1/2] Total Physical Cores : 16
[20:24:47][Step 1/2] Total Virtual Cores : 16
[20:24:47][Step 1/2] Hyper Threading : OFF
[20:24:47][Step 1/2] NUMA Nodes : 1
[20:24:47][Step 1/2] Warning: NUMA node is not enough for the best performance, at least 16
[20:24:47][Step 1/2] ********** Software Information **********
[20:24:47][Step 1/2] OS Version : Linux version 3.10.0_3-0-0-15 () (gcc version 4.8.2 (GCC) ) #1 SMP Fri Jan 12 18:18:11 CST 2018
[20:24:47][Step 1/2] Kernel Release Version : 3.10.0_3-0-0-15
[20:24:47][Step 1/2] Kernel Patch Version : #1 SMP Fri Jan 12 18:18:11 CST 2018
[20:24:47][Step 1/2] GCC Version : 8.2.0
- CPU log: Maybe our machine lack some Instruction set
processor : 15
vendor_id : GenuineIntel
cpu family : 6
model : 85
model name : Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz
stepping : 4
microcode : 0x1
cpu MHz : 2000.131
cache size : 4096 KB
physical id : 15
siblings : 1
core id : 0
cpu cores : 1
apicid : 15
initial apicid : 15
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch arat fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f rdseed adx smap avx512cd xsaveopt xsavec xgetbv1
bogomips : 4017.26
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
Paddle_PR_CI_Night_36227.log Paddle_PR_CI_Night_36228.log
@wozna @jczaja @lidanqing-intel Please help fix it.