TestActivationGrad failed
Created by: wangkuiyi
I got the fail from a CI failure: https://paddleci.ngrok.io/viewLog.html?buildId=11309&buildTypeId=Paddle_PrCi&tab=buildLog
I have no idea of the reason, so I copied the log here and rerun the test in hope that my change https://github.com/PaddlePaddle/Paddle/pull/4590, which has nothing with C++ source code, could pass the CI and get merged.
[10:38:51] 50/190 Test #53: test_ActivationGrad .........................***Failed 1.22 sec
[10:38:51] I1004 17:38:50.390895 29487 Util.cpp:166] commandline: /paddle/build/paddle/gserver/tests/test_ActivationGrad
[10:38:51] [==========] Running 2 tests from 2 test cases.
[10:38:51] [----------] Global test environment set-up.
[10:38:51] [----------] 1 test from Activation
[10:38:51] [ RUN ] Activation.activation
[10:38:51] I1004 17:38:51.096824 29487 test_ActivationGrad.cpp:32] test activation:
[10:38:51] I1004 17:38:51.096846 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1004 17:38:51.143564 29487 LayerGradUtil.cpp:724] cost 493.063
[10:38:51] I1004 17:38:51.143661 29487 LayerGradUtil.cpp:43] _activation layer_0 step=1e-06 cost1=493.144 cost2=492.982 true_delta=0.16217 analytic_delta=0.16216 diff=6.12916e-05
[10:38:51] I1040 17:38:51.143689 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.149119 29487 LayerGradUtil.cpp:724] cost 486.253
[10:38:51] I1040 17:38:51.150136 29487 LayerGradUtil.cpp:43] _activation layer_0 step=1e-06 cost1=486.337 cost2=486.168 true_delta=0.168457 analytic_delta=0.168457 diff=-2.74215e-06
[10:38:51] I1040 17:38:51.150169 29487 test_ActivationGrad.cpp:32] test activation: abs
[10:38:51] I1040 17:38:51.150177 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.150233 29487 LayerGradUtil.cpp:724] cost 513.028
[10:38:51] I1040 17:38:51.150287 29487 LayerGradUtil.cpp:43] abs_activation layer_0 step=1e-06 cost1=513.107 cost2=512.948 true_delta=0.159241 analytic_delta=0.159257 diff=-0.000101894
[10:38:51] I1040 17:38:51.150303 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.150971 29487 LayerGradUtil.cpp:724] cost 521.735
[10:38:51] I1040 17:38:51.152357 29487 LayerGradUtil.cpp:43] abs_activation layer_0 step=1e-06 cost1=521.817 cost2=521.654 true_delta=0.162842 analytic_delta=0.162841 diff=4.9414e-06
[10:38:51] I1040 17:38:51.152387 29487 test_ActivationGrad.cpp:32] test activation: brelu
[10:38:51] I1040 17:38:51.152395 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.152439 29487 LayerGradUtil.cpp:724] cost 516.154
[10:38:51] I1040 17:38:51.152493 29487 LayerGradUtil.cpp:43] brelu_activation layer_0 step=1e-06 cost1=516.235 cost2=516.074 true_delta=0.16156 analytic_delta=0.161579 diff=-0.000115001
[10:38:51] I1040 17:38:51.152509 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.153892 29487 LayerGradUtil.cpp:724] cost 488.409
[10:38:51] I1040 17:38:51.157428 29487 LayerGradUtil.cpp:43] brelu_activation layer_0 step=1e-06 cost1=488.491 cost2=488.326 true_delta=0.164978 analytic_delta=0.164952 diff=0.000159805
[10:38:51] I1040 17:38:51.157475 29487 test_ActivationGrad.cpp:32] test activation: exponential
[10:38:51] I1040 17:38:51.157485 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.157624 29487 LayerGradUtil.cpp:724] cost 1670.02
[10:38:51] I1040 17:38:51.157707 29487 LayerGradUtil.cpp:43] exponential_activation layer_0 step=1e-06 cost1=1670.24 cost2=1669.8 true_delta=0.444946 analytic_delta=0.444873 diff=0.000163792
[10:38:51] I1040 17:38:51.157723 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.159214 29487 LayerGradUtil.cpp:724] cost 1689.41
[10:38:51] I1040 17:38:51.162405 29487 LayerGradUtil.cpp:43] exponential_activation layer_0 step=1e-06 cost1=1689.64 cost2=1689.18 true_delta=0.459961 analytic_delta=0.459924 diff=8.0998e-05
[10:38:51] I1040 17:38:51.162587 29487 test_ActivationGrad.cpp:32] test activation: linear
[10:38:51] I1040 17:38:51.162600 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.162660 29487 LayerGradUtil.cpp:724] cost 507.521
[10:38:51] I1040 17:38:51.162708 29487 LayerGradUtil.cpp:43] linear_activation layer_0 step=1e-06 cost1=507.601 cost2=507.441 true_delta=0.160675 analytic_delta=0.160676 diff=-4.82251e-06
[10:38:51] I1040 17:38:51.162726 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.163892 29487 LayerGradUtil.cpp:724] cost 506.193
[10:38:51] I1040 17:38:51.166452 29487 LayerGradUtil.cpp:43] linear_activation layer_0 step=1e-06 cost1=506.273 cost2=506.112 true_delta=0.160706 analytic_delta=0.160701 diff=3.05069e-05
[10:38:51] I1040 17:38:51.166496 29487 test_ActivationGrad.cpp:32] test activation: log
[10:38:51] I1040 17:38:51.166507 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.166576 29487 LayerGradUtil.cpp:724] cost -685.931
[10:38:51] I1040 17:38:51.167474 29487 LayerGradUtil.cpp:43] log_activation layer_0 step=1e-06 cost1=-685.593 cost2=-686.268 true_delta=0.675354 analytic_delta=0.675384 diff=-4.40383e-05
[10:38:51] I1040 17:38:51.167497 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.168609 29487 LayerGradUtil.cpp:724] cost -691.745
[10:38:51] I1040 17:38:51.172569 29487 LayerGradUtil.cpp:43] log_activation layer_0 step=1e-06 cost1=-691.402 cost2=-692.089 true_delta=0.686218 analytic_delta=0.68622 diff=-2.43206e-06
[10:38:51] I1040 17:38:51.172616 29487 test_ActivationGrad.cpp:32] test activation: reciprocal
[10:38:51] I1040 17:38:51.172626 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.172693 29487 LayerGradUtil.cpp:724] cost 2061.17
[10:38:51] I1040 17:38:51.172756 29487 LayerGradUtil.cpp:43] reciprocal_activation layer_0 step=1e-06 cost1=2062.8 cost2=2059.53 true_delta=3.27417 analytic_delta=3.27427 diff=-2.95632e-05
[10:38:51] I1040 17:38:51.172776 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.174762 29487 LayerGradUtil.cpp:724] cost 1996.74
[10:38:51] I1040 17:38:51.178227 29487 LayerGradUtil.cpp:43] reciprocal_activation layer_0 step=1e-06 cost1=1998.35 cost2=1995.13 true_delta=3.22461 analytic_delta=3.22455 diff=1.78931e-05
[10:38:51] I1040 17:38:51.178267 29487 test_ActivationGrad.cpp:32] test activation: relu
[10:38:51] I1040 17:38:51.178277 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.178328 29487 LayerGradUtil.cpp:724] cost 510.296
[10:38:51] I1040 17:38:51.178383 29487 LayerGradUtil.cpp:43] relu_activation layer_0 step=1e-06 cost1=510.376 cost2=510.215 true_delta=0.160828 analytic_delta=0.160827 diff=1.11184e-06
[10:38:51] I1040 17:38:51.178401 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.180193 29487 LayerGradUtil.cpp:724] cost 518.481
[10:38:51] I1040 17:38:51.183256 29487 LayerGradUtil.cpp:43] relu_activation layer_0 step=1e-06 cost1=518.562 cost2=518.399 true_delta=0.162048 analytic_delta=0.162038 diff=6.46486e-05
[10:38:51] I1040 17:38:51.183311 29487 test_ActivationGrad.cpp:32] test activation: sigmoid
[10:38:51] I1040 17:38:51.183322 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.183382 29487 LayerGradUtil.cpp:724] cost 631.022
[10:38:51] I1040 17:38:51.183442 29487 LayerGradUtil.cpp:43] sigmoid_activation layer_0 step=1e-06 cost1=631.027 cost2=631.018 true_delta=0.00909424 analytic_delta=0.00905445 diff=0.00439482
[10:38:51] I1040 17:38:51.183460 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.185401 29487 LayerGradUtil.cpp:724] cost 625.49
[10:38:51] I1040 17:38:51.189941 29487 LayerGradUtil.cpp:43] sigmoid_activation layer_0 step=1e-06 cost1=625.495 cost2=625.486 true_delta=0.00897217 analytic_delta=0.00902048 diff=-0.00535626
[10:38:51] I1040 17:38:51.189988 29487 test_ActivationGrad.cpp:32] test activation: softmax
[10:38:51] I1040 17:38:51.190001 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.190071 29487 LayerGradUtil.cpp:724] cost 99.6977
[10:38:51] I1040 17:38:51.190168 29487 LayerGradUtil.cpp:43] softmax_activation layer_0 step=1e-06 cost1=99.6985 cost2=99.6969 true_delta=0.00163269 analytic_delta=0.00162817 diff=0.00277336
[10:38:51] I1040 17:38:51.191215 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.192163 29487 LayerGradUtil.cpp:724] cost 101.257
[10:38:51] I1040 17:38:51.196768 29487 LayerGradUtil.cpp:43] softmax_activation layer_0 step=1e-06 cost1=101.258 cost2=101.256 true_delta=0.00158691 analytic_delta=0.00158489 diff=0.00127963
[10:38:51] I1040 17:38:51.196884 29487 test_ActivationGrad.cpp:32] test activation: softrelu
[10:38:51] I1040 17:38:51.196897 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.197073 29487 LayerGradUtil.cpp:724] cost 984.459
[10:38:51] I1040 17:38:51.197177 29487 LayerGradUtil.cpp:43] softrelu_activation layer_0 step=1e-06 cost1=984.49 cost2=984.427 true_delta=0.0627441 analytic_delta=0.0627736 diff=-0.00046918
[10:38:51] I1040 17:38:51.197198 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.201499 29487 LayerGradUtil.cpp:724] cost 972.249
[10:38:51] I1040 17:38:51.207567 29487 LayerGradUtil.cpp:43] softrelu_activation layer_0 step=1e-06 cost1=972.28 cost2=972.218 true_delta=0.0622559 analytic_delta=0.0622713 diff=-0.00024743
[10:38:51] I1040 17:38:51.207615 29487 test_ActivationGrad.cpp:32] test activation: sqrt
[10:38:51] I1040 17:38:51.207628 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.207689 29487 LayerGradUtil.cpp:724] cost 729.939
[10:38:51] I1040 17:38:51.207756 29487 LayerGradUtil.cpp:43] sqrt_activation layer_0 step=1e-06 cost1=729.98 cost2=729.898 true_delta=0.0826416 analytic_delta=0.082625 diff=0.000201267
[10:38:51] I1040 17:38:51.207775 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.209758 29487 LayerGradUtil.cpp:724] cost 696.517
[10:38:51] I1040 17:38:51.214112 29487 LayerGradUtil.cpp:43] sqrt_activation layer_0 step=1e-06 cost1=696.559 cost2=696.475 true_delta=0.0841675 analytic_delta=0.0842092 diff=-0.000495914
[10:38:51] I1040 17:38:51.214154 29487 test_ActivationGrad.cpp:32] test activation: square
[10:38:51] I1040 17:38:51.214164 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.214218 29487 LayerGradUtil.cpp:724] cost 253.507
[10:38:51] I1040 17:38:51.214277 29487 LayerGradUtil.cpp:43] square_activation layer_0 step=1e-06 cost1=253.59 cost2=253.424 true_delta=0.165344 analytic_delta=0.165345 diff=-3.06414e-06
[10:38:51] I1040 17:38:51.214295 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.219662 29487 LayerGradUtil.cpp:724] cost 250.433
[10:38:51] I1040 17:38:51.226078 29487 LayerGradUtil.cpp:43] square_activation layer_0 step=1e-06 cost1=250.516 cost2=250.35 true_delta=0.166458 analytic_delta=0.166469 diff=-6.436e-05
[10:38:51] I1040 17:38:51.226145 29487 test_ActivationGrad.cpp:32] test activation: stanh
[10:38:51] I1040 17:38:51.226156 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.226404 29487 LayerGradUtil.cpp:724] cost 566.806
[10:38:51] I1040 17:38:51.226511 29487 LayerGradUtil.cpp:43] stanh_activation layer_0 step=1e-06 cost1=566.894 cost2=566.717 true_delta=0.176514 analytic_delta=0.1765 diff=7.53921e-05
[10:38:51] I1040 17:38:51.226531 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.227622 29487 LayerGradUtil.cpp:724] cost 554.589
[10:38:51] I1040 17:38:51.232861 29487 LayerGradUtil.cpp:43] stanh_activation layer_0 step=1e-06 cost1=554.673 cost2=554.505 true_delta=0.168823 analytic_delta=0.168854 diff=-0.000181616
[10:38:51] I1040 17:38:51.232965 29487 test_ActivationGrad.cpp:32] test activation: tanh
[10:38:51] I1040 17:38:51.232977 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.233240 29487 LayerGradUtil.cpp:724] cost 486.95
[10:38:51] I1040 17:38:51.233342 29487 LayerGradUtil.cpp:43] tanh_activation layer_0 step=1e-06 cost1=487.001 cost2=486.899 true_delta=0.101837 analytic_delta=0.101846 diff=-9.1078e-05
[10:38:51] I1040 17:38:51.233362 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.234861 29487 LayerGradUtil.cpp:724] cost 457.794
[10:38:51] I1040 17:38:51.240965 29487 LayerGradUtil.cpp:43] tanh_activation layer_0 step=1e-06 cost1=457.845 cost2=457.743 true_delta=0.101807 analytic_delta=0.101796 diff=0.000102541
[10:38:51] [ OK ] Activation.activation (145 ms)
[10:38:51] [----------] 1 test from Activation (145 ms total)
[10:38:51]
[10:38:51] [----------] 1 test from SequenceSoftmaxActivation
[10:38:51] [ RUN ] SequenceSoftmaxActivation.activation
[10:38:51] I1040 17:38:51.241061 29487 test_ActivationGrad.cpp:88] hasSubseq = 0
[10:38:51] I1040 17:38:51.241066 29487 test_ActivationGrad.cpp:61] test activation: sequence softmax
[10:38:51] I1040 17:38:51.241075 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.241135 29487 LayerGradUtil.cpp:724] cost 22.7935
[10:38:51] I1040 17:38:51.241173 29487 LayerGradUtil.cpp:43] sequence_softmax layer_0 step=1.9132e-06 cost1=22.7936 cost2=22.7933 true_delta=0.000228882 analytic_delta=0.000227935 diff=0.0041557
[10:38:51] I1040 17:38:51.241192 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.242135 29487 LayerGradUtil.cpp:724] cost 10.6591
[10:38:51] I1040 17:38:51.246783 29487 LayerGradUtil.cpp:43] sequence_softmax layer_0 step=1.28013e-06 cost1=10.6592 cost2=10.6591 true_delta=0.000103951 analytic_delta=0.000106591 diff=-0.0247752 ***
[10:38:51] /paddle/paddle/gserver/tests/LayerGradUtil.cpp:773: Failure
[10:38:51] Expected: (fabs(maxDiff)) <= (epsilon), actual: 0.0247752 vs 0.02
[10:38:51] I1040 17:38:51.246886 29487 test_ActivationGrad.cpp:88] hasSubseq = 1
[10:38:51] I1040 17:38:51.246892 29487 test_ActivationGrad.cpp:61] test activation: sequence softmax
[10:38:51] I1040 17:38:51.246898 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=0
[10:38:51] I1040 17:38:51.246970 29487 LayerGradUtil.cpp:724] cost 34.5668
[10:38:51] I1040 17:38:51.247009 29487 LayerGradUtil.cpp:43] sequence_softmax layer_0 step=1.12568e-06 cost1=34.567 cost2=34.5667 true_delta=0.000343323 analytic_delta=0.000345668 diff=-0.00678559
[10:38:51] I1040 17:38:51.247026 29487 LayerGradUtil.cpp:684] layer_type=addto useGpu=1
[10:38:51] I1040 17:38:51.248395 29487 LayerGradUtil.cpp:724] cost 3.73835
[10:38:51] I1040 17:38:51.253573 29487 LayerGradUtil.cpp:43] sequence_softmax layer_0 step=1e-06 cost1=3.73864 cost2=3.73807 true_delta=0.000561237 analytic_delta=0.000561875 diff=-0.00113416
[10:38:51] [ FAILED ] SequenceSoftmaxActivation.activation (12 ms)
[10:38:51] [----------] 1 test from SequenceSoftmaxActivation (12 ms total)
[10:38:51]
[10:38:51] [----------] Global test environment tear-down
[10:38:51] [==========] 2 tests from 2 test cases ran. (157 ms total)
[10:38:51] [ PASSED ] 1 test.
[10:38:51] [ FAILED ] 1 test, listed below:
[10:38:51] [ FAILED ] SequenceSoftmaxActivation.activation
[10:38:51]
[10:38:51] 1 FAILED TEST