Created by: qpfhuan

执行

CUDA_VISIBLE_DEVICES=0 python tools/cpp_infer.py --config_path tools/cpp_demo.yml --model_path output/cpp_infer_object365_cascade_rcnn_cls_aware_r200_vd_fpn_dcnv2_nonlocal_softnms/ --infer_img demo/orange_71.jpg

使用obj365训练的cascade模型导出后，使用cpp_infer进行Inference报错。（已将模型的softnms部分改成nms） cpp_demo.yml中， use_python_inference设置为False时报错。但设置为True时为可以运行。inference time 为 290.35 ms 一张图, 请问如果必须要使用python_inference的话，还可以如何加速。

以下为cpp_demo.yml设置，以及相应报错

use_python_inference: False # whether to use python inference
mode: fluid # trt_fp32, trt_fp16, trt_int8, fluid
arch: RCNN # YOLO, SSD, RCNN, RetinaNet
min_subgraph_size: 40 # need 3 for YOLO arch

# visualize the predicted image
metric: COCO # COCO, VOC
draw_threshold: 0.5

Preprocess:
- type: Resize
  target_size: 640
  max_size: 640
- type: Normalize
  mean:
  - 0.485
  - 0.456
  - 0.406
  std:
  - 0.229
  - 0.224
  - 0.225
  is_scale: True
- type: Permute
  to_bgr: False
- type: PadStride
  stride: 0 # set 32 on FPN and 128 on RetinaNet











2020-05-18 17:17:36,189-INFO: The architecture is RCNN
2020-05-18 17:17:36,190-INFO: Extra info: im_info, im_shape
2020-05-18 17:17:36,194-INFO: min_subgraph_size = 40.
2020-05-18 17:17:36,194-INFO: Run inference by Fluid FP32.
I0518 17:17:36.705852  2811 analysis_predictor.cc:84] Profiler is deactivated, and no profiling report will be generated.
I0518 17:17:36.862859  2811 analysis_predictor.cc:833] MODEL VERSION: 1.7.2
I0518 17:17:36.862900  2811 analysis_predictor.cc:835] PREDICTOR VERSION: 1.7.2
--- Running analysis [ir_graph_build_pass]
--- Running analysis [ir_graph_clean_pass]
--- Running analysis [ir_analysis_pass]
--- Running IR pass [is_test_pass]
--- Running IR pass [simplify_with_basic_ops_pass]
--- Running IR pass [conv_affine_channel_fuse_pass]
--- Running IR pass [conv_eltwiseadd_affine_channel_fuse_pass]
--- Running IR pass [conv_bn_fuse_pass]
I0518 17:17:39.706467  2811 graph_pattern_detector.cc:101] ---  detected 142 subgraphs
--- Running IR pass [conv_eltwiseadd_bn_fuse_pass]
--- Running IR pass [multihead_matmul_fuse_pass]
--- Running IR pass [fc_fuse_pass]
I0518 17:18:02.741768  2811 graph_pattern_detector.cc:101] ---  detected 6 subgraphs
I0518 17:18:02.755161  2811 graph_pattern_detector.cc:101] ---  detected 6 subgraphs
--- Running IR pass [fc_elementwise_layernorm_fuse_pass]
--- Running IR pass [conv_elementwise_add_act_fuse_pass]
I0518 17:18:03.398761  2811 graph_pattern_detector.cc:101] ---  detected 77 subgraphs
--- Running IR pass [conv_elementwise_add2_act_fuse_pass]
I0518 17:18:03.702769  2811 graph_pattern_detector.cc:101] ---  detected 66 subgraphs
--- Running IR pass [conv_elementwise_add_fuse_pass]
I0518 17:18:03.779069  2811 graph_pattern_detector.cc:101] ---  detected 85 subgraphs
--- Running IR pass [transpose_flatten_concat_fuse_pass]
--- Running IR pass [runtime_context_cache_pass]
--- Running analysis [ir_params_sync_among_devices_pass]
I0518 17:18:03.842639  2811 ir_params_sync_among_devices_pass.cc:41] Sync params from CPU to GPU
--- Running analysis [adjust_cudnn_workspace_size_pass]
--- Running analysis [inference_op_replace_pass]
--- Running analysis [ir_graph_to_program_pass]
I0518 17:18:04.405479  2811 analysis_predictor.cc:462] ======= optimize end =======
2020-05-18 17:18:04,413-INFO: warmup...
W0518 17:18:04.662607  2811 device_context.cc:237] Please NOTE: device: 0, CUDA Capability: 61, Driver API Version: 10.1, Runtime API Version: 9.0
W0518 17:18:04.672266  2811 device_context.cc:245] device: 0, cuDNN Version: 7.0.
W0518 17:18:04.672323  2811 device_context.cc:271] WARNING: device: 0. The installed Paddle is compiled with CUDNN 7.3, but CUDNN version in your machine is 7.0, which may cause serious incompatible bug. Please recompile or reinstall Paddle with compatible CUDNN version.
W0518 17:18:05.723701  2811 naive_executor.cc:45] The NaiveExecutor can not work properly if the cmake flag ON_INFER is not set.
W0518 17:18:05.723738  2811 naive_executor.cc:47] Unlike the training phase, all the scopes and variables will be reused to save the allocation overhead.
W0518 17:18:05.723744  2811 naive_executor.cc:50] Please re-compile the inference library by setting the cmake flag ON_INFER=ON if you are running Paddle Inference
Traceback (most recent call last):
  File "tools/cpp_infer.py", line 319, in <module>
    infer()
  File "tools/cpp_infer.py", line 252, in infer
    outs = predict.run(inputs)
paddle.fluid.core_avx.EnforceNotMet: 

--------------------------------------------
C++ Call Stacks (More useful to developers):
--------------------------------------------
0   std::string paddle::platform::GetTraceBackString<char const*>(char const*&&, char const*, int)
1   paddle::platform::EnforceNotMet::EnforceNotMet(std::__exception_ptr::exception_ptr, char const*, int)
2   cudnnActivationStruct* paddle::platform::ScopedActivationDescriptor::descriptor<float>(std::string const&, double)
3   paddle::operators::CUDNNConvFusionOpKernel<float>::Compute(paddle::framework::ExecutionContext const&) const
4   std::_Function_handler<void (paddle::framework::ExecutionContext const&), paddle::framework::OpKernelRegistrarFunctor<paddle::platform::CUDAPlace, false, 0ul, paddle::operators::CUDNNConvFusionOpKernel<float>, paddle::operators::CUDNNConvFusionOpKernel<double> >::operator()(char const*, char const*, int) const::{lambda(paddle::framework::ExecutionContext const&)#1}>::_M_invoke(std::_Any_data const&, paddle::framework::ExecutionContext const&)
5   paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&, paddle::framework::RuntimeContext*) const
6   paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, paddle::platform::Place const&) const
7   paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, paddle::platform::Place const&)
8   paddle::framework::NaiveExecutor::Run()
9   paddle::AnalysisPredictor::Run(std::vector<paddle::PaddleTensor, std::allocator<paddle::PaddleTensor> > const&, std::vector<paddle::PaddleTensor, std::allocator<paddle::PaddleTensor> >*, int)

------------------------------------------
Python Call Stacks (More useful to users):
------------------------------------------
  File "/usr/local/lib/python3.6/dist-packages/paddle/fluid/framework.py", line 2525, in append_op
    attrs=kwargs.get("attrs", None))
  File "/usr/local/lib/python3.6/dist-packages/paddle/fluid/layer_helper.py", line 43, in append_op
    return self.main_program.current_block().append_op(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/paddle/fluid/layers/nn.py", line 1403, in conv2d
    "data_format": data_format,
  File "/home/qipengfei7/qpf/baidu_paddle/PaddleDetection/ppdet/modeling/backbones/resnet.py", line 162, in _conv_norm
    name=_name + '.conv2d.output.1')
  File "/home/qipengfei7/qpf/baidu_paddle/PaddleDetection/ppdet/modeling/backbones/resnet.py", line 260, in _shortcut
    return self._conv_norm(input, ch_out, 1, stride, name=name)
  File "/home/qipengfei7/qpf/baidu_paddle/PaddleDetection/ppdet/modeling/backbones/resnet.py", line 319, in bottleneck
    name=shortcut_name)
  File "/home/qipengfei7/qpf/baidu_paddle/PaddleDetection/ppdet/modeling/backbones/resnet.py", line 398, in layer_warp
    gcb_name=gcb_name)
  File "/home/qipengfei7/qpf/baidu_paddle/PaddleDetection/ppdet/modeling/backbones/resnet.py", line 455, in __call__
    res = self.layer_warp(res, i)
  File "/home/qipengfei7/qpf/baidu_paddle/PaddleDetection/ppdet/modeling/architectures/cascade_rcnn_cls_aware.py", line 91, in build
    body_feats = self.backbone(im)
  File "/home/qipengfei7/qpf/baidu_paddle/PaddleDetection/ppdet/modeling/architectures/cascade_rcnn_cls_aware.py", line 217, in test
    return self.build(feed_vars, 'test')
  File "tools/export_model.py", line 97, in main
    test_fetches = model.test(feed_vars)
  File "tools/export_model.py", line 114, in <module>
    main()

----------------------
Error Message Summary:
----------------------
Error: An error occurred here. There is no accurate error hint for this error yet. We are continuously in the process of increasing hint for this kind of error check. It would be helpful if you could inform us of how this conversion went by opening a github issue. And we will resolve it with high priority.
  - New issue link: https://github.com/PaddlePaddle/Paddle/issues/new
  - Recommended issue content: all error stack information
  [Hint: CUDNN_STATUS_BAD_PARAM] at (/paddle/paddle/fluid/platform/cudnn_helper.h:463)
  [operator < conv2d_fusion > error]