Paddlle_TRT推理报错
Created by: DYJNG
1)PaddlePaddle版本:PaddlePaddle-gpu-1.8.2、TensorRT-6.0.1.5 3)GPU:1080Ti、CUDA9.0、CUDNN7.6 4)系统环境:Python3.6.6 -预测信息 1)Python预测,CentOS7系统下编译预测库
- 问题描述:使用Paddle-TRT 运行paddleslim量化后的模型,设置precision_mode=AnalysisConfig.Precision.Int8时,能够完成推理,但是速度相比于不启用TRT差别不大,比量化前的模型推理速度甚至还慢一些。此外,在设置precision_mode=AnalysisConfig.Precision.Float32时,出现报错,报错信息如下:
--------------------------------------------
C++ Call Stacks (More useful to developers):
--------------------------------------------
0 std::string paddle::platform::GetTraceBackString<std::string>(std::string&&, char const*, int)
1 paddle::platform::EnforceNotMet::EnforceNotMet(paddle::platform::ErrorSummary const&, char const*, int)
2 paddle::inference::tensorrt::ElementwiseWeightOpConverter::operator()(paddle::framework::proto::OpDesc const&, paddle::framework::Scope const&, bool)
3 paddle::inference::tensorrt::OpConverter::ConvertOp(paddle::framework::proto::OpDesc const&, std::unordered_set<std::string, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::string> > const&, paddle::framework::Scope const&, paddle::inference::tensorrt::TensorRTEngine*, bool)
4 paddle::inference::tensorrt::OpConverter::ConvertBlockToTRTEngine(paddle::framework::BlockDesc*, paddle::framework::Scope const&, std::vector<std::string, std::allocator<std::string> > const&, std::unordered_set<std::string, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, paddle::inference::tensorrt::TensorRTEngine*)
5 paddle::inference::analysis::TensorRtSubgraphPass::CreateTensorRTOp(paddle::framework::ir::Node*, paddle::framework::ir::Graph*, std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> >*) const
6 paddle::inference::analysis::TensorRtSubgraphPass::ApplyImpl(paddle::framework::ir::Graph*) const
7 paddle::framework::ir::Pass::Apply(paddle::framework::ir::Graph*) const
8 paddle::inference::analysis::IRPassManager::Apply(std::unique_ptr<paddle::framework::ir::Graph, std::default_delete<paddle::framework::ir::Graph> >)
9 paddle::inference::analysis::IrAnalysisPass::RunImpl(paddle::inference::analysis::Argument*)
10 paddle::inference::analysis::Analyzer::RunAnalysis(paddle::inference::analysis::Argument*)
11 paddle::AnalysisPredictor::OptimizeInferenceProgram()
12 paddle::AnalysisPredictor::PrepareProgram(std::shared_ptr<paddle::framework::ProgramDesc> const&)
13 paddle::AnalysisPredictor::Init(std::shared_ptr<paddle::framework::Scope> const&, std::shared_ptr<paddle::framework::ProgramDesc> const&)
14 std::unique_ptr<paddle::PaddlePredictor, std::default_delete<paddle::PaddlePredictor> > paddle::CreatePaddlePredictor<paddle::AnalysisConfig, (paddle::PaddleEngineKind)2>(paddle::AnalysisConfig const&)
15 std::unique_ptr<paddle::PaddlePredictor, std::default_delete<paddle::PaddlePredictor> > paddle::CreatePaddlePredictor<paddle::AnalysisConfig>(paddle::AnalysisConfig const&)
----------------------
Error Message Summary:
----------------------
InvalidArgumentError: The size of bias's dims is 1 and bias's size is 1. TensorRT doesn't support this shape for Elementwise op! at (/home/.../Paddle-1.8.2/paddle/fluid/inference/tensorrt/convert/elementwise_op.cc:147)
请问该如何解决?