Very slow reading data for large batch sizes in C-API inference
Created by: wojtuss
Reading data in C-API inference (via PaddlePredictor::Run() method) is very slow and appears to be a bottleneck for inference performance.
Is the following piece of code required? If yes, can we optimize it somehow?
https://github.com/PaddlePaddle/Paddle/blob/8cd17c04c1d0eadebb96c2a9dc887c8411e7e6c8/paddle/fluid/inference/api/api_impl.cc#L197
Can we optimize reading data also in any other way?