Created by: lidanqing-intel

Related to #20842 (closed) ernie large model issue. Problem: Slow latency. We are trying to use 3-dimension MKLDNN mul.

Hi, sorry, I forgot I was on my own machine i9 and got slow latency. On 6148, num_threads=10 latency is as follow. The latency with num_threads=1 is in later comments.

I1101 10:09:51.044067  5401 analysis_predictor.cc:474] ======= optimize end =======
I1101 10:09:51.050020  5401 inference.cc:213] Load 10 samples from /home/lidanqing/data/test_ds_10
I1101 10:09:58.605739  5401 inference.cc:353] Run 10 samples, average latency: 755.564 ms per sample.
I1101 10:09:58.605875  5401 inference.cc:358] Run 5 samples, average latency [exclude 5 warmup steps]: 559.852
ms per sample.