MobileNet-v1 on INT8 v2 performance gap problem
Created by: bingyanghuang
Description: In our internal benchmark test for INT8 v1 and INT8 v2 solution, on mobilenet-v1, batch size =128, INT8 v2 got about 14% performance regression on throughput compared with INT8 v1.
Potential Reason: 1. INT8 v1 benchmark data and INT8 v2 benchmark data tested with different validation team, so the different hardware configuration and testing environment diff may cause this final performance diff. 2. Some unknown technical bug may cause this performance regression.
To do: 1. Reproducing the INT8 v1 benchmark data on the same environment as INT8 v2 benchmark, and find the root cause for the environment diff influence. 2. Investigate the whether it exits potential bug for this performance regression.
@kbinias Could you help follow this issue and update your investigation. This investigation can be started after all the Top high priority work finished.