[LITE][COMPILE] Fix clang compile slow for arm cpu and abnormal exit for opencl. test=develop (#3763)

* [LITE][COMPILE] Fix clang compile slow for arm cpu and abnormal exit for opencl. test=develop

* add note for compilation option. test=develop