The macro should be defined by compiler rather than by source.

test=develop

* fix inference on gpu out of mem

the transfer logic in operator.cc will keep creating new scopes.

test=develop

* picked pr

* "fix ci"

* Optimize CRF decoding with AVX/AVX2 instruction

* Enable the AVX2 flags for compiling

* Clean the code and decrease the count of multiply calculation

* Add the support of AVX512 instruction to optimize CRF Decoding

* Clean the code

* Enable the AVX512f flags for compiling

* Clean the code for the invaluable switch

* Fixed the issue to check AVX512F status

* Clean the code

* Add some explanation of the key points

* cudnn widndows

* "add comment"

* "windows support"

* "fix cmake error"

* remove patchelf in windowls

* "follow comment"

* wip

* use executor.complete to end trainer

* fix build

* fix build with distribute off

* fix typo

* fix cmake typo

* fix build

1. Add option WITH_AMD_GPU.
2. Add cmake/hip.cmake for HIP toolchain.
3. Some external module such as eigen may need HIP port.
4. Add macro hip_library/hip_binary/hip_test to cmake/generic.cmake.
5. Add one HIP source concat.hip.cu as an example. Each .cu may have its corresponding .hip.cu.

* Make CUPTI_LIB_PATH not passing by macro.
* Add missing header

* "add nccl enforce"

* Dev

* Update comment

* Add nccl test

* Follow comments