test=develop

add trt int8 support

test=develop, refine_error_message for data type

Fix prefetch one parameter

Refine INT8 calibration API

move ngraph_bridge to ngraph directory 

test=develop

Add Shuffle Channel Operator

test=develop

jit benchmark use tensor with alignment

Optimize cpp reader

test=develop

* fix mistakes in merge_ids, test=develop

Add check: conv_fusion op runs with cudnn version > 7100 .

* enable ngraph_engine_op
test=develop

* merge develop test=develop

* avoid const_cast test=develop

* rm ngraph_operator test=develop

* Added TODO to move EnableNgraph test=develop

* Add TODO to remove const_cast test=develop

test=develop

* Revert "set constant for loss"

This reverts commit 167933f6.

* Revert "remove workspace_handle"
test=develop
This reverts commit b4aca8ed.

test=develop

* Refine the beam_search op and test.

* A basic CUDA implementation of beam_search for small batch_size.

* Implement CUDA kernel for beam_search_op.

* Use multiple CUDA threads in the same block to select the top beam.

* Update the python api of beam_search op.

* Enable extend function in CPU kernel of beam_search op.

* Unify the CUDA codes.
test=develop

* Unify the CPU kernel of beam_search op.

* Ensure the seletced items of beam_search_op's CPU kernel sorted by scores.

* Update the description of beam_search in API.spec.

* Enable the use of CUDA kernel in beam_search op.

* Exclude the beam_search's CUDA unittest when there is no CUDA gpu, and delete some debuging statements.
test=develop

* Follow comments.
test=develop

* Call the CPU kernel for beam_search op when batch_size > 4.
test=develop

* Remove the except of is_empty op in PrepareData.
test=develop

1. graph and program_desc alignment
2. trt stream

test=develop

Disable eager deletion unittest temporarily since random failure.

Fix bug when user set CUDA_VISIBLE_DEVICES be empty and run CPU-only models

Turn on remove_unnecessary_lock by default