test=develop

test=develop

test=develop

add module cleanup
test=develop

test=develop

This reverts commit 7f6d8ace.

* code style fix, test=develop

test=develop

* wip

* wip

* wip

* wip for test

* add fp16 tests test=develop

* fix cpu build test=develop

* fix test=develop

* fix py3 tests test=develop

* fix lr_scheduler dtype test=develop

* fix test=dvelop

* test fix ci compile test=develop

* fix build and merge test=develop

* fallback momentumop change to general test=develop

* make fp16 lr schedule simple test=develop

* fix ut test=develop

* fix tests test=develop

* remove fp16 learning rate cast test=develop

Stack OP has much higher CPU cost than expected in release mode.
Caused by DebugStringEx() in base class OperatorWithKernel. Actually
this issue occur for each OP which hasn't implement it's own
GetExpectedKernelType().

test=develop

* refine tensor
test=develop

* refine tensor
test=develop

* fix device_context log
test=develop

test=develop

* Add Temporal Allocator

* add Temporay Allocator to DeviceContext
test=develop

* code refine
test=develop

* fix mean_iou
test=develop

* Add DeviceTemporaryAllocator
test=develop

* fix conv_op bug
test=develop

* small fix
test=develop

* code refine
test=develop

* log refine
test=develop

* fix unit test
test=develop

* move double check

* refine concat_and_split
test=develop

* add limit_of_temporary_allocation
test=develop

* fix name
test=develop

fix comment
test=develop

test=develop

test=develop

test=develop

This reverts commit 3d750f9c.

test=develop

test=develop

* refine parallel_executor

* remove uncessary code
test=develop

* wip

* wip

* wip

* wip for test

* add fp16 tests test=develop

* fix cpu build test=develop

* fix test=develop

* fix py3 tests test=develop

* fix lr_scheduler dtype test=develop

* fix test=dvelop

* test fix ci compile test=develop

* fix build and merge test=develop

* fallback momentumop change to general test=develop

* MKLDNN residual connection fuse: conv op reused

test=develop

* MKLDNN residual connection fuse: added prints for checking fuse

test=develop

* MKLDNN residual connection fuse: add more prints

test=develop

* MKLDNN residual connection fuse: add hash function. test=develop

* MKLDNN residual connection fuse: add hash to elementwise_add

test=develop

* MKLDNN residual connection fuse: add more hashes. test=develop

* MKLDNN residual connection fuse: added hashes to relu

test=develop

* MKLDNN residual connection fuse: do not fuse when fuse_relu is on

* MKLDNN residual connection fuse: check if fuse_relu attribute is set

test=develop

* MKLDNN residual connection fuse: comment out some printouts

* MKLDNN residual connection fuse: remove unused functions in the pass code

* MKLDNN residual connection fuse: delete commented hashes and printouts

* MKLDNN residual connection fuse: remove unnecessary includes. test=develop

test=develop

test=develop

test=develop

test=develop

test=develop

* Enable mul operator for a ngraph
test=develop

* Enable activation ops test
test=develop

* Remove unused line
test=develop

test=develop

test=develop

test=develop

test=develop

test=develop

test=develop

* follow comments. test=develop

* Fix typo

* fix compile error. test=develop

* merge develop branch. test=develop

* Remove set_equal

* Polish code

* Delete unused functions

test=develop

* polish code. test=develop

* follow comment

* polish code.

* fix windows compile error. test=develop

* fix op handle.

* rerun ci. test=develop

* rerun ci. test=develop

* rerun macci. test=develop

* polish code. test=develop

* rewrite sort code. test=develop

* remove unused code. test=develop

* fix tests. test=develop

* fix conflict. test=develop

* follow comment. test=develop

* merge develop branch. test=develop

* fix tests. test=develop

* remove ToTypeIndex. test=develop

* rerun ci. test=develop

test=develop

test=develop

test=develop