* refine parallel_exe
test=develop

* rename shared_var_device

* code refine

* add test_weight_decay

* remove Sort
test=develop

* Add SortForReduce
test=develop

* code refine
test=develop

* follow comment
test=develop

add module cleanup
test=develop

test=develop

This reverts commit 7f6d8ace.

test=develop

* code style fix, test=develop

test=develop

test=develop

* wip

* wip

* wip

* wip for test

* add fp16 tests test=develop

* fix cpu build test=develop

* fix test=develop

* fix py3 tests test=develop

* fix lr_scheduler dtype test=develop

* fix test=dvelop

* test fix ci compile test=develop

* fix build and merge test=develop

* fallback momentumop change to general test=develop

* make fp16 lr schedule simple test=develop

* fix ut test=develop

* fix tests test=develop

* remove fp16 learning rate cast test=develop

Stack OP has much higher CPU cost than expected in release mode.
Caused by DebugStringEx() in base class OperatorWithKernel. Actually
this issue occur for each OP which hasn't implement it's own
GetExpectedKernelType().

test=develop

* refine tensor
test=develop

* refine tensor
test=develop

* fix device_context log
test=develop

test=develop

test=develop

* Add Temporal Allocator

* add Temporay Allocator to DeviceContext
test=develop

* code refine
test=develop

* fix mean_iou
test=develop

* Add DeviceTemporaryAllocator
test=develop

* fix conv_op bug
test=develop

* small fix
test=develop

* code refine
test=develop

* log refine
test=develop

* fix unit test
test=develop

* move double check

* refine concat_and_split
test=develop

* add limit_of_temporary_allocation
test=develop

* fix name
test=develop

fix comment
test=develop

test=develop

test=develop

test=develop

make forward declaration
test=develop

This reverts commit 3d750f9c.

test=develop

test=develop

test why MAC CI fail again
test=develop

* refine parallel_executor

* remove uncessary code
test=develop

* wip

* wip

* wip

* wip for test

* add fp16 tests test=develop

* fix cpu build test=develop

* fix test=develop

* fix py3 tests test=develop

* fix lr_scheduler dtype test=develop

* fix test=dvelop

* test fix ci compile test=develop

* fix build and merge test=develop

* fallback momentumop change to general test=develop

try to fix mac compile bug?
test=develop

* MKLDNN residual connection fuse: conv op reused

test=develop

* MKLDNN residual connection fuse: added prints for checking fuse

test=develop

* MKLDNN residual connection fuse: add more prints

test=develop

* MKLDNN residual connection fuse: add hash function. test=develop

* MKLDNN residual connection fuse: add hash to elementwise_add

test=develop

* MKLDNN residual connection fuse: add more hashes. test=develop

* MKLDNN residual connection fuse: added hashes to relu

test=develop

* MKLDNN residual connection fuse: do not fuse when fuse_relu is on

* MKLDNN residual connection fuse: check if fuse_relu attribute is set

test=develop

* MKLDNN residual connection fuse: comment out some printouts

* MKLDNN residual connection fuse: remove unused functions in the pass code

* MKLDNN residual connection fuse: delete commented hashes and printouts

* MKLDNN residual connection fuse: remove unnecessary includes. test=develop

test=develop

test=develop

test=develop

test=develop

test=develop

test=develop

* Enable mul operator for a ngraph
test=develop

* Enable activation ops test
test=develop

* Remove unused line
test=develop

test=develop

test=develop

test=develop

test=develop

test=develop

test=develop