test=develop

test=develop

test=develop

added depthwise conv mkldnn pass which for MKLDNN changes depthwise_conv operator to conv operator because for mkldnn this is the same api
test=develop

test=develop

test=develop

* fix nccl2 lars dist support

* put lars in momentum op

* add tests lars

* fix ci

* fix cpu kernel

* soft warning

* remove lars in test_recognize_digits.py

* move to another op

* add file

* update api.spec test=develop

* update test=develop

* fix api.spec test=develop

* wip

* wip, finish grad merge ops

* wip, finish graph build

* wip test running

* work on 1 gpu

* workable version

* update

* fix tests

* fuse broadcast op

* fix compile failed

* refine

* add batch merge test mnist

* fix CI test=develop

* fix build

* use independent bn params for batch merge test=develop

* update api.spec

* follow comments and for test

* wip

* refine tests test=develop

* follow comments test=develop

* remove startup bn modify test=develop

* follow comments test=develop

* fix merge test=develop

* fix graph num hang test=develop

* re-enable tests test=develop

* re-enable graph num check test=develop

* fix multi device pass role check test=develop

test=develop

This reverts commit 0e722c5e

test=develop

MKLDNN conv + elementwise_add fusion: UT for missing bias added. UTs refactored. Some minor changes in the pass

test=develop

test=develop

test=develop

macro used.

test=develop

MKLDNN conv + elementwise_add fusion: skip connection attribute renamed. Comments about patterns added.

test=develop

test=develop

MKLDNN conv + elementwise_add fusion: some refactoring: consts, function calls instead of constant values

MKLDNN conv + elementwise_add fusion: graph is corrected based on actual argument name, not formal argument name

MKLDNN conv + elementwise_add fusion: output and elemwise param share data in conv primitive. Output is properly allocated

MKLDNN conv + elementwise_add fusion: inputs in pass modified. Support for new conv parameter. UTs corrected