* fix nccl2 lars dist support

* put lars in momentum op

* add tests lars

* fix ci

* fix cpu kernel

* soft warning

* remove lars in test_recognize_digits.py

* move to another op

* add file

* update api.spec test=develop

* update test=develop

* fix api.spec test=develop

* wip

* wip, finish grad merge ops

* wip, finish graph build

* wip test running

* work on 1 gpu

* workable version

* update

* fix tests

* fuse broadcast op

* fix compile failed

* refine

* add batch merge test mnist

* fix CI test=develop

* fix build

* use independent bn params for batch merge test=develop

* update api.spec

* follow comments and for test

* wip

* refine tests test=develop

* follow comments test=develop

* remove startup bn modify test=develop

* follow comments test=develop

* fix merge test=develop

test=release/1.1

[1.1] Cpu for 1.1 merge with shape

test=develop

* add strategy doc
test=develop

* fix doc
test=develop

* add ParallelExecutor arg doc
test=develop

test=develop

test=develop

test=develop

This reverts commit 21bb9e91, reversing
changes made to 3fa68dc1.

test=develop

test=develop

* hide operator API

* update

* update api.spec

* fix merge

* fix test

* wip

* clean up

* should fix running with memopt

* add ut

* mark lr schedule op role

* hide lr_schedule_guard

* use op_role_var instead of ufind

* unify dist test name

* wip for py3 support

* fix var deref

* fix python3 mem_opt order

* remove comments

* Add Preface

* Add demo code

* Save file

* Refine code

* seems can work

* use elementwise strategy

* Use ElementwiseComputeEx

* Add comments

* extract functions from operator

* Refine code

* Follow comment

* code refine

* add op_fuse  pass

* add backward

* code refine

* use TopologySortOperations

* follow comments

* refine IsFusible

* code enhance

* fix op_fusion_pass

* refine code

* refine fuse_elemwise_act_op

* adjust the input and output

* refine logic

* add intermediate_edge

* disable inplace

* follow comments

* refine logic

* follow comments

* Remove the removable IntermediateOut

* change strategy

* code refine

* enable fuse backward

* code refine

* code refine

* rename unit test

* follow comments

This reverts commit 1f270275.

* Support int8 type.

This reverts commit b2df1700.

* dist transpiler add control dependency var between send and recv

* fix async deps

* follow comments and refine

* fix deps connect for rpc ops

* Add Python Callstacks when Op::Run error

* Skip op with sub-block

* refactor: refine callstack info's format

* Reshape only support matrix

* Polish Python code

* Fix UT

* Fix Py3

Optimize selected rows for dist lookup table with rwlock 

* hide misc APIs

* update

* fix transformer test

* update API.spec