* fix nccl2 lars dist support

* put lars in momentum op

* add tests lars

* fix ci

* fix cpu kernel

* soft warning

* remove lars in test_recognize_digits.py

* move to another op

* add file

* update api.spec test=develop

* update test=develop

* fix api.spec test=develop

* wip

* wip, finish grad merge ops

* wip, finish graph build

* wip test running

* work on 1 gpu

* workable version

* update

* fix tests

* fuse broadcast op

* fix compile failed

* refine

* add batch merge test mnist

* fix CI test=develop

* fix build

* use independent bn params for batch merge test=develop

* update api.spec

* follow comments and for test

* wip

* refine tests test=develop

* follow comments test=develop

* remove startup bn modify test=develop

* follow comments test=develop

* fix merge test=develop

* add check for opt op

* fix opt op
test=develop

* fix test fail
test=develop

* fix optimization doc
test=develop

* test=develop

* Fix serious bug in nesterov momentum optimizer.

* Fix attribute naming for momentum_op

* Fix minor typo in comment

* Fix attribute name

* Fix names in test_optimizer

* Fix python wrapper

* Changing learning rate from attribute to input(float)
* Removing obsolete code

Fix #3902

* use operator context

* optimize code

* update net infershape

* update InferShape

* disable override InferShape(scope) in OperatorBase

* change InferShapeImpl to InferShape

* add template to OperatorContext Input/Output

* merge Input InputVar, Output OutputVar

* change Inputs to MultiInput

* fix conflict

* fix MultiInput bugs and add unit test

* rename KernelContext to ExecutionContext

* clean code

* change InferShape to protected

* fix template bug

* refine code

* use InputVar instead of Input<Variable>

* typo

* optimize code

Make implement an operator less noisy.

* a simplest SGD op

* add inputs

* add ut for multiple inputs

* fix AddToLayer

* op_desc -> op_proto

* CreateArgumentOffsetMap -> CreateInOutOffsetMap

* move CreateInOutOffsetMap from OperatorBase to op registry

* arg_idxs_ -> in_out_idxs_

* Convert `op` --> `operators`
* Remove AddType in OpProtoMaker, because type is part of registry.
* Rename CPU_OR_GPU --> DEVICE_TYPE in registry macro.

* Refine register methods, make Op can get rid of whole-archieve
* `USE_OP` before a op is used.
* Add unittest for add_op.