* add nccl2 dist api

* update apispec

* update

* update api spec

* wip

* clean up

* should fix running with memopt

* add ut

* mark lr schedule op role

* hide lr_schedule_guard

* use op_role_var instead of ufind

* unify dist test name

* wip for py3 support

* fix var deref

* fix python3 mem_opt order

* remove comments

* hide program APIs

* fix merge error

* update

* fix dist table send hang problem

* revert sync_mode config

* fix async send table

* trainer auto wait pserver port ready

* add file

* fix docstring

* add option to not wait

* update api spec

* clean

* fix test hang

Resume prelu_op_test in python2

* fix dist train reduce mode

* fix previous fix

* fix dist transpile with memopt

* update api.spec

* polish dist transpile api

* update apispec

* update apispec

* refine dist train RPC deps

* clean up

* clean up

* fix ut

* remove input for fetch_barrier

* follow comments

* dist transpiler add control dependency var between send and recv

* fix async deps

* follow comments and refine

* fix deps connect for rpc ops

* Fuse Convolution and Eltwise Add into Conv+Bias

* Reduce bias branching at conv_mkldnn_op

* Add MKLDNN build checks for Conv Bias

* Conv-bias: check if bias input exist befor assignment

* Conv-bias: Remove Bias dim check from infershape

It was causing conv3d test to crash upon\ncalling HasInput(Bias)

* add async dist tests

* update delta

* fix transformer test

* refine rmsprop transpile

* update

* fix dist seresnet

Port dist_transpiler new added codes
Port ut for clone desc

Add dependency to send recv