* add nccl2 dist api

* update apispec

* update

* update api spec

* wip

* clean up

* should fix running with memopt

* add ut

* mark lr schedule op role

* hide lr_schedule_guard

* use op_role_var instead of ufind

* unify dist test name

* wip for py3 support

* fix var deref

* fix python3 mem_opt order

* remove comments

* hide program APIs

* fix merge error

* update

* fix dist table send hang problem

* revert sync_mode config

* fix async send table

* trainer auto wait pserver port ready

* add file

* fix docstring

* add option to not wait

* update api spec

* clean

* fix test hang

Resume prelu_op_test in python2

* fix dist train reduce mode

* fix previous fix

* fix dist transpile with memopt

* update api.spec

* polish dist transpile api

* update apispec

* update apispec

* refine dist train RPC deps

* clean up

* clean up

* fix ut

* remove input for fetch_barrier

* follow comments

* dist transpiler add control dependency var between send and recv

* fix async deps

* follow comments and refine

* fix deps connect for rpc ops

* add async dist tests

* update delta

* fix transformer test

* refine rmsprop transpile

* update

* fix dist seresnet

Port dist_transpiler new added codes
Port ut for clone desc

Add dependency to send recv

* dist transpile can pass startup program by argument

* update API.spec

* hide misc APIs

* update

* fix transformer test

* update API.spec

* fix dist train with rmsprop

* add rmsprop transpiler test

* update by comment

Fix str and bytes problem
Fix sorted problem
Fix math problem
Fix CI problem