only mode

* add nccl2 dist api

* update apispec

* update

* update api spec

* wip

* clean up

* should fix running with memopt

* add ut

* mark lr schedule op role

* hide lr_schedule_guard

* use op_role_var instead of ufind

* unify dist test name

* wip for py3 support

* fix var deref

* fix python3 mem_opt order

* remove comments

* Add roi perspective transform.

* Add roi_perspective_transform_op.

* Fix code style.

* Add python api and fix doc.

* Fix API.spec

* Fix python api.

* Fix API.spec

* Move src to detection.

This reverts commit bd79e046, reversing
changes made to 6b4d290c.

Add warning when disable test_fuse_elewise_add_act_pass on Mac

* Optimize GRU with AVX instruction

* Clean code

* Add the Unitest and fix the align issue

* Remove the remanent part of the unitest part

* Code clean

* Fix the parameters length issue for fusion_gru to pass CI

* Change the default type as float32

* Add Preface

* Add demo code

* Save file

* Refine code

* seems can work

* use elementwise strategy

* Use ElementwiseComputeEx

* Add comments

* extract functions from operator

* Refine code

* Follow comment

* code refine

* add op_fuse  pass

* add backward

* code refine

* use TopologySortOperations

* follow comments

* refine IsFusible

* code enhance

* fix op_fusion_pass

* refine code

* refine fuse_elemwise_act_op

* adjust the input and output

* refine logic

* add intermediate_edge

* disable inplace

* follow comments

* refine logic

* follow comments

* Remove the removable IntermediateOut

* change strategy

* code refine

* enable fuse backward

* code refine

* code refine

* rename unit test

* follow comments

Sequence Scatter Op

In the future, we'd like to encourage user to save
everything during training.
This allows us to
1. Do more flexible optimization passes
2. Re-train and fune-tune

* add unit test for nested drnn

* add nested dyn_rnn

* refine while_op

* fix bug

* Add truncated gaussian initializer.

* Fix unitest.

* Update API.spec

* Fix code style and fix bug.

* Fix code style.

* Small fix.

* Add python api for expand op.

* Fix unitest.

* Remove 'out' from arguments and fix code style.

* fix API.spec

* Fix API

* Fix unitest

* loosen the restriction of output_size in conv2d_transpose

* test and docs

* fix code style

* fix ci test error

* bug fix

* fix python3 issue

* hide program APIs

* fix merge error

* update

* Add sequence_expand_as_op

* follow comment