test=develop

test=develop

* test=release/1.4, fix hsigmoid dereference nullptr

* test=release/1.4, refine condition

* test=release/1.4, refine comments

test=release/1.4

Support ShuffleNet and MobileNet-v2, test=release/1.4

* cherry-pick 16763,test=release/1.4

* cherry-pick 16763, test=release/1.4

* fix some comments, include cosine_decay,l2_normalize,pixel_shuffle

* Add api.spec, test=develop

* update api.spec, test=develop

* add api.spec,test=develop

* test=develop

* test=develop

* fix conflict,test=develop

* Add Pixel shuffle OP (#15782)

* add pixel_shuffle op

* add pixel_shuffle op, test=develop

* rewrite code, test=develop

* delete useless comment, test=develop

* Refine pixel_shuffle_op and unit testing

* refine code,test=develop

* refine .cu,test=develop

* fix unittest,test=develop

* Fix unit testing
test=develop

* resolve conflict, test=develop

* fix test, test=develop

* fix API, test=develop

* fix test datatype bug,test=develop

* polish comments,test=develop

* add API,test=develop

* test=develop

* Add Pixel_Shuffle OP,test=develop

* support python3,test=develop

* add include memory to travis CI bug,test=develop

* cherry-pick 16763,15782 , test=release/1.4

test=release/1.4

* fix the bug of reusing different types of variables in memory_optimiz… (#16547)

* fix the bug of reusing different types of variables in memory_optimize_pass, test=develop

* disable SELECTED_ROWS AND LOD_TENSOR_ARRAY reusage, test=develop

* only use the latest version variable for inplace strategy (#16736)

* bug-fix, test=develop

* tweak code, test=develop

* cherry-pick PR 16547,16736,16739 test=release/1.4

* Imperative deep-first backward process (#16605)

* Fix bug of gradient interface

* shrink transformer

* Right transformer

* Change from width-first backward to deep-first backward process

test=develop

* Reverse iterator op's input

test=develop

* Polish code

* Change the iteration direction in ingrads' map slots

test=develop

* Polish code

test=develop

* test=develop, cherry-pick fix for transformer in dygraph

* test=develop, fix transformer in dygraph
/

test=release/1.4

test=release/1.4

test=release/1.4

* fix the bug of all_reduce_deps_pass
test=release/1.4

fix load bug about context.place

(cherry picked from commit bddb2cd3)

* fix preprocess script with processbar, integrity check and logs

* delete unnecessary empty lines, change function name

test=release/1.4

(cherry picked from commit bddb2cd3)

* Add linear learning warmup method

This warmup lr can be combinated with other learning rate strategies.
For example:
            decayed_lr = fluid.layers.linear_lr_warmup(
                fluid.layers.piecewise_decay(boundaries, lr_steps),
                warmup_steps, start_lr, end_lr)

* fix cudnn batch norm accuracy test=develop

* fix cudnn batch norm accuracy test=develop

* disable failed test for later fix test=develop

test=develop

test=develop

test=develop

test=develop

test=develop

* fix compiled
test=develop

* follow comments test=develop

test=develop

* Add DecoupledWeightDecay