Created by: typhoonzero
As shown in the graph output by parallel executor if we use piecewise_decay
there's a chance that some conditions may be run after sgd_op
since there is no dependence between them. After this change, the graph will be like:
This will harm the performance, will do the enhance ( remove assign ops when building graph).