Created by: sneaxiy
Refine ReferenceCountPass
by:
- not finding downstream
ComputationOpHandles
if some last live ops of a certain var is notComputationOpHandle
. This is because in allreduce mode, this case only happens to parameter, and the last lived ops of parameters must be optimizer ops; in reduce mode, this case only happens to parameter too, and the last lived ops of parameter must be optimizer ops (if the gpu card is responsible to update the parameter) or none (if the gpu card is not responsible to update the parameter, so that the last lived ops would beBroadcast
). - do not precede to find last live ops. Preceding is wrong, because last lived ops must be the generated op of var if pending ops are not the last lived ops.