Created by: zhiqiu
In Paddle, there are several operators that registered some variables as inputs while never use them during computation in kernel. For example, instance_norm_grad
registered kernel
in InstanceNormGradMaker
, which is not used in InstanceNormGradKernel
.
This could have two problems,
- Unnecessary memory consumption, especially GPU memory. Paddle uses Garbage Collection(GC) to save GPU memory, and GC relies on reference count calculation. When variables are registered as inputs, their reference count increased, thus these unused variables' memory can not be released.
- This makes the code confusing.
In order to reduce unused variables, this PR,
- Count variable usage in
InputVar
,MultiInput
,MultiInputVar
, compare them with op's inputs registered, and report unused variables during operator running. - Add
unused_var_check
to CI job. (PR_CI_Coverage) - Keep a white list which contains the existing operators that have unused variables. These operators should be fixed in the future.
See wiki for more details.