Created by: tpatejko

I've been working on fusion pass that fuses MKLDNN convolution with elementwise_add operator, and I use framework::ir graph and pass infrastructure.

The fusion pass I've been working on transforms the following set of operations:

y = conv(W, z)
out = elementwise_add(x, y)

into

y = x
y = conv`(W, z)

where

conv`(W, z) = y + conv(W, z)   <- done in MKLDNN

because MKLDNN convolution primitive that fuses elementwise_add expects data, that it will add convolution output to, already in the output memory.

I'm having some trouble implementing assignment operation y = x. x and y are nodes of variable type in a graph, so creating an edge between them is not correct, because, if I understand correctly, nodes allow only operations in their adjacency lists.

These are solutions I could think of:

1. I could use PaddlePaddle assign_op operator so it would look like this: y = assign_op(x). However, it copies x to y so it introduces performance overhead.
2. I could add additional input parameter to PaddlePaddle MKLDNN convolution operator that takes x as an argument and initializes output memory of MKLDNN convolution primitive.
3. I could turn y into a reference to x but I'm not sure how to do it without abusing graph API (making two Node* pointers pointing into a single std::unique_ptr<Node> object, as, if I'm not mistaken, nodes container is implemented as std::map<Node*, std::unique_ptr<Node>>).

So, I guess option 2. is the most reasonable one. What do you think about it? Do you have any other options?