Created by: songyouwei
This PR add Linear
layer, a fully-connected linear transformation layer.
Different from FC
layer, Linear
layer takes only one Tensor
input.
The Linear
layer uses matmul
op while FC
layer uses mul
op, so they have different output shape.
-
Linear
multiplies input tensor with weight matrix and produces an output Tensor of shape[N, *, output_dim]
, whereN
is batch size and*
means any number of additional dimensions. -
FC
can take one or multipleTensor
as its inputs, and output shape is[N, size]
, whereN
is batch size.