Problem

In PaddlePaddle’s Design, one Operator may have multiple kernels. Users may have some personal preference to choose a certain type of kernel for an operator, such as force_cpu to choose a CPU kernel, use_cudnn to choose a CUDNN kernel, we need to provide a way for users to do this.

In the current design, we use KernelType to describe one kernel.

struct KernelType {
  Place place_;
  DataType data_type_;
  LayoutType layout_;
};

place_ data_type_ and layout_ can be got from the input tensors of the operator, GetActualKernelType(inputs) use inputs to infer the proper kernel key that fit the incoming data, but users can not directly configure it.

The design also provides a virtual method GetExpectedKernelType that user can overload and use to choose the KernelType they want to use.

So we should send the information user defined in proto to GetExpectedKernelType for choosing a kernel.

The problem is, how should we define and send the information for GetExpectedKernelType to use?

Solution

Potential choice

1. Do nothing, let the user add the information they want to operator‘s attribute and get them inside GetExpectedKernelType, this can work properly. But there is a little problem that users may define many kinds of hints for the same purpose, such as force_cpu, use_cpu, cpu_kernel to choose CPU kernel, and use_cudnn, force_cudnn, cudnn_kernel to choose CUDNN kernel.
2. Pre-define all the needed option and use a single attr key such as kernel_hint for the user, this is not so flexible if the user wants to define some more kind of hint.

Final choice

To provide enough flexibility while avoiding confusion definition, we can define some global constants for these attribute names, such as force_cpu, use_cudnn, use_mkldnn for a user to choose.

In C++

const std::string kForceCPU = "force_cpu";
const std::string kUseCUDNN = "use_cudnn";
const std::string kUseMKLDNN = "use_mkldnn";

KernelType GetExpectedKernelType() {
  if (Attr<bool>(kForceCPU)) {
    return KernelType(CPUPlace, ...)
  } else {
    ...
  }
}

In Python code

FORCE_CPU = core.kForceCPU()

def xx_layer(..., force_cpu=false):
  layer_helper = LayerHelper(...)
  layer_helper.append_op(
    type="xx",
    attr={FORCE_CPU: force_cpu})