diff --git a/doc/design/kernel_hint_design.md b/doc/design/kernel_hint_design.md index 75f0e4ea72779f9bf51048738a9af00a5e5600a5..da4856bb6a0ebdf289708469bd41a88031b8f34a 100644 --- a/doc/design/kernel_hint_design.md +++ b/doc/design/kernel_hint_design.md @@ -1,7 +1,7 @@ ## Problem -In PaddlePaddle's [Design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/switch_kernel.md), one Operator may have multiple kernels. Users may have some personal preference to choose a certain type of kernel for an operator, such as `force_cpu` to use a CPU kernel, `use_cudnn` to choose a CUDNN kernel, we need to provide a way for a user to do this. +In PaddlePaddle's [Design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/switch_kernel.md), one Operator may have multiple kernels. Users may have some personal preference to choose a certain type of kernel for an operator, such as `force_cpu` to choose a CPU kernel, `use_cudnn` to choose a CUDNN kernel, we need to provide a way for users to do this. -In the current design, we use KernelType to describe one kernel. +In the current design, we use KernelType to describe one kernel. ```cpp struct KernelType { @@ -10,33 +10,33 @@ struct KernelType { LayoutType layout_; }; ``` - `place_` `data_type_` and `layout_` can come from the input tensor of the operator, `GetActualKernelType(inputs)` use inputs to infer the proper kernel key that fit the incoming data, user can not config it. + `place_` `data_type_` and `layout_` can be got from the input tensors of the operator, `GetActualKernelType(inputs)` use inputs to infer the proper kernel key that fit the incoming data, but users can not directly configure it. -The design also provides a virtual method `GetExpectedKernelType` that user can overload and choose the KernelType they want to use. +The [design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/switch_kernel.md) also provides a virtual method `GetExpectedKernelType` that user can overload and use to choose the KernelType they want to use. -so, we should send the information user defined in proto to `GetExpectedKernelType` for choosing a kernel. +So we should send the information user defined in proto to `GetExpectedKernelType` for choosing a kernel. The problem is, how should we define and send the information for `GetExpectedKernelType` to use? ## Solution -### potential choice -1, Do nothing, let the user add the information they want to operator‘s attribute and get them inside `GetExpectedKernelType`, this can work right. But there is a little problem that users may define many kinds of hints for the same purpose, such as `force_cpu`, `use_cpu`, `CPU` for CPU kernel, and `use_cudnn`, `force_cudnn`, `cudnn_kernel` for use of CUDNN kernel. +### Potential choice +1. Do nothing, let the user add the information they want to operator‘s attribute and get them inside `GetExpectedKernelType`, this can work properly. But there is a little problem that users may define many kinds of hints for the same purpose, such as `force_cpu`, `use_cpu`, `cpu_kernel` to choose CPU kernel, and `use_cudnn`, `force_cudnn`, `cudnn_kernel` to choose CUDNN kernel. -2, Pre-define all the needed option and use a single attr key such as `kernel_hint` for the user, this is not so flexible if the user wants to define some more kind of hint. +2. Pre-define all the needed option and use a single attr key such as `kernel_hint` for the user, this is not so flexible if the user wants to define some more kind of hint. -### final choice -To provide enough flexibility while avoiding confusion definition, we can predefine some options, such as `force_cpu`, `use_cudnn`, `use_mkldnn` for a user to choose. +### Final choice +To provide enough flexibility while avoiding confusion definition, we can define some global constants for these attribute names, such as `force_cpu`, `use_cudnn`, `use_mkldnn` for a user to choose. + +In C++ ```cpp -const std::string kNonHint = ""; const std::string kForceCPU = "force_cpu"; const std::string kUseCUDNN = "use_cudnn"; const std::string kUseMKLDNN = "use_mkldnn"; -KernelType GetExpectedKernelTyp() { - // "kernel_hint" is a user defined attribute name - if (Attr("kernel_hint") == kForceCPU) { +KernelType GetExpectedKernelType() { + if (Attr(kForceCPU)) { return KernelType(CPUPlace, ...) } else { ... @@ -47,10 +47,11 @@ KernelType GetExpectedKernelTyp() { In Python code ```python -def xx_layer(..., kernel_hint=None): - layer_helper = ... - layer_helper .append_op( +FORCE_CPU = core.kForceCPU() + +def xx_layer(..., force_cpu=false): + layer_helper = LayerHelper(...) + layer_helper.append_op( type="xx", - # "kernel_hint" should be the same with the attr name in CPP - attr={"kernel_hint": kernel_hint or ""}) + attr={FORCE_CPU: force_cpu}) ```