In PaddlePaddle's [Design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/switch_kernel.md), one Operator may have multiple kernels. Users may have some personal preference to choose a certain type of kernel for an operator, such as `force_cpu` to choose a CPU kernel, `use_cudnn` to choose a CUDNN kernel, we need to provide a way for users to do this.
In the current design, we use KernelType to describe one kernel.
```cpp
struct KernelType {
Place place_;
DataType data_type_;
LayoutType layout_;
};
```
`place_` `data_type_` and `layout_` can be got from the input tensors of the operator, `GetActualKernelType(inputs)` use inputs to infer the proper kernel key that fit the incoming data, but users can not directly configure it.
The [design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/switch_kernel.md) also provides a virtual method `GetExpectedKernelType` that user can overload and use to choose the KernelType they want to use.
So we should send the information user defined in proto to `GetExpectedKernelType` for choosing a kernel.
The problem is, how should we define and send the information for `GetExpectedKernelType` to use?
## Solution
### Potential choice
1. Do nothing, let the user add the information they want to operator‘s attribute and get them inside `GetExpectedKernelType`, this can work properly. But there is a little problem that users may define many kinds of hints for the same purpose, such as `force_cpu`, `use_cpu`, `cpu_kernel` to choose CPU kernel, and `use_cudnn`, `force_cudnn`, `cudnn_kernel` to choose CUDNN kernel.
2. Pre-define all the needed option and use a single attr key such as `kernel_hint` for the user, this is not so flexible if the user wants to define some more kind of hint.
### Final choice
To provide enough flexibility while avoiding confusion definition, we can define some global constants for these attribute names, such as `force_cpu`, `use_cudnn`, `use_mkldnn` for a user to choose.
<spanid="problem"></span><h1>Problem<aclass="headerlink"href="#problem"title="Permalink to this headline">¶</a></h1>
<p>In PaddlePaddle’s <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/switch_kernel.md">Design</a>, one Operator may have multiple kernels. Users may have some personal preference to choose a certain type of kernel for an operator, such as <codeclass="docutils literal"><spanclass="pre">force_cpu</span></code> to choose a CPU kernel, <codeclass="docutils literal"><spanclass="pre">use_cudnn</span></code> to choose a CUDNN kernel, we need to provide a way for users to do this.</p>
<p>In the current design, we use KernelType to describe one kernel.</p>
<p><codeclass="docutils literal"><spanclass="pre">place_</span></code><codeclass="docutils literal"><spanclass="pre">data_type_</span></code> and <codeclass="docutils literal"><spanclass="pre">layout_</span></code> can be got from the input tensors of the operator, <codeclass="docutils literal"><spanclass="pre">GetActualKernelType(inputs)</span></code> use inputs to infer the proper kernel key that fit the incoming data, but users can not directly configure it.</p>
<p>The <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/switch_kernel.md">design</a> also provides a virtual method <codeclass="docutils literal"><spanclass="pre">GetExpectedKernelType</span></code> that user can overload and use to choose the KernelType they want to use.</p>
<p>So we should send the information user defined in proto to <codeclass="docutils literal"><spanclass="pre">GetExpectedKernelType</span></code> for choosing a kernel.</p>
<p>The problem is, how should we define and send the information for <codeclass="docutils literal"><spanclass="pre">GetExpectedKernelType</span></code> to use?</p>
</div>
<divclass="section"id="solution">
<spanid="solution"></span><h1>Solution<aclass="headerlink"href="#solution"title="Permalink to this headline">¶</a></h1>
<divclass="section"id="potential-choice">
<spanid="potential-choice"></span><h2>Potential choice<aclass="headerlink"href="#potential-choice"title="Permalink to this headline">¶</a></h2>
<olclass="simple">
<li>Do nothing, let the user add the information they want to operator‘s attribute and get them inside <codeclass="docutils literal"><spanclass="pre">GetExpectedKernelType</span></code>, this can work properly. But there is a little problem that users may define many kinds of hints for the same purpose, such as <codeclass="docutils literal"><spanclass="pre">force_cpu</span></code>, <codeclass="docutils literal"><spanclass="pre">use_cpu</span></code>, <codeclass="docutils literal"><spanclass="pre">cpu_kernel</span></code> to choose CPU kernel, and <codeclass="docutils literal"><spanclass="pre">use_cudnn</span></code>, <codeclass="docutils literal"><spanclass="pre">force_cudnn</span></code>, <codeclass="docutils literal"><spanclass="pre">cudnn_kernel</span></code> to choose CUDNN kernel.</li>
<li>Pre-define all the needed option and use a single attr key such as <codeclass="docutils literal"><spanclass="pre">kernel_hint</span></code> for the user, this is not so flexible if the user wants to define some more kind of hint.</li>
</ol>
</div>
<divclass="section"id="final-choice">
<spanid="final-choice"></span><h2>Final choice<aclass="headerlink"href="#final-choice"title="Permalink to this headline">¶</a></h2>
<p>To provide enough flexibility while avoiding confusion definition, we can define some global constants for these attribute names, such as <codeclass="docutils literal"><spanclass="pre">force_cpu</span></code>, <codeclass="docutils literal"><spanclass="pre">use_cudnn</span></code>, <codeclass="docutils literal"><spanclass="pre">use_mkldnn</span></code> for a user to choose.</p>
Built with <ahref="http://sphinx-doc.org/">Sphinx</a> using a <ahref="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <ahref="https://readthedocs.org">Read the Docs</a>.
In PaddlePaddle's [Design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/switch_kernel.md), one Operator may have multiple kernels. Users may have some personal preference to choose a certain type of kernel for an operator, such as `force_cpu` to choose a CPU kernel, `use_cudnn` to choose a CUDNN kernel, we need to provide a way for users to do this.
In the current design, we use KernelType to describe one kernel.
```cpp
struct KernelType {
Place place_;
DataType data_type_;
LayoutType layout_;
};
```
`place_` `data_type_` and `layout_` can be got from the input tensors of the operator, `GetActualKernelType(inputs)` use inputs to infer the proper kernel key that fit the incoming data, but users can not directly configure it.
The [design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/switch_kernel.md) also provides a virtual method `GetExpectedKernelType` that user can overload and use to choose the KernelType they want to use.
So we should send the information user defined in proto to `GetExpectedKernelType` for choosing a kernel.
The problem is, how should we define and send the information for `GetExpectedKernelType` to use?
## Solution
### Potential choice
1. Do nothing, let the user add the information they want to operator‘s attribute and get them inside `GetExpectedKernelType`, this can work properly. But there is a little problem that users may define many kinds of hints for the same purpose, such as `force_cpu`, `use_cpu`, `cpu_kernel` to choose CPU kernel, and `use_cudnn`, `force_cudnn`, `cudnn_kernel` to choose CUDNN kernel.
2. Pre-define all the needed option and use a single attr key such as `kernel_hint` for the user, this is not so flexible if the user wants to define some more kind of hint.
### Final choice
To provide enough flexibility while avoiding confusion definition, we can define some global constants for these attribute names, such as `force_cpu`, `use_cudnn`, `use_mkldnn` for a user to choose.
<p>In PaddlePaddle’s <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/switch_kernel.md">Design</a>, one Operator may have multiple kernels. Users may have some personal preference to choose a certain type of kernel for an operator, such as <codeclass="docutils literal"><spanclass="pre">force_cpu</span></code> to choose a CPU kernel, <codeclass="docutils literal"><spanclass="pre">use_cudnn</span></code> to choose a CUDNN kernel, we need to provide a way for users to do this.</p>
<p>In the current design, we use KernelType to describe one kernel.</p>
<p><codeclass="docutils literal"><spanclass="pre">place_</span></code><codeclass="docutils literal"><spanclass="pre">data_type_</span></code> and <codeclass="docutils literal"><spanclass="pre">layout_</span></code> can be got from the input tensors of the operator, <codeclass="docutils literal"><spanclass="pre">GetActualKernelType(inputs)</span></code> use inputs to infer the proper kernel key that fit the incoming data, but users can not directly configure it.</p>
<p>The <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/switch_kernel.md">design</a> also provides a virtual method <codeclass="docutils literal"><spanclass="pre">GetExpectedKernelType</span></code> that user can overload and use to choose the KernelType they want to use.</p>
<p>So we should send the information user defined in proto to <codeclass="docutils literal"><spanclass="pre">GetExpectedKernelType</span></code> for choosing a kernel.</p>
<p>The problem is, how should we define and send the information for <codeclass="docutils literal"><spanclass="pre">GetExpectedKernelType</span></code> to use?</p>
<li>Do nothing, let the user add the information they want to operator‘s attribute and get them inside <codeclass="docutils literal"><spanclass="pre">GetExpectedKernelType</span></code>, this can work properly. But there is a little problem that users may define many kinds of hints for the same purpose, such as <codeclass="docutils literal"><spanclass="pre">force_cpu</span></code>, <codeclass="docutils literal"><spanclass="pre">use_cpu</span></code>, <codeclass="docutils literal"><spanclass="pre">cpu_kernel</span></code> to choose CPU kernel, and <codeclass="docutils literal"><spanclass="pre">use_cudnn</span></code>, <codeclass="docutils literal"><spanclass="pre">force_cudnn</span></code>, <codeclass="docutils literal"><spanclass="pre">cudnn_kernel</span></code> to choose CUDNN kernel.</li>
<li>Pre-define all the needed option and use a single attr key such as <codeclass="docutils literal"><spanclass="pre">kernel_hint</span></code> for the user, this is not so flexible if the user wants to define some more kind of hint.</li>
<p>To provide enough flexibility while avoiding confusion definition, we can define some global constants for these attribute names, such as <codeclass="docutils literal"><spanclass="pre">force_cpu</span></code>, <codeclass="docutils literal"><spanclass="pre">use_cudnn</span></code>, <codeclass="docutils literal"><spanclass="pre">use_mkldnn</span></code> for a user to choose.</p>
Built with <ahref="http://sphinx-doc.org/">Sphinx</a> using a <ahref="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <ahref="https://readthedocs.org">Read the Docs</a>.
