提交 8867303f 编写于 作者: M mindspore-ci-bot 提交者: Gitee

!841 update the English version of architecture.md

Merge pull request !841 from mengchunyang/r0.7
# Overall Architecture # Overall Architecture
<a href="https://gitee.com/mindspore/docs/blob/r0.7/lite/docs/source_en/architecture.md" target="_blank"><img src="../_static/logo_source.png"></a> <a href="https://gitee.com/mindspore/docs/blob/r0.7/lite/docs/source_en/architecture.md" target="_blank"><img src="./_static/logo_source.png"></a>
The overall architecture of MindSpore Lite is as follows:
![architecture](./images/MindSpore-Lite-architecture.png)
- **Frontend:** generates models. You can use the model building API to build models and convert third-party models and models trained by MindSpore to MindSpore Lite models. Third-party models include TensorFlow Lite, Caffe 1.0, and ONNX models.
- **IR:** defines the tensors, operators, and graphs of MindSpore.
- **Backend:** optimizes graphs based on IR, including graph high level optimization (GHLO), graph low level optimization (GLLO), and quantization. GHLO is responsible for hardware-independent optimization, such as operator fusion and constant folding. GLLO is responsible for hardware-related optimization. Quantizer supports quantization methods after training, such as weight quantization and activation value quantization.
- **Runtime:** inference runtime of intelligent devices. Sessions are responsible for session management and provide external APIs. The thread pool and parallel primitives are responsible for managing the thread pool used for graph execution. Memory allocation is responsible for memory overcommitment of each operator during graph execution. The operator library provides the CPU and GPU operators.
- **Micro:** runtime of IoT devices, including the model generation .c file, thread pool, memory overcommitment, and operator library.
Runtime and Micro share the underlying infrastructure layers, such as the operator library, memory allocation, thread pool, and parallel primitives.
...@@ -22,9 +22,9 @@ The Benchmark tool is used to perform benchmark testing on a MindSpore Lite mode ...@@ -22,9 +22,9 @@ The Benchmark tool is used to perform benchmark testing on a MindSpore Lite mode
To use the Benchmark tool, you need to prepare the environment as follows: To use the Benchmark tool, you need to prepare the environment as follows:
- Compilation: Install compilation dependencies and perform compilation. The code of the Benchmark tool is stored in the `mindspore/lite/tools/benchmark` directory of the MindSpore source code. For details about the compilation operations, see the [Environment Requirements] (https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id2) and [Compilation Example] (https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id5) in the deployment document. - Compilation: Install compilation dependencies and perform compilation. The code of the Benchmark tool is stored in the `mindspore/lite/tools/benchmark` directory of the MindSpore source code. For details about the compilation operations, see the [Environment Requirements](https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id2) and [Compilation Example](https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id5) in the deployment document.
- Run: Obtain the `Benchmark` tool and configure environment variables. For details, see [Output Description] (https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id4) in the deployment document. - Run: Obtain the `Benchmark` tool and configure environment variables. For details, see [Output Description](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id4) in the deployment document.
## Parameter Description ## Parameter Description
...@@ -48,7 +48,7 @@ The following describes the parameters in detail. ...@@ -48,7 +48,7 @@ The following describes the parameters in detail.
| `--accuracyThreshold=<ACCURACYTHRESHOLD>` | Optional | Specifies the accuracy threshold. | Float | 0.5 | - | | `--accuracyThreshold=<ACCURACYTHRESHOLD>` | Optional | Specifies the accuracy threshold. | Float | 0.5 | - |
| `--calibDataPath=<CALIBDATAPATH>` | Optional | Specifies the file path of the benchmark data. The benchmark data, as the comparison output of the tested model, is output from the forward inference of the tested model under other deep learning frameworks using the same input. | String | Null | - | | `--calibDataPath=<CALIBDATAPATH>` | Optional | Specifies the file path of the benchmark data. The benchmark data, as the comparison output of the tested model, is output from the forward inference of the tested model under other deep learning frameworks using the same input. | String | Null | - |
| `--cpuBindMode=<CPUBINDMODE>` | Optional | Specifies the type of the CPU core bound to the model inference program. | Integer | 1 | −1: medium core<br/>1: large core<br/>0: not bound | | `--cpuBindMode=<CPUBINDMODE>` | Optional | Specifies the type of the CPU core bound to the model inference program. | Integer | 1 | −1: medium core<br/>1: large core<br/>0: not bound |
| `--device=<DEVICE>` | Optional | Specifies the type of the device on which the model inference program runs. | String | CPU | CPU, NPU, or GPU | | `--device=<DEVICE>` | Optional | Specifies the type of the device on which the model inference program runs. | String | CPU | CPU or GPU |
| `--help` | Optional | Displays the help information about the `benchmark` command. | - | - | - | | `--help` | Optional | Displays the help information about the `benchmark` command. | - | - | - |
| `--inDataPath=<INDATAPATH>` | Optional | Specifies the file path of the input data of the tested model. If this parameter is not set, a random value will be used. | String | Null | - | | `--inDataPath=<INDATAPATH>` | Optional | Specifies the file path of the input data of the tested model. If this parameter is not set, a random value will be used. | String | Null | - |
| `--inDataType=<INDATATYPE>` | Optional | Specifies the file type of the input data of the tested model. | String | Bin | Img: The input data is an image. Bin: The input data is a binary file.| | `--inDataType=<INDATATYPE>` | Optional | Specifies the file type of the input data of the tested model. | String | Bin | Img: The input data is an image. Bin: The input data is a binary file.|
...@@ -68,13 +68,13 @@ When using the Benchmark tool to perform benchmark testing on different MindSpor ...@@ -68,13 +68,13 @@ When using the Benchmark tool to perform benchmark testing on different MindSpor
The main test indicator of the performance test performed by the Benchmark tool is the duration of a single forward inference. In a performance test, you do not need to set benchmark data parameters such as `calibDataPath`. For example: The main test indicator of the performance test performed by the Benchmark tool is the duration of a single forward inference. In a performance test, you do not need to set benchmark data parameters such as `calibDataPath`. For example:
```bash ```bash
./benchmark --modelPath=./models/face_age.ms ./benchmark --modelPath=./models/test_benchmark.ms
``` ```
This command uses a random input, and other parameters use default values. After this command is executed, the following statistics are displayed. The statistics include the minimum duration, maximum duration, and average duration of a single inference after the tested model runs for the specified number of inference rounds. This command uses a random input, and other parameters use default values. After this command is executed, the following statistics are displayed. The statistics include the minimum duration, maximum duration, and average duration of a single inference after the tested model runs for the specified number of inference rounds.
``` ```
Model = face_age.ms, numThreads = 2, MinRunTime = 72.228996 ms, MaxRuntime = 73.094002 ms, AvgRunTime = 72.556000 ms Model = test_benchmark.ms, numThreads = 2, MinRunTime = 72.228996 ms, MaxRuntime = 73.094002 ms, AvgRunTime = 72.556000 ms
``` ```
### Accuracy Test ### Accuracy Test
...@@ -82,10 +82,10 @@ Model = face_age.ms, numThreads = 2, MinRunTime = 72.228996 ms, MaxRuntime = 73. ...@@ -82,10 +82,10 @@ Model = face_age.ms, numThreads = 2, MinRunTime = 72.228996 ms, MaxRuntime = 73.
The accuracy test performed by the Benchmark tool is to verify the accuracy of the MinSpore model output by setting benchmark data. In an accuracy test, in addition to the `modelPath` parameter, the `calibDataPath` parameter must be set. For example: The accuracy test performed by the Benchmark tool is to verify the accuracy of the MinSpore model output by setting benchmark data. In an accuracy test, in addition to the `modelPath` parameter, the `calibDataPath` parameter must be set. For example:
```bash ```bash
./benchmark --modelPath=./models/face_age.ms --inDataPath=./input/face_age.bin --device=NPU --accuracyThreshold=3 --calibDataPath=./output/face_age.out ./benchmark --modelPath=./models/test_benchmark.ms --inDataPath=./input/test_benchmark.bin --device=CPU --accuracyThreshold=3 --calibDataPath=./output/test_benchmark.out
``` ```
This command specifies the input data and benchmark data of the tested model, specifies that the model inference program runs on the NPU, and sets the accuracy threshold to 3%. After this command is executed, the following statistics are displayed, including the single input data of the tested model, output result and average deviation rate of the output node, and average deviation rate of all nodes. This command specifies the input data and benchmark data of the tested model, specifies that the model inference program runs on the CPU, and sets the accuracy threshold to 3%. After this command is executed, the following statistics are displayed, including the single input data of the tested model, output result and average deviation rate of the output node, and average deviation rate of all nodes.
``` ```
InData0: 139.947 182.373 153.705 138.945 108.032 164.703 111.585 227.402 245.734 97.7776 201.89 134.868 144.851 236.027 18.1142 22.218 5.15569 212.318 198.43 221.853 InData0: 139.947 182.373 153.705 138.945 108.032 164.703 111.585 227.402 245.734 97.7776 201.89 134.868 144.851 236.027 18.1142 22.218 5.15569 212.318 198.43 221.853
......
...@@ -20,7 +20,7 @@ The TimeProfiler tool can be used to analyze the time consumption of forward inf ...@@ -20,7 +20,7 @@ The TimeProfiler tool can be used to analyze the time consumption of forward inf
To use the TimeProfiler tool, you need to prepare the environment as follows: To use the TimeProfiler tool, you need to prepare the environment as follows:
- Compilation: Install compilation dependencies and perform compilation. The code of the TimeProfiler tool is stored in the `mindspore/lite/tools/time_profiler` directory of the MindSpore source code. For details about the compilation operations, see the [Environment Requirements] (https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id2) and [Compilation Example] (https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id5) in the deployment document. - Compilation: Install compilation dependencies and perform compilation. The code of the TimeProfiler tool is stored in the `mindspore/lite/tools/time_profiler` directory of the MindSpore source code. For details about the compilation operations, see the [Environment Requirements](https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id2) and [Compilation Example](https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id5) in the deployment document.
- Run: Obtain the `time_profiler` tool and configure environment variables by referring to [Output Description](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id4) in the deployment document. - Run: Obtain the `time_profiler` tool and configure environment variables by referring to [Output Description](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id4) in the deployment document.
...@@ -46,10 +46,10 @@ The following describes the parameters in detail. ...@@ -46,10 +46,10 @@ The following describes the parameters in detail.
## Example ## Example
Take the `tcpclassify.ms` model as an example and set the number of model inference cycles to 10. The command for using TimeProfiler to analyze the time consumption at the network layer is as follows: Take the `test_timeprofiler.ms` model as an example and set the number of model inference cycles to 10. The command for using TimeProfiler to analyze the time consumption at the network layer is as follows:
```bash ```bash
./timeprofiler --modelPath=./models/tcpclassify.ms --loopCount=10 ./timeprofiler --modelPath=./models/test_timeprofiler.ms --loopCount=10
``` ```
After this command is executed, the TimeProfiler tool outputs the statistics on the running time of the model at the network layer. In this example, the command output is as follows: The statistics are displayed by`opName` and `optype`. `opName` indicates the operator name, `optype` indicates the operator type, and `avg` indicates the average running time of the operator per single run, `percent` indicates the ratio of the operator running time to the total operator running time, `calledTimess` indicates the number of times that the operator is run, and `opTotalTime` indicates the total time that the operator is run for a specified number of times. Finally, `total time` and `kernel cost` show the average time consumed by a single inference operation of the model and the sum of the average time consumed by all operators in the model inference, respectively. After this command is executed, the TimeProfiler tool outputs the statistics on the running time of the model at the network layer. In this example, the command output is as follows: The statistics are displayed by`opName` and `optype`. `opName` indicates the operator name, `optype` indicates the operator type, and `avg` indicates the average running time of the operator per single run, `percent` indicates the ratio of the operator running time to the total operator running time, `calledTimess` indicates the number of times that the operator is run, and `opTotalTime` indicates the total time that the operator is run for a specified number of times. Finally, `total time` and `kernel cost` show the average time consumed by a single inference operation of the model and the sum of the average time consumed by all operators in the model inference, respectively.
......
...@@ -48,7 +48,7 @@ Benchmark工具是一款可以对MindSpore Lite模型进行基准测试的工具 ...@@ -48,7 +48,7 @@ Benchmark工具是一款可以对MindSpore Lite模型进行基准测试的工具
| `--accuracyThreshold=<ACCURACYTHRESHOLD>` | 可选 | 指定准确度阈值。 | Float | 0.5 | - | | `--accuracyThreshold=<ACCURACYTHRESHOLD>` | 可选 | 指定准确度阈值。 | Float | 0.5 | - |
| `--calibDataPath=<CALIBDATAPATH>` | 可选 | 指定标杆数据的文件路径。标杆数据作为该测试模型的对比输出,是该测试模型使用相同输入并由其它深度学习框架前向推理而来。 | String | null | - | | `--calibDataPath=<CALIBDATAPATH>` | 可选 | 指定标杆数据的文件路径。标杆数据作为该测试模型的对比输出,是该测试模型使用相同输入并由其它深度学习框架前向推理而来。 | String | null | - |
| `--cpuBindMode=<CPUBINDMODE>` | 可选 | 指定模型推理程序运行时绑定的CPU核类型。 | Integer | 1 | -1:表示中核<br/>1:表示大核<br/>0:表示不绑定 | | `--cpuBindMode=<CPUBINDMODE>` | 可选 | 指定模型推理程序运行时绑定的CPU核类型。 | Integer | 1 | -1:表示中核<br/>1:表示大核<br/>0:表示不绑定 |
| `--device=<DEVICE>` | 可选 | 指定模型推理程序运行的设备类型。 | String | CPU | CPU、NPU、GPU | | `--device=<DEVICE>` | 可选 | 指定模型推理程序运行的设备类型。 | String | CPU | CPU、GPU |
| `--help` | 可选 | 显示`benchmark`命令的帮助信息。 | - | - | - | | `--help` | 可选 | 显示`benchmark`命令的帮助信息。 | - | - | - |
| `--inDataPath=<INDATAPATH>` | 可选 | 指定测试模型输入数据的文件路径。如果未设置,则使用随机输入。 | String | null | - | | `--inDataPath=<INDATAPATH>` | 可选 | 指定测试模型输入数据的文件路径。如果未设置,则使用随机输入。 | String | null | - |
| `--inDataType=<INDATATYPE>` | 可选 | 指定测试模型输入数据的文件类型。 | String | bin | img:表示输入数据的文件类型为图片<br>bin:表示输入数据的类型为二进制文件 | | `--inDataType=<INDATATYPE>` | 可选 | 指定测试模型输入数据的文件类型。 | String | bin | img:表示输入数据的文件类型为图片<br>bin:表示输入数据的类型为二进制文件 |
...@@ -68,13 +68,13 @@ Benchmark工具是一款可以对MindSpore Lite模型进行基准测试的工具 ...@@ -68,13 +68,13 @@ Benchmark工具是一款可以对MindSpore Lite模型进行基准测试的工具
Benchmark工具进行的性能测试主要的测试指标为模型单次前向推理的耗时。在性能测试任务中,不需要设置`calibDataPath`等标杆数据参数。例如: Benchmark工具进行的性能测试主要的测试指标为模型单次前向推理的耗时。在性能测试任务中,不需要设置`calibDataPath`等标杆数据参数。例如:
```bash ```bash
./benchmark --modelPath=./models/face_age.ms ./benchmark --modelPath=./models/test_benchmark.ms
``` ```
这条命令使用随机输入,其他参数使用默认值。该命令执行后会输出如下统计信息,该信息显示了测试模型在运行指定推理轮数后所统计出的单次推理最短耗时、单次推理最长耗时和平均推理耗时。 这条命令使用随机输入,其他参数使用默认值。该命令执行后会输出如下统计信息,该信息显示了测试模型在运行指定推理轮数后所统计出的单次推理最短耗时、单次推理最长耗时和平均推理耗时。
``` ```
Model = face_age.ms, numThreads = 2, MinRunTime = 72.228996 ms, MaxRuntime = 73.094002 ms, AvgRunTime = 72.556000 ms Model = test_benchmark.ms, numThreads = 2, MinRunTime = 72.228996 ms, MaxRuntime = 73.094002 ms, AvgRunTime = 72.556000 ms
``` ```
### 精度测试 ### 精度测试
...@@ -82,10 +82,10 @@ Model = face_age.ms, numThreads = 2, MinRunTime = 72.228996 ms, MaxRuntime = 73. ...@@ -82,10 +82,10 @@ Model = face_age.ms, numThreads = 2, MinRunTime = 72.228996 ms, MaxRuntime = 73.
Benchmark工具进行的精度测试主要是通过设置标杆数据来对比验证MindSpore Lite模型输出的精确性。在精确度测试任务中,除了需要设置`modelPath`参数以外,还必须设置`calibDataPath`参数。例如: Benchmark工具进行的精度测试主要是通过设置标杆数据来对比验证MindSpore Lite模型输出的精确性。在精确度测试任务中,除了需要设置`modelPath`参数以外,还必须设置`calibDataPath`参数。例如:
```bash ```bash
./benchmark --modelPath=./models/face_age.ms --inDataPath=./input/face_age.bin --device=NPU --accuracyThreshold=3 --calibDataPath=./output/face_age.out ./benchmark --modelPath=./models/test_benchmark.ms --inDataPath=./input/test_benchmark.bin --device=CPU --accuracyThreshold=3 --calibDataPath=./output/test_benchmark.out
``` ```
这条命令指定了测试模型的输入数据、标杆数据,同时指定了模型推理程序在NPU上运行,并指定了准确度阈值为3%。该命令执行后会输出如下统计信息,该信息显示了测试模型的单条输入数据、输出节点的输出结果和平均偏差率以及所有节点的平均偏差率。 这条命令指定了测试模型的输入数据、标杆数据,同时指定了模型推理程序在CPU上运行,并指定了准确度阈值为3%。该命令执行后会输出如下统计信息,该信息显示了测试模型的单条输入数据、输出节点的输出结果和平均偏差率以及所有节点的平均偏差率。
``` ```
InData0: 139.947 182.373 153.705 138.945 108.032 164.703 111.585 227.402 245.734 97.7776 201.89 134.868 144.851 236.027 18.1142 22.218 5.15569 212.318 198.43 221.853 InData0: 139.947 182.373 153.705 138.945 108.032 164.703 111.585 227.402 245.734 97.7776 201.89 134.868 144.851 236.027 18.1142 22.218 5.15569 212.318 198.43 221.853
......
...@@ -46,10 +46,10 @@ TimeProfiler工具可以对MindSpore Lite模型网络层的前向推理进行耗 ...@@ -46,10 +46,10 @@ TimeProfiler工具可以对MindSpore Lite模型网络层的前向推理进行耗
## 使用示例 ## 使用示例
使用TimeProfiler对`tcpclassify.ms`模型的网络层进行耗时分析,并且设置模型推理循环运行次数为10,则其命令代码如下: 使用TimeProfiler对`test_timeprofiler.ms`模型的网络层进行耗时分析,并且设置模型推理循环运行次数为10,则其命令代码如下:
```bash ```bash
./timeprofiler --modelPath=./models/tcpclassify.ms --loopCount=10 ./timeprofiler --modelPath=./models/test_timeprofiler.ms --loopCount=10
``` ```
该条命令执行后,TimeProfiler工具会输出模型网络层运行耗时的相关统计信息。对于本例命令,输出的统计信息如下。其中统计信息按照`opName``optype`两种划分方式分别显示,`opName`表示算子名,`optype`表示算子类别,`avg`表示该算子的平均单次运行时间,`percent`表示该算子运行耗时占所有算子运行总耗时的比例,`calledTimess`表示该算子的运行次数,`opTotalTime`表示该算子运行指定次数的总耗时。最后,`total time``kernel cost`分别显示了该模型单次推理的平均耗时和模型推理中所有算子的平均耗时之和。 该条命令执行后,TimeProfiler工具会输出模型网络层运行耗时的相关统计信息。对于本例命令,输出的统计信息如下。其中统计信息按照`opName``optype`两种划分方式分别显示,`opName`表示算子名,`optype`表示算子类别,`avg`表示该算子的平均单次运行时间,`percent`表示该算子运行耗时占所有算子运行总耗时的比例,`calledTimess`表示该算子的运行次数,`opTotalTime`表示该算子运行指定次数的总耗时。最后,`total time``kernel cost`分别显示了该模型单次推理的平均耗时和模型推理中所有算子的平均耗时之和。
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册