diff --git a/lite/tutorials/source_en/build.md b/lite/tutorials/source_en/build.md new file mode 100644 index 0000000000000000000000000000000000000000..12d600b2f3e5871316621dbc39090cd54fd505f3 --- /dev/null +++ b/lite/tutorials/source_en/build.md @@ -0,0 +1,180 @@ +# Build + + + +- [compilation](#compilation) + - [Linux Environment Compilation](#linux-environment-compilation) + - [Environment Requirements](#environment-requirements) + - [Compilation Options](#compilation-options) + - [Compilation Example](#compilation-example) + - [Output Description](#output-description) + - [Description of Converter's Directory Structure](#description-of-converter-directory-structure) + - [Description of Runtime and Other tools' Directory Structure](#description-of-runtime-and-other-tools-directory-structure) + + + + + +This chapter introduces how to quickly compile MindSpore Lite, which includes the following modules: + +| Module | Support Platform | Description | +| --- | ---- | ---- | +| converter | Linux | Model Conversion Tool | +| runtime | Linux、Android | Model Inference Framework | +| benchmark | Linux、Android | Benchmarking Tool | +| time_profiler | Linux、Android | Performance Analysis Tool | + +## Linux Environment Compilation + +### Environment Requirements + +- The compilation environment supports Linux x86_64 only. Ubuntu 18.04.02 LTS is recommended. + +- Compilation dependencies of runtime、benchmark and time_profiler: + - [CMake](https://cmake.org/download/) >= 3.14.1 + - [GCC](https://gcc.gnu.org/releases.html) >= 7.3.0 + - [Android_NDK r20b](https://dl.google.com/android/repository/android-ndk-r20b-linux-x86_64.zip) + - [Git](https://git-scm.com/downloads) >= 2.28.0 + +- Compilation dependencies of converter: + - [CMake](https://cmake.org/download/) >= 3.14.1 + - [GCC](https://gcc.gnu.org/releases.html) >= 7.3.0 + - [Android_NDK r20b](https://dl.google.com/android/repository/android-ndk-r20b-linux-x86_64.zip) + - [Git](https://git-scm.com/downloads) >= 2.28.0 + - [Autoconf](http://ftp.gnu.org/gnu/autoconf/) >= 2.69 + - [Libtool](https://www.gnu.org/software/libtool/) >= 2.4.6 + - [LibreSSL](http://www.libressl.org/) >= 3.1.3 + - [Automake](https://www.gnu.org/software/automake/) >= 1.11.6 + - [Libevent](https://libevent.org) >= 2.0 + - [M4](https://www.gnu.org/software/m4/m4.html) >= 1.4.18 + - [OpenSSL](https://www.openssl.org/) >= 1.1.1 + +> - To install and use `Android_NDK`, you need to configure environment variables. The command example is `export ANDROID_NDK={$NDK_PATH}/android-ndk-r20b`. +> - In the `build.sh` script, run the `git clone` command to obtain the code in the third-party dependency library. Ensure that the network settings of Git are correct. + +### Compilation Options + +MindSpore Lite provides a compilation script `build.sh` for one-click compilation, located in the root directory of MindSpore. This script can be used to compile the code of training and inference. The following describes the compilation options of MindSpore Lite. + +| Parameter | Parameter Description | Value Range | Mandatory or Not | +| -------- | ----- | ---- | ---- | +| **-I** | **Selects an applicable architecture. This option is required when compile MindSpore Lite.** | **arm64, arm32, or x86_64** | **Yes** | +| -d | If this parameter is set, the debug version is compiled. Otherwise, the release version is compiled. | None | No | +| -i | If this parameter is set, incremental compilation is performed. Otherwise, full compilation is performed. | None | No | +| -j[n] | Sets the number of threads used during compilation. Otherwise, the number of threads is set to 8 by default. | Integer | No | +| -e | In the Arm architecture, select the backend operator and set the `gpu` parameter. The built-in GPU operator of the framework is compiled at the same time. | GPU | No | +| -h | Displays the compilation help information. | None | No | + +> When the `-I` parameter changes, such as `-I x86_64` is converted to `-I arm64`, adding `-i` for parameter compilation does not take effect. + +### Compilation Example + +First, download source code from the MindSpore code repository. + +```bash +git clone https://gitee.com/mindspore/mindspore.git +``` + +Then, run the following commands in the root directory of the source code to compile MindSpore Lite of different versions: + +- Debug version of the x86_64 architecture: + ```bash + bash build.sh -I x86_64 -d + ``` + +- Release version of the x86_64 architecture, with the number of threads set: + ```bash + bash build.sh -I x86_64 -j32 + ``` + +- Release version of the Arm 64-bit architecture in incremental compilation mode, with the number of threads set: + ```bash + bash build.sh -I arm64 -i -j32 + ``` + +- Release version of the Arm 64-bit architecture in incremental compilation mode, with the built-in GPU operator compiled: + ```bash + bash build.sh -I arm64 -e gpu + ``` + +### Output Description + +After the compilation is complete, go to the `mindspore/output` directory of the source code to view the file generated after compilation. The file is divided into two parts. +- `mindspore-lite-{version}-converter-{os}.tar.gz`：Contains model conversion tool. +- `mindspore-lite-{version}-runtime-{os}-{device}.tar.gz`：Contains model inference framework、benchmarking tool and performance analysis tool. + +> version: version of the output, consistent with that of the MindSpore. +> +> device: Currently divided into cpu (built-in CPU operator) and gpu (built-in CPU and GPU operator). +> +> os: Operating system on which the output will be deployed. + +Execute the decompression command to obtain the compiled output: + +```bash +tar -xvf mindspore-lite-{version}-converter-{os}.tar.gz +tar -xvf mindspore-lite-{version}-runtime-{os}-{device}.tar.gz +``` +#### Description of Converter's Directory Structure + +The conversion tool is only available under the `-I x86_64` compilation option, and the content includes the following parts: + +``` +| +├── mindspore-lite-{version}-converter-{os} +│ └── converter # Model conversion Ttool +│ └── third_party # Header files and libraries of third party libraries +│ ├── protobuf # Dynamic library of Protobuf + +``` + +#### Description of Runtime and Other tools' Directory Structure + +The inference framework can be obtained under `-I x86_64`, `-I arm64` and `-I arm32` compilation options, and the content includes the following parts: + +- When the compilation option is `-I x86_64`: + ``` + | + ├── mindspore-lite-{version}-runtime-x86-cpu + │ └── benchmark # Benchmarking Tool + │ └── lib # Inference framework dynamic library + │ ├── libmindspore-lite.so # Dynamic library of infernece framework in MindSpore Lite + │ └── third_party # Header files and libraries of third party libraries + │ ├── flatbuffers # Header files of FlatBuffers + │ └── include # Header files of inference framework + │ └── time_profiler # Model network layer time-consuming analysis tool + + ``` + +- When the compilation option is `-I arm64`: + ``` + | + ├── mindspore-lite-{version}-runtime-arm64-cpu + │ └── benchmark # Benchmarking Tool + │ └── lib # Inference framework dynamic library + │ ├── libmindspore-lite.so # Dynamic library of infernece framework in MindSpore Lite + │ ├── liboptimize.so # Operator performance optimization library in MindSpore Lite + │ └── third_party # Header files and libraries of third party libraries + │ ├── flatbuffers # Header files of FlatBuffers + │ └── include # Header files of inference framework + │ └── time_profiler # Model network layer time-consuming analysis tool + + ``` + +- When the compilation option is `-I arm32`: + ``` + | + ├── mindspore-lite-{version}-runtime-arm64-cpu + │ └── benchmark # Benchmarking Tool + │ └── lib # Inference framework dynamic library + │ ├── libmindspore-lite.so # Dynamic library of infernece framework in MindSpore Lite + │ └── third_party # Header files and libraries of third party libraries + │ ├── flatbuffers # Header files of FlatBuffers + │ └── include # Header files of inference framework + │ └── time_profiler # Model network layer time-consuming analysis tool + + ``` + +> 1. `liboptimize.so` only exists in the output package of runtime-arm64 and is only used on ARMv8.2 and CPUs that support fp16. +> 2. Compile ARM64 to get the inference framework output of arm64-cpu by default, if you add `-e gpu`, you will get the inference framework output of arm64-gpu, and the package name is `mindspore-lite-{version}-runtime-arm64-gpu.tar.gz`, compiling ARM32 is in the same way. +> 3. Before running the tools in the converter, benchmark or time_profiler directory, you need to configure environment variables, and configure the path where the dynamic libraries of MindSpore Lite and Protobuf are located to the path where the system searches for dynamic libraries. Take the CPU compiled under version 0.7.0-beta as an example: configure converter: `export LD_LIBRARY_PATH=./output/mindspore-lite-0.7.0-converter-ubuntu/third_party/protobuf/lib:${LD_LIBRARY_PATH}`; configure benchmark and time_profiler: `export LD_LIBRARY_PATH= ./output/mindspore-lite-0.7.0-runtime-x86-cpu/lib:${LD_LIBRARY_PATH}`. diff --git a/lite/tutorials/source_en/deploy.md b/lite/tutorials/source_en/deploy.md deleted file mode 100644 index a9ff1c3b3eeeb526c921381dd280790d3c618aad..0000000000000000000000000000000000000000 --- a/lite/tutorials/source_en/deploy.md +++ /dev/null @@ -1,127 +0,0 @@ -# Deploy - - - -- [Deployment](#deployment) - - [Environment Requirements](#environment-requirements) - - [Compilation Options](#compilation-options) - - [Output Description](#output-description) - - [Compilation Example](#compilation-example) - - - - - -This document describes how to quickly install MindSpore Lite on the Ubuntu system. - -## Environment Requirements - -- The compilation environment supports Linux x86_64 only. Ubuntu 18.04.02 LTS is recommended. - -- Compilation dependencies (basics): - - [CMake](https://cmake.org/download/) >= 3.14.1 - - [GCC](https://gcc.gnu.org/releases.html) >= 7.3.0 - - [Android_NDK r20b](https://dl.google.com/android/repository/android-ndk-r20b-linux-x86_64.zip) - - > - `Android_NDK` needs to be installed only when the Arm version is compiled. Skip this dependency when the x86_64 version is compiled. - > - To install and use `Android_NDK`, you need to configure environment variables. The command example is `export ANDROID_NDK={$NDK_PATH}/android-ndk-r20b`. - -- Compilation dependencies (additional dependencies required by the MindSpore Lite model conversion tool, which is required only for compilation of the x86_64 version) - - [Autoconf](http://ftp.gnu.org/gnu/autoconf/) >= 2.69 - - [Libtool](https://www.gnu.org/software/libtool/) >= 2.4.6 - - [LibreSSL](http://www.libressl.org/) >= 3.1.3 - - [Automake](https://www.gnu.org/software/automake/) >= 1.11.6 - - [Libevent](https://libevent.org) >= 2.0 - - [M4](https://www.gnu.org/software/m4/m4.html) >= 1.4.18 - - [OpenSSL](https://www.openssl.org/) >= 1.1.1 - - -## Compilation Options - -MindSpore Lite provides multiple compilation options. You can select different compilation options as required. - -| Parameter | Parameter Description | Value Range | Mandatory or Not | -| -------- | ----- | ---- | ---- | -| -d | If this parameter is set, the debug version is compiled. Otherwise, the release version is compiled. | - | No | -| -i | If this parameter is set, incremental compilation is performed. Otherwise, full compilation is performed. | - | No | -| -j[n] | Sets the number of threads used during compilation. Otherwise, the number of threads is set to 8 by default. | - | No | -| -I | Selects an applicable architecture. | arm64, arm32, or x86_64 | Yes | -| -e | In the Arm architecture, select the backend operator and set the `gpu` parameter. The built-in GPU operator of the framework is compiled at the same time. | GPU | No | -| -h | Displays the compilation help information. | - | No | - -> When the `-I` parameter changes, that is, the applicable architecture is changed, the `-i` parameter cannot be used for incremental compilation. - -## Output Description - -After the compilation is complete, go to the `mindspore/output` directory of the source code to view the file generated after compilation. The file is named `mindspore-lite-{version}-{function}-{OS}.tar.gz`. After decompression, the tool package named `mindspore-lite-{version}-{function}-{OS}` can be obtained. - -> version: version of the output, consistent with that of the MindSpore. -> -> function: function of the output. `convert` indicates the output of the conversion tool and `runtime` indicates the output of the inference framework. -> -> OS: OS on which the output will be deployed. - -```bash -tar -xvf mindspore-lite-{version}-{function}-{OS}.tar.gz -``` - -For the x86 architecture, you can obtain the output of the conversion tool and inference framework；But for the ARM architecture, you only get inference framework. - -Generally, the compiled output files include the following types. The architecture selection affects the types of output files. - -> For the Arm 64-bit architecture, you can obtain the output of the `arm64-cpu` inference framework. If `-e gpu` is added, you can obtain the output of the `arm64-gpu` inference framework. The compilation for arm 64-bit is the same as that for arm 32-bit. - -| Directory | Description | converter | runtime | -| --- | --- | --- | --- | -| include | Inference framework header file | No | Yes | -| lib | Inference framework dynamic library | No | Yes | -| benchmark | Benchmark test tool | No | Yes | -| time_profiler | Time consumption analysis tool at the model network layer| No | Yes | -| converter | Model conversion tool | Yes | No | No | -| third_party | Header file and library of the third-party library | Yes | Yes | - -Take the 0.7.0-beta version and CPU as an example. The contents of `third party` and `lib` vary depending on the architecture as follows: -- `mindspore-lite-0.7.0-converter-ubuntu`: include `protobuf` (Protobuf dynamic library). -- `mindspore-lite-0.7.0-runtime-x86-cpu`: include `flatbuffers` (FlatBuffers header file). -TODO: Add document content. - -> Before running the tools in the `converter`, `benchmark`, or `time_profiler` directory, you need to configure environment variables and set the paths of the dynamic libraries of MindSpore Lite and Protobuf to the paths of the system dynamic libraries. The following uses the 0.7.0-beta version as an example: `export LD_LIBRARY_PATH=./mindspore-lite-0.7.0/lib:./mindspore-lite-0.7.0/third_party/protobuf/lib:${LD_LIBRARY_PATH}`. - -## Compilation Example - -First, download source code from the MindSpore code repository. - -```bash -git clone https://gitee.com/mindspore/mindspore.git -b r0.7 -``` - -Then, run the following commands in the root directory of the source code to compile MindSpore Lite of different versions: - -- Debug version of the x86_64 architecture: - ```bash - bash build.sh -I x86_64 -d - ``` - -- Release version of the x86_64 architecture, with the number of threads set: - ```bash - bash build.sh -I x86_64 -j32 - ``` - -- Release version of the Arm 64-bit architecture in incremental compilation mode, with the number of threads set: - ```bash - bash build.sh -I arm64 -i -j32 - ``` - -- Release version of the Arm 64-bit architecture in incremental compilation mode, with the built-in GPU operator compiled: - ```bash - bash build.sh -I arm64 -e gpu - ``` - -> - In the `build.sh` script, run the `git clone` command to obtain the code in the third-party dependency library. Ensure that the network settings of Git are correct. - -Take the 0.7.0-beta version as an example. After the release version of the x86_64 architecture is compiled, go to the `mindspore/output` directory and run the following decompression command to obtain the output files `include`, `lib`, `benchmark`, `time_profiler`, `converter`, and `third_party`: - -```bash -tar -xvf mindspore-lite-0.7.0-converter-ubuntu.tar.gz -tar -xvf mindspore-lite-0.7.0-runtime-x86-cpu.tar.gz -``` diff --git a/lite/tutorials/source_en/images/lite_quick_start_app_result.jpg b/lite/tutorials/source_en/images/lite_quick_start_app_result.jpg new file mode 100644 index 0000000000000000000000000000000000000000..9287aad111992c39145c70f6a473818e31402bc7 Binary files /dev/null and b/lite/tutorials/source_en/images/lite_quick_start_app_result.jpg differ diff --git a/lite/tutorials/source_en/images/lite_quick_start_home.png b/lite/tutorials/source_en/images/lite_quick_start_home.png new file mode 100644 index 0000000000000000000000000000000000000000..c48cf581b33afbc15dbf27be495215b999e1be60 Binary files /dev/null and b/lite/tutorials/source_en/images/lite_quick_start_home.png differ diff --git a/lite/tutorials/source_en/images/lite_quick_start_project_structure.png b/lite/tutorials/source_en/images/lite_quick_start_project_structure.png new file mode 100644 index 0000000000000000000000000000000000000000..ade37a61ef97a479401240215e302011c014824c Binary files /dev/null and b/lite/tutorials/source_en/images/lite_quick_start_project_structure.png differ diff --git a/lite/tutorials/source_en/images/lite_quick_start_run_app.PNG b/lite/tutorials/source_en/images/lite_quick_start_run_app.PNG new file mode 100644 index 0000000000000000000000000000000000000000..2557b6293de5b3d7fefe7f6e58b57c03deabb55d Binary files /dev/null and b/lite/tutorials/source_en/images/lite_quick_start_run_app.PNG differ diff --git a/lite/tutorials/source_en/images/lite_quick_start_sdk.png b/lite/tutorials/source_en/images/lite_quick_start_sdk.png new file mode 100644 index 0000000000000000000000000000000000000000..1fcb8acabc9ba9d289efbe7e82ee5e2da8bfe073 Binary files /dev/null and b/lite/tutorials/source_en/images/lite_quick_start_sdk.png differ diff --git a/lite/tutorials/source_en/images/side_infer_process.png b/lite/tutorials/source_en/images/side_infer_process.png new file mode 100644 index 0000000000000000000000000000000000000000..cda8aa1fbe4aa9833fc7ff32597e6fbf7b38747f Binary files /dev/null and b/lite/tutorials/source_en/images/side_infer_process.png differ diff --git a/lite/tutorials/source_en/index.rst b/lite/tutorials/source_en/index.rst index 8d538a482f590e27536f975f1b2486e33ab64bb7..976ba9a3c6896dff60c0b32f7bc537c5f5437815 100644 --- a/lite/tutorials/source_en/index.rst +++ b/lite/tutorials/source_en/index.rst @@ -11,7 +11,7 @@ MindSpore Lite Tutorials :maxdepth: 1 :caption: Quick Start - deploy + build quick_start/quick_start .. toctree:: diff --git a/lite/tutorials/source_en/use/benchmark_tool.md b/lite/tutorials/source_en/use/benchmark_tool.md index 713ddc4a4b8ffb07f6f9c1531f0c8c9ae6cf17ae..90e113b493679cb797d76d1b8d5a79387bd4f5b0 100644 --- a/lite/tutorials/source_en/use/benchmark_tool.md +++ b/lite/tutorials/source_en/use/benchmark_tool.md @@ -16,22 +16,22 @@ ## Overview -The Benchmark tool is used to perform benchmark testing on a MindSpore Lite model and is implemented using the C++ language. It can not only perform quantitative analysis (performance) on the forward inference execution duration of a MindSpore Lite model, but also perform comparative error analysis (accuracy) based on the output of the specified model. +The Benchmark tool is used to perform benchmark testing on a MindSpore Lite model. It can not only perform quantitative analysis (performance) on the forward inference execution duration of a MindSpore Lite model, but also perform comparative error analysis (accuracy) based on the output of the specified model. ## Environment Preparation To use the Benchmark tool, you need to prepare the environment as follows: -- Compilation: Install compilation dependencies and perform compilation. The code of the Benchmark tool is stored in the `mindspore/lite/tools/benchmark` directory of the MindSpore source code. For details about the compilation operations, see the [Environment Requirements](https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id2) and [Compilation Example](https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id5) in the deployment document. +- Compilation: Install build dependencies and perform build. The code of the Benchmark tool is stored in the `mindspore/lite/tools/benchmark` directory of the MindSpore source code. For details about the build operations, see the [Environment Requirements](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#environment-requirements) and [Compilation Example](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#compilation-example) in the build document. -- Run: Obtain the `Benchmark` tool and configure environment variables. For details, see [Output Description](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id4) in the deployment document. +- Run: Obtain the `Benchmark` tool and configure environment variables. For details, see [Output Description](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#output-description) in the build document. ## Parameter Description The command used for benchmark testing based on the compiled Benchmark tool is as follows: ```bash -./benchmark --modelPath= [--accuracyThreshold=] +./benchmark [--modelPath=] [--accuracyThreshold=] [--calibDataPath=] [--cpuBindMode=] [--device=] [--help] [--inDataPath=] [--inDataType=] [--loopCount=] @@ -50,12 +50,9 @@ The following describes the parameters in detail. | `--cpuBindMode=` | Optional | Specifies the type of the CPU core bound to the model inference program. | Integer | 1 | −1: medium core
1: large core
0: not bound | | `--device=` | Optional | Specifies the type of the device on which the model inference program runs. | String | CPU | CPU or GPU | | `--help` | Optional | Displays the help information about the `benchmark` command. | - | - | - | -| `--inDataPath=` | Optional | Specifies the file path of the input data of the tested model. If this parameter is not set, a random value will be used. | String | Null | - | -| `--inDataType=` | Optional | Specifies the file type of the input data of the tested model. | String | Bin | Img: The input data is an image. Bin: The input data is a binary file.| +| `--inDataPath=` | Optional | Specifies the file path of the input data of the tested model. If this parameter is not set, a random value will be used. | String | Null | - | | `--loopCount=` | Optional | Specifies the number of forward inference times of the tested model when the Benchmark tool is used for the benchmark testing. The value is a positive integer. | Integer | 10 | - | | `--numThreads=` | Optional | Specifies the number of threads for running the model inference program. | Integer | 2 | - | -| `--omModelPath=` | Optional | Specifies the file path of the OM model. This parameter is optional only when the `device` type is NPU. | String | Null | - | -| `--resizeDims=` | Optional | Specifies the size to be adjusted for the input data of the tested model. | String | Null | - | | `--warmUpLoopCount=` | Optional | Specifies the number of preheating inference times of the tested model before multiple rounds of the benchmark test are executed. | Integer | 3 | - | | `--fp16Priority=` | Optional | Specifies whether the float16 operator is preferred. | Bool | false | true, false | diff --git a/lite/tutorials/source_en/use/converter_tool.md b/lite/tutorials/source_en/use/converter_tool.md index 8b37e5512a4cdd2a841ce244fbf6876de696deb6..13404a7eebc28e9e4a8174abeca5619a8347841f 100644 --- a/lite/tutorials/source_en/use/converter_tool.md +++ b/lite/tutorials/source_en/use/converter_tool.md @@ -4,10 +4,10 @@ - [Model Conversion Tool](#model-conversion-tool) - [Overview](#overview) - - [Environment Preparation](#environment-preparation) - - [Parameter Description](#parameter-description) - - [Model Visualization](#model-visualization) - - [Example](#example) + - [Linux Environment Instructions](#linux-environment-instructions) + - [Environment Preparation](#environment-preparation) + - [Parameter Description](#parameter-description) + - [Example](#example) @@ -15,26 +15,28 @@ ## Overview -MindSpore Lite provides a tool for offline model conversion. It supports conversion of multiple types of models and visualization of converted models. The converted models can be used for inference. The command line parameters contain multiple personalized options, providing a convenient conversion method for users. +MindSpore Lite provides a tool for offline model conversion. It supports conversion of multiple types of models. The converted models can be used for inference. The command line parameters contain multiple personalized options, providing a convenient conversion method for users. Currently, the following input formats are supported: MindSpore, TensorFlow Lite, Caffe, and ONNX. -## Environment Preparation +## Linux Environment Instructions + +### Environment Preparation To use the MindSpore Lite model conversion tool, you need to prepare the environment as follows: -- Compilation: Install basic and additional compilation dependencies and perform compilation. The compilation version is x86_64. The code of the model conversion tool is stored in the `mindspore/lite/tools/converter` directory of the MindSpore source code. For details about the compilation operations, see the [Environment Requirements] (https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id2) and [Compilation Example] (https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id5) in the deployment document. +- Compilation: Install basic and additional build dependencies and perform build. The build version is x86_64. The code of the model conversion tool is stored in the `mindspore/lite/tools/converter` directory of the MindSpore source code. For details about the build operations, see the [Environment Requirements](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#environment-requirements) and [Compilation Example](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#compilation-example) in the build document. -- Run: Obtain the `converter` tool and configure environment variables by referring to [Output Description](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id4) in the deployment document. +- Run: Obtain the `converter` tool and configure environment variables by referring to [Output Description](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#output-description) in the build document. -## Parameter Description +### Parameter Description You can use `./converter_lite ` to complete the conversion. In addition, you can set multiple parameters as required. You can enter `./converter_lite --help` to obtain help information in real time. The following describes the parameters in detail. - + | Parameter | Mandatory or Not | Parameter Description | Value Range | Default Value | | -------- | ------- | ----- | --- | ---- | | `--help` | No | Prints all help information. | - | - | @@ -42,20 +44,19 @@ The following describes the parameters in detail. | `--modelFile=` | Yes | Path of the input model. | - | - | | `--outputFile=` | Yes | Path of the output model. (If the path does not exist, a directory will be automatically created.) The suffix `.ms` can be automatically generated. | - | - | | `--weightFile=` | Yes (for Caffe models only) | Path of the weight file of the input model. | - | - | -| `--quantType=` | No | Sets the training type of the model. | PostTraining: quantization after training
AwareTraining: perceptual quantization | - | +| `--quantType=` | No | Sets the quant type of the model. | PostTraining: quantization after training
AwareTraining: perceptual quantization | - | +|`--inputInferenceType=` | No(supported by aware quant models only) | Sets the input data type of the converted model. If the type is different from the origin model, the convert tool will insert data type convert op before the model to make sure the input data type is same as the input of origin model. | FLOAT or INT8 | FLOAT | +|`--inferenceType= `| No(supported by aware quant models only) | Sets the output data type of the converted model. If the type is different from the origin model, the convert tool will insert data type convert op before the model to make sure the output data type is same as the input of origin model. | FLOAT or INT8 | FLOAT | +|`--stdDev=`| No(supported by aware quant models only) | Sets the standard deviation of the input data. | （0，+∞） | 128 | +|`--mean=`| No(supported by aware quant models only) | Sets the mean value of the input data. | [-128, 127] | -0.5 | > - The parameter name and parameter value are separated by an equal sign (=) and no space is allowed between them. > - The Caffe model is divided into two files: model structure `*.prototxt`, corresponding to the `--modelFile` parameter; model weight `*.caffemodel`, corresponding to the `--weightFile` parameter -## Model Visualization - -The model visualization tool provides a method for checking the model conversion result. You can run the JSON command to generate a `*.json` file and compare it with the original model to determine the conversion effect. - -TODO: This function is under development now. -## Example +### Example -First, in the root directory of the source code, run the following command to perform compilation. For details, see `deploy.md`. +First, in the root directory of the source code, run the following command to perform compilation. For details, see `build.md`. ```bash bash build.sh -I x86_64 ``` @@ -94,15 +95,17 @@ The following describes how to use the conversion command by using several commo ./converter_lite --fmk=ONNX --modelFile=model.onnx --outputFile=model ``` - - TensorFlow Lite perceptual quantization model `model_quant.tflite` + - TensorFlow Lite aware quantization model `model_quant.tflite` ```bash ./converter_lite --fmk=TFLITE --modelFile=model.tflite --outputFile=model --quantType=AwareTraining ``` + + - TensorFlow Lite aware quantization model `model_quant.tflite` set the input and output data type to be int8 + ```bash + ./converter_lite --fmk=TFLITE --modelFile=model.tflite --outputFile=model --quantType=AwareTraining --inputInferenceType=INT8 --inferenceType=INT8 + ``` In the preceding scenarios, the following information is displayed, indicating that the conversion is successful. In addition, the target file `model.ms` is obtained. ``` INFO [converter/converter.cc:190] Runconverter] CONVERTER RESULT: SUCCESS! ``` - - -You can use the model visualization tool to visually check the converted MindSpore Lite model. This function is under development. \ No newline at end of file diff --git a/lite/tutorials/source_en/use/runtime.md b/lite/tutorials/source_en/use/runtime.md index 636c7cfdf000595e812ae9a234263939f2a55ad0..b12f6c23ca8183bad58878b45718b71e27008506 100644 --- a/lite/tutorials/source_en/use/runtime.md +++ b/lite/tutorials/source_en/use/runtime.md @@ -1,3 +1,400 @@ -# Runtime +# Runtime User Guide + + + +- [Runtime User Guide](#runtime-user-guide) + - [Overview](#overview) + - [Reading Models](#reading-models) + - [Session Creation](#session-creation) + - [Creating Contexts](#creating-contexts) + - [Creating Sessions](#creating-sessions-1) + - [Example](#example) + - [Graph Compilation](#graph-compilation) + - [Variable Dimension](#variable-dimension) + - [Example](#example-1) + - [Compiling Graphs](#compiling-graphs) + - [Example](#example-2) + - [Data Input](#data-input) + - [Obtaining Input Tensors](#obtaining-input-tensors) + - [Copying Data](#copying-data) + - [Example](#example-3) + - [Graph Execution](#graph-execution) + - [Executing Sessions](#executing-sessions) + - [Core Binding](#core-binding) + - [Callback Running](#callback-running) + - [Example](#example-4) + - [Obtaining Outputs](#obtaining-outputs) + - [Obtaining Output Tensors](#obtaining-output-tensors) + - [Example](#example-5) + - [Obtaining Version String](#obtaining-version-string) + - [Example](#example-6) + + + + +## Overview + +After model conversion using MindSpore Lite, the model inference process needs to be completed in Runtime. + +The procedure for using Runtime is shown in the following figure: + +![img](../images/side_infer_process.png) + +Its components and their functions are described as follows: +- `Model`: model used by MindSpore Lite, which instantiates the list of operator prototypes through image composition or direct network loading. +- `Lite Session`: provides the graph compilation function and calls the graph executor for inference. +- `Scheduler`: operator heterogeneous scheduler. It can select a proper kernel for each operator based on the heterogeneous scheduling policy, construct a kernel list, and split a graph into subgraphs. +- `Executor`: graph executor, which executes the kernel list to dynamically allocate and release tensors. +- `Operator`: operator prototype, including operator attributes and methods for inferring the shape, data type, and format. +- `Kernel`: operator, which provides specific operator implementation and the operator forwarding function. +- `Tensor`: tensor used by MindSpore Lite, which provides functions and APIs for tensor memory operations. + +## Reading Models + +In MindSpore Lite, a model file is an `.ms` file converted using the model conversion tool. During model inference, the model needs to be loaded from the file system and parsed. Related operations are mainly implemented in the Model component. The Model component holds model data such as weight data and operator attributes. + +A model is created based on memory data using the static `Import` method of the Model class. The `Model` instance returned by the function is a pointer, which is created by using `new`. If the pointer is not required, you need to release it by using `delete`. + +## Session Creation + +When MindSpore Lite is used for inference, sessions are the main entrance of inference. You can compile and execute graphs through sessions. + +### Creating Contexts + +Contexts save some basic configuration parameters required by sessions to guide graph compilation and execution. The definition of context is as follows: + +MindSpore Lite supports heterogeneous inference. The preferred backend for inference is specified by `device_ctx_` in `Context` and is CPU by default. During graph compilation, operator selection and scheduling are performed based on the preferred backend. + +MindSpore Lite has a built-in thread pool shared by processes. During inference, `thread_num_` is used to specify the maximum number of threads in the thread pool. The default maximum number is 2. It is recommended that the maximum number be no more than 4. Otherwise, the performance may be affected. + +MindSpore Lite supports dynamic memory allocation and release. If `allocator` is not specified, a default `allocator` is generated during inference. You can also use the `Context` method to allow multiple `Context` to share the memory allocator. + +### Creating Sessions + +Use the `Context` created in the previous step to call the static `CreateSession` method of LiteSession to create `LiteSession`. The `LiteSession` instance returned by the function is a pointer, which is created by using `new`. If the pointer is not required, you need to release it by using `delete`. + +### Example + +The following sample code demonstrates how to create a `Context` and how to allow two `LiteSession` to share a memory pool. + +```cpp +auto context = new (std::nothrow) lite::Context; +if (context == nullptr) { + MS_LOG(ERROR) << "New context failed while running %s", modelName.c_str(); + return RET_ERROR; +} +// The preferred backend is GPU, which means, if there is a GPU operator, it will run on the GPU first, otherwise it will run on the CPU. +context->device_ctx_.type = lite::DT_GPU; +// The medium core takes priority in thread and core binding methods. This parameter will work in the BindThread interface. For specific binding effect, see the "Run Graph" section. +context->cpu_bind_mode_ = MID_CPU; +// Configure the number of worker threads in the thread pool to 2, including the main thread. +context->thread_num_ = 2; +// Allocators can be shared across multiple Contexts. +auto *context2 = new Context(context->thread_num_, context->allocator, context->device_ctx_); +context2->cpu_bind_mode_ = context->cpu_bind_mode_; +// Use Context to create Session. +auto session1 = session::LiteSession::CreateSession(context); +// After the LiteSession is created, the Context can be released. +delete (context); +if (session1 == nullptr) { + MS_LOG(ERROR) << "CreateSession failed while running %s", modelName.c_str(); + return RET_ERROR; +} +// session1 and session2 can share one memory pool. +auto session2 = session::LiteSession::CreateSession(context2); +delete (context2); +if (session == nullptr) { + MS_LOG(ERROR) << "CreateSession failed while running %s", modelName.c_str(); + return RET_ERROR; +} +``` + +## Graph Compilation + +### Variable Dimension + +When using MindSpore Lite for inference, after the session creation and graph compilation have been completed, if you need to resize the input shape, you can reset the shape of the input tensor, and then call the session's Resize() interface. + +### Example + +The following code demonstrates how to resize the input of MindSpore Lite: + +```cpp +// Assume we have created a LiteSession instance named session. +auto inputs = session->GetInputs(); +std::vector resize_shape = {1, 128, 128, 3}; +// Assume the model has only one input,resize input shape to [1, 128, 128, 3] +inputs[0]->set_shape(resize_shape); +session->Resize(inputs); +``` + +### Compiling Graphs + +Before graph execution, call the `CompileGraph` API of the `LiteSession` to compile graphs and further parse the Model instance loaded from the file, mainly for subgraph split and operator selection and scheduling. This process takes a long time. Therefore, it is recommended that `ListSession` achieve multiple executions with one creation and one compilation. + +### Example + +The following code demonstrates how to compile graph of MindSpore Lite: + +```cpp +// Assume we have created a LiteSession instance named session and a Model instance named model before. +// The methods of creating model and session can refer to "Import Model" and "Create Session" two sections. +auto ret = session->CompileGraph(model); +if (ret != RET_OK) { + std::cerr << "CompileGraph failed" << std::endl; + // session and model need to be released by users manually. + delete (session); + delete (model); + return ret; +} +``` + +## Data Input + +### Obtaining Input Tensors + +Before graph execution, you need to copy the input data to model input tensors. + +MindSpore Lite provides the following methods to obtain model input tensors. + +1. Use the `GetInputsByName` method to obtain vectors of the model input tensors that are connected to the model input node based on the node name. +2. Use the `GetInputs` method to directly obtain the vectors of all model input tensors. + +### Copying Data + +After model input tensors are obtained, you need to enter data into the tensors. Use the `Size` method of `MSTensor` to obtain the size of the data to be entered into tensors, use the `data_type` method to obtain the data type of tensors, and use the `MutableData` method of `MSTensor` to obtain the writable pointer. + +### Example + +The following sample code shows how to obtain the entire graph input `MSTensor` from `LiteSession` and enter the model input data to `MSTensor`. + +```cpp +// Assume we have created a LiteSession instance named session. +auto inputs = session->GetInputs(); +// Assume that the model has only one input tensor. +auto in_tensor = inputs.front(); +if (in_tensor == nullptr) { + std::cerr << "Input tensor is nullptr" << std::endl; + return -1; +} +// It is omitted that users have read the model input file and generated a section of memory buffer: input_buf, as well as the byte size of input_buf: data_size. +if (in_tensor->Size() != data_size) { + std::cerr << "Input data size is not suit for model input" << std::endl; + return -1; +} +auto *in_data = in_tensor->MutableData(); +if (in_data == nullptr) { + std::cerr << "Data of in_tensor is nullptr" << std::endl; + return -1; +} +memcpy(in_data, input_buf, data_size); +// Users need to free input_buf. +// The elements in the inputs are managed by MindSpore Lite so that users do not need to free inputs. +``` + +Note: +- The data layout in the model input tensors of MindSpore Lite must be NHWC. +- The model input `input_buf` is read from disks. After it is copied to model input tensors, you need to release `input_buf`. +- Vectors returned by using the `GetInputs` and `GetInputsByName` methods do not need to be released by users. + +## Graph Execution + +### Executing Sessions + +After a MindSpore Lite session performs graph compilation, you can use `RunGraph` of `LiteSession` for model inference. + +### Core Binding + +The built-in thread pool of MindSpore Lite supports core binding and unbinding. By calling the `BindThread` API, you can bind working threads in the thread pool to specified CPU cores for performance analysis. The core binding operation is related to the context specified when `LiteSession` is created. The core binding operation sets the affinity between a thread and CPU based on the core binding policy in the context. + +Note that core binding is an affinity operation, which is affected by system scheduling. Therefore, successful binding to the specified CPU core cannot be ensured. After executing the code of core binding, you need to perform the unbinding operation. The following is an example: + +```cpp +// Assume we have created a LiteSession instance named session. +session->BindThread(true); +auto ret = session->RunGraph(); +if (ret != mindspore::lite::RET_OK) { + std::cerr << "RunGraph failed" << std::endl; + delete session; + return -1; +} +session->BindThread(false); +``` + +> Core binding parameters can be used to bind big cores first or middle cores first. +> The rule for determining big core or middle core is based on the CPU core frequency instead of CPU architecture. For the CPU architecture where big, middle, and little cores are not distinguished, this rule can be used. +> Big core first indicates that threads in the thread pool are bound to cores according to core frequency. The first thread is bound to the core with the highest frequency, and the second thread is bound to the core with the second highest frequency. This rule also applies to other threads. +> Middle cores are defined based on experience. By default, middle cores are cores with the third and fourth highest frequency. Middle core first indicates that threads are bound to middle cores preferentially. When there are no available middle cores, threads are bound to little cores. + +### Callback Running + +MindSpore Lite can transfer two `KernelCallBack` function pointers to call back the inference model when calling `RunGraph`. Compared with common graph execution, callback running can obtain extra information during the running process to help developers analyze performance and fix bugs. The extra information includes: +- Name of the running node +- Input and output tensors before inference of the current node +- Input and output tensors after inference of the current node + +### Example + +The following sample code demonstrates how to use `LiteSession` to compile a graph, defines two callback functions as the before-callback pointer and after-callback pointer, transfers them to the `RunGraph` API for callback inference, and demonstrates the scenario of multiple graph executions with one graph compilation. + +```cpp +// Assume we have created a LiteSession instance named session and a Model instance named model before. +// The methods of creating model and session can refer to "Import Model" and "Create Session" two sections. +auto ret = session->CompileGraph(model); +if (ret != RET_OK) { + std::cerr << "CompileGraph failed" << std::endl; + // session and model need to be released by users manually. + delete (session); + delete (model); + return ret; +} +// Copy input data into the input tensor. Users can refer to the "Input Data" section. We uses random data here. +auto inputs = session->GetInputs(); +for (auto in_tensor : inputs) { + in_tensor = inputs.front(); + if (in_tensor == nullptr) { + std::cerr << "Input tensor is nullptr" << std::endl; + return -1; + } + // When calling the MutableData method, if the data in MSTensor is not allocated, it will be malloced. After allocation, the data in MSTensor can be considered as random data. + (void) in_tensor->MutableData(); +} +// Definition of callback function before forwarding operator. +auto before_call_back_ = [&](const std::vector &before_inputs, + const std::vector &before_outputs, + const session::CallBackParam &call_param) { + std::cout << "Before forwarding " << call_param.name_callback_param << std::endl; + return true; +}; +// Definition of callback function after forwarding operator. +auto after_call_back_ = [&](const std::vector &after_inputs, + const std::vector &after_outputs, + const session::CallBackParam &call_param) { + std::cout << "After forwarding " << call_param.name_callback_param << std::endl; + return true; +}; +// Call the callback function when performing the model inference process. +ret = session_->RunGraph(before_call_back_, after_call_back_); +if (ret != RET_OK) { + MS_LOG(ERROR) << "Run graph failed."; + return RET_ERROR; +} +// CompileGraph would cost much time, a better solution is calling CompileGraph only once and RunGraph much more times. +for (size_t i = 0; i < 10; i++) { + auto ret = session_->RunGraph(); + if (ret != RET_OK) { + MS_LOG(ERROR) << "Run graph failed."; + return RET_ERROR; + } +} +// session and model needs to be released by users manually. +delete (session); +delete (model); +``` + +## Obtaining Outputs + +### Obtaining Output Tensors + +After performing inference, MindSpore Lite can obtain the model inference result. + +MindSpore Lite provides the following methods to obtain the model output `MSTensor`. +1. Use the `GetOutputsByName` method to obtain vectors of the model output `MSTensor` that is connected to the model output node based on the node name. +2. Use the `GetOutputMapByNode` method to directly obtain the mapping between the names of all model output nodes and the model output `MSTensor` connected to the nodes. +3. Use the `GetOutputByTensorName` method to obtain the model output `MSTensor` based on the tensor name. +4. Use the `GetOutputMapByTensor` method to directly obtain the mapping between the names of all model output tensors and the model output `MSTensor`. + +After model output tensors are obtained, you need to enter data into the tensors. Use the `Size` method of `MSTensor` to obtain the size of the data to be entered into tensors, use the `data_type` method to obtain the data type of `MSTensor`, and use the `MutableData` method of `MSTensor` to obtain the writable pointer. + +### Example + +The following sample code shows how to obtain the output `MSTensor` from `LiteSession` using the `GetOutputMapByNode` method and print the first ten data or all data records of each output `MSTensor`. + +```cpp +// Assume we have created a LiteSession instance named session before. +auto output_map = session->GetOutputMapByNode(); +// Assume that the model has only one output node. +auto out_node_iter = output_map.begin(); +std::string name = out_node_iter->first; +// Assume that the unique output node has only one output tensor. +auto out_tensor = out_node_iter->second.front(); +if (out_tensor == nullptr) { + std::cerr << "Output tensor is nullptr" << std::endl; + return -1; +} +// Assume that the data format of output data is float 32. +if (out_tensor->data_type() != mindspore::TypeId::kNumberTypeFloat32) { + std::cerr << "Output of lenet should in float32" << std::endl; + return -1; +} +auto *out_data = reinterpret_cast(out_tensor->MutableData()); +if (out_data == nullptr) { + std::cerr << "Data of out_tensor is nullptr" << std::endl; + return -1; +} +// Print the first 10 float data or all output data of the output tensor. +std::cout << "Output data: "; +for (size_t i = 0; i < 10 && i < out_tensor->ElementsNum(); i++) { + std::cout << " " << out_data[i]; +} +std::cout << std::endl; +// The elements in outputs do not need to be free by users, because outputs are managed by the MindSpore Lite. +``` + +Note that the vectors or map returned by the `GetOutputsByNodeName`, `GetOutputMapByNode`, `GetOutputByTensorName` and `GetOutputMapByTensor` methods do not need to be released by users. + +The following sample code shows how to obtain the output `MSTensor` from `LiteSession` using the `GetOutputsByNodeName` method. + +```cpp +// Assume we have created a LiteSession instance named session before. +// Assume that model has a output node named output_node_name_0. +auto output_vec = session->GetOutputsByNodeName("output_node_name_0"); +// Assume that output node named output_node_name_0 has only one output tensor. +auto out_tensor = output_vec.front(); +if (out_tensor == nullptr) { + std::cerr << "Output tensor is nullptr" << std::endl; + return -1; +} +``` + +The following sample code shows how to obtain the output `MSTensor` from `LiteSession` using the `GetOutputMapByTensor` method. + +```cpp +// Assume we have created a LiteSession instance named session before. +auto output_map = session->GetOutputMapByTensor(); +// Assume that output node named output_node_name_0 has only one output tensor. +auto out_tensor = output_vec.front(); +if (out_tensor == nullptr) { + std::cerr << "Output tensor is nullptr" << std::endl; + return -1; +} +``` + +The following sample code shows how to obtain the output `MSTensor` from `LiteSession` using the `GetOutputByTensorName` method. + +```cpp +// We can use GetOutputTensorNames method to get all name of output tensor of model which is in order. +auto tensor_names = this->GetOutputTensorNames(); +// Assume we have created a LiteSession instance named session before. +// Use output tensor name returned by GetOutputTensorNames as key +for (auto tensor_name : tensor_names) { + auto out_tensor = this->GetOutputByTensorName(tensor_name); + if (out_tensor == nullptr) { + std::cerr << "Output tensor is nullptr" << std::endl; + return -1; + } +} +``` + +## Obtaining Version String + +### Example + +The following sample code shows how to obtain version string using `Version` method. + +```cpp +#include "include/version.h" +std::string version = mindspore::lite::Version(); +``` diff --git a/lite/tutorials/source_en/use/timeprofiler_tool.md b/lite/tutorials/source_en/use/timeprofiler_tool.md index 40d9548419f60a9842a9e660333f035b90b2d7f3..6388a27abc86fb1f1ae9ef1143c41e685bdb5f98 100644 --- a/lite/tutorials/source_en/use/timeprofiler_tool.md +++ b/lite/tutorials/source_en/use/timeprofiler_tool.md @@ -20,9 +20,9 @@ The TimeProfiler tool can be used to analyze the time consumption of forward inf To use the TimeProfiler tool, you need to prepare the environment as follows: -- Compilation: Install compilation dependencies and perform compilation. The code of the TimeProfiler tool is stored in the `mindspore/lite/tools/time_profiler` directory of the MindSpore source code. For details about the compilation operations, see the [Environment Requirements](https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id2) and [Compilation Example](https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id5) in the deployment document. +- Compilation: Install build dependencies and perform build. The code of the TimeProfiler tool is stored in the `mindspore/lite/tools/time_profiler` directory of the MindSpore source code. For details about the build operations, see the [Environment Requirements](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#environment-requirements) and [Compilation Example](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#compilation-example) in the build document. -- Run: Obtain the `time_profiler` tool and configure environment variables by referring to [Output Description](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id4) in the deployment document. +- Run: Obtain the `time_profiler` tool and configure environment variables by referring to [Output Description](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#output-description) in the build document. ## Parameter Description diff --git a/lite/tutorials/source_zh_cn/build.md b/lite/tutorials/source_zh_cn/build.md new file mode 100644 index 0000000000000000000000000000000000000000..467c7ad4db95d9ef55126fb8d7807df1cfe5a2f5 --- /dev/null +++ b/lite/tutorials/source_zh_cn/build.md @@ -0,0 +1,182 @@ +# 编译 + + + +- [编译](#编译) + - [Linux环境编译](#linux环境编译) + - [环境要求](#环境要求) + - [编译选项](#编译选项) + - [编译示例](#编译示例) + - [编译输出](#编译输出) + - [模型转换工具converter目录结构说明](#模型转换工具converter目录结构说明) + - [模型推理框架runtime及其他工具目录结构说明](#模型推理框架runtime及其他工具目录结构说明) + + + + + + +本章节介绍如何快速编译出MindSpore Lite，其包含的模块如下： + +| 模块 | 支持平台 | 说明 | +| --- | ---- | ---- | +| converter | Linux | 模型转换工具 | +| runtime | Linux、Android | 模型推理框架 | +| benchmark | Linux、Android | 基准测试工具 | +| time_profiler | Linux、Android | 性能分析工具 | + +## Linux环境编译 + +### 环境要求 + +- 系统环境：Linux x86_64，推荐使用Ubuntu 18.04.02LTS + +- runtime、benchmark、time_profiler编译依赖 + - [CMake](https://cmake.org/download/) >= 3.14.1 + - [GCC](https://gcc.gnu.org/releases.html) >= 7.3.0 + - [Android_NDK](https://dl.google.com/android/repository/android-ndk-r20b-linux-x86_64.zip) >= r20 + - [Git](https://git-scm.com/downloads) >= 2.28.0 + +- converter编译依赖 + - [CMake](https://cmake.org/download/) >= 3.14.1 + - [GCC](https://gcc.gnu.org/releases.html) >= 7.3.0 + - [Android_NDK](https://dl.google.com/android/repository/android-ndk-r20b-linux-x86_64.zip) >= r20 + - [Git](https://git-scm.com/downloads) >= 2.28.0 + - [Autoconf](http://ftp.gnu.org/gnu/autoconf/) >= 2.69 + - [Libtool](https://www.gnu.org/software/libtool/) >= 2.4.6 + - [LibreSSL](http://www.libressl.org/) >= 3.1.3 + - [Automake](https://www.gnu.org/software/automake/) >= 1.11.6 + - [Libevent](https://libevent.org) >= 2.0 + - [M4](https://www.gnu.org/software/m4/m4.html) >= 1.4.18 + - [OpenSSL](https://www.openssl.org/) >= 1.1.1 + +> - 当安装完依赖项Android_NDK后，需配置环境变量:`export ANDROID_NDK={$NDK_PATH}/android-ndk-r20b`。 +> - 编译脚本中会执行`git clone`获取第三方依赖库的代码，请提前确保git的网络设置正确可用。 + +### 编译选项 + +MindSpore Lite提供编译脚本`build.sh`用于一键式编译，位于MindSpore根目录下，该脚本可用于MindSpore训练及推理的编译。下面对MindSpore Lite的编译选项进行说明。 + +| 选项 | 参数说明 | 取值范围 | 是否必选 | +| -------- | ----- | ---- | ---- | +| **-I** | **选择适用架构，编译MindSpore Lite此选项必选** | **arm64、arm32、x86_64** | **是** | +| -d | 设置该参数，则编译Debug版本，否则编译Release版本 | 无 | 否 | +| -i | 设置该参数，则进行增量编译，否则进行全量编译 | 无 | 否 | +| -j[n] | 设定编译时所用的线程数，否则默认设定为8线程 | Integer | 否 | +| -e | 选择除CPU之外的其他内置算子类型，仅在ARM架构下适用，当前仅支持GPU | GPU | 否 | +| -h | 显示编译帮助信息 | 无 | 否 | + +> 在`-I`参数变动时，如`-I x86_64`变为`-I arm64`，添加`-i`参数进行增量编译不生效。 + +### 编译示例 + +首先，在进行编译之前，需从MindSpore代码仓下载源码。 + +```bash +git clone https://gitee.com/mindspore/mindspore.git +``` + +然后，在源码根目录下执行如下命令，可编译不同版本的MindSpore Lite。 + +- 编译x86_64架构Debug版本。 + ```bash + bash build.sh -I x86_64 -d + ``` + +- 编译x86_64架构Release版本，同时设定线程数。 + ```bash + bash build.sh -I x86_64 -j32 + ``` + +- 增量编译ARM64架构Release版本，同时设定线程数。 + ```bash + bash build.sh -I arm64 -i -j32 + ``` + +- 编译ARM64架构Release版本，同时编译内置的GPU算子。 + ```bash + bash build.sh -I arm64 -e gpu + ``` + +### 编译输出 + +编译完成后，进入`mindspore/output/`目录，可查看编译后生成的文件。文件分为两部分： +- `mindspore-lite-{version}-converter-{os}.tar.gz`：包含模型转换工具converter。 +- `mindspore-lite-{version}-runtime-{os}-{device}.tar.gz`：包含模型推理框架runtime、基准测试工具benchmark和性能分析工具time_profiler。 + +> version：输出件版本号，与所编译的分支代码对应的版本一致。 +> +> device：当前分为cpu（内置CPU算子）和gpu（内置CPU和GPU算子）。 +> +> os：输出件应部署的操作系统。 + +执行解压缩命令，获取编译后的输出件： + +```bash +tar -xvf mindspore-lite-{version}-converter-{os}.tar.gz +tar -xvf mindspore-lite-{version}-runtime-{os}-{device}.tar.gz +``` + +#### 模型转换工具converter目录结构说明 + +转换工具仅在`-I x86_64`编译选项下获得，内容包括以下几部分： + +``` +| +├── mindspore-lite-{version}-converter-{os} +│ └── converter # 模型转换工具 +│ └── third_party # 第三方库头文件和库 +│ ├── protobuf # Protobuf的动态库 + +``` + +#### 模型推理框架runtime及其他工具目录结构说明 + +推理框架可在`-I x86_64`、`-I arm64`和`-I arm32`编译选项下获得，内容包括以下几部分： + +- 当编译选项为`-I x86_64`时： + ``` + | + ├── mindspore-lite-{version}-runtime-x86-cpu + │ └── benchmark # 基准测试工具 + │ └── lib # 推理框架动态库 + │ ├── libmindspore-lite.so # MindSpore Lite推理框架的动态库 + │ └── third_party # 第三方库头文件和库 + │ ├── flatbuffers # FlatBuffers头文件 + │ └── include # 推理框架头文件 + │ └── time_profiler # 模型网络层耗时分析工具 + + ``` + +- 当编译选项为`-I arm64`时： + ``` + | + ├── mindspore-lite-{version}-runtime-arm64-cpu + │ └── benchmark # 基准测试工具 + │ └── lib # 推理框架动态库 + │ ├── libmindspore-lite.so # MindSpore Lite推理框架的动态库 + │ ├── liboptimize.so # MindSpore Lite算子性能优化库 + │ └── third_party # 第三方库头文件和库 + │ ├── flatbuffers # FlatBuffers头文件 + │ └── include # 推理框架头文件 + │ └── time_profiler # 模型网络层耗时分析工具 + + ``` + +- 当编译选项为`-I arm32`时： + ``` + | + ├── mindspore-lite-{version}-runtime-arm64-cpu + │ └── benchmark # 基准测试工具 + │ └── lib # 推理框架动态库 + │ ├── libmindspore-lite.so # MindSpore Lite推理框架的动态库 + │ └── third_party # 第三方库头文件和库 + │ ├── flatbuffers # FlatBuffers头文件 + │ └── include # 推理框架头文件 + │ └── time_profiler # 模型网络层耗时分析工具 + + ``` + +> 1. `liboptimize.so`仅在runtime-arm64的输出包中存在，仅在ARMv8.2和支持fp16特性的CPU上使用。 +> 2. 编译ARM64默认可获得arm64-cpu的推理框架输出件，若添加`-e gpu`则获得arm64-gpu的推理框架输出件，此时包名为`mindspore-lite-{version}-runtime-arm64-gpu.tar.gz`，编译ARM32同理。 +> 3. 运行converter、benchmark或time_profiler目录下的工具前，都需配置环境变量，将MindSpore Lite和Protobuf的动态库所在的路径配置到系统搜索动态库的路径中。以0.7.0-beta版本下编译CPU为例：配置converter：`export LD_LIBRARY_PATH=./output/mindspore-lite-0.7.0-converter-ubuntu/third_party/protobuf/lib:${LD_LIBRARY_PATH}`；配置benchmark和time_profiler：`export LD_LIBRARY_PATH=./output/mindspore-lite-0.7.0-runtime-x86-cpu/lib:${LD_LIBRARY_PATH}` diff --git a/lite/tutorials/source_zh_cn/deploy.md b/lite/tutorials/source_zh_cn/deploy.md deleted file mode 100644 index c2f11edac194f9fd42cdde096500590162cbacbc..0000000000000000000000000000000000000000 --- a/lite/tutorials/source_zh_cn/deploy.md +++ /dev/null @@ -1,187 +0,0 @@ -# 部署 - - - -- [部署](#部署) - - [Linux环境部署](#linux环境部署) - - [环境要求](#环境要求) - - [编译选项](#编译选项) - - [输出件说明](#输出件说明) - - [编译示例](#编译示例) - - [Windows环境部署](#windows环境部署) - - [环境要求](#环境要求-1) - - [编译选项](#编译选项-1) - - [输出件说明](#输出件说明-1) - - [编译示例](#编译示例-1) - - - - - -本文档介绍如何在Ubuntu和Windows系统上快速安装MindSpore Lite。 - -## Linux环境部署 - -### 环境要求 - -- 编译环境仅支持x86_64版本的Linux：推荐使用Ubuntu 18.04.02LTS - -- 编译依赖（基本项） - - [CMake](https://cmake.org/download/) >= 3.14.1 - - [GCC](https://gcc.gnu.org/releases.html) >= 7.3.0 - - [Android_NDK r20b](https://dl.google.com/android/repository/android-ndk-r20b-linux-x86_64.zip) - - > - 仅在编译ARM版本时需要安装`Android_NDK`，编译x86_64版本可跳过此项。 - > - 如果安装并使用`Android_NDK`，需配置环境变量，命令参考：`export ANDROID_NDK={$NDK_PATH}/android-ndk-r20b`。 - -- 编译依赖（MindSpore Lite模型转换工具所需附加项，仅编译x86_64版本时需要） - - [Autoconf](http://ftp.gnu.org/gnu/autoconf/) >= 2.69 - - [Libtool](https://www.gnu.org/software/libtool/) >= 2.4.6 - - [LibreSSL](http://www.libressl.org/) >= 3.1.3 - - [Automake](https://www.gnu.org/software/automake/) >= 1.11.6 - - [Libevent](https://libevent.org) >= 2.0 - - [M4](https://www.gnu.org/software/m4/m4.html) >= 1.4.18 - - [OpenSSL](https://www.openssl.org/) >= 1.1.1 - - -### 编译选项 - -MindSpore Lite提供多种编译方式，用户可根据需要选择不同的编译选项。 - -| 参数 | 参数说明 | 取值范围 | 是否必选 | -| -------- | ----- | ---- | ---- | -| -d | 设置该参数，则编译Debug版本，否则编译Release版本 | - | 否 | -| -i | 设置该参数，则进行增量编译，否则进行全量编译 | - | 否 | -| -j[n] | 设定编译时所用的线程数，否则默认设定为8线程 | - | 否 | -| -I | 选择适用架构 | arm64、arm32、x86_64 | 是 | -| -e | 在ARM架构下，选择后端算子，设置`gpu`参数，会同时编译框架内置的GPU算子 | gpu | 否 | -| -h | 设置该参数，显示编译帮助信息 | - | 否 | - -> 在`-I`参数变动时，即切换适用架构时，无法使用`-i`参数进行增量编译。 - -### 输出件说明 - -编译完成后，进入源码的`mindspore/output`目录，可查看编译后生成的文件，命名为`mindspore-lite-{version}-{function}-{OS}.tar.gz`。解压后，即可获得编译后的工具包，名称为`mindspore-lite-{version}-{function}-{OS}`。 - -> version：输出件版本，与所编译的MindSpore版本一致。 -> -> function：输出件功能，`convert`表示为转换工具的输出件，`runtime`表示为推理框架的输出件。 -> -> OS：输出件应部署的操作系统。 - -```bash -tar -xvf mindspore-lite-{version}-{function}-{OS}.tar.gz -``` -编译x86可获得转换工具`converter`与推理框架`runtime`功能的输出件，编译ARM仅能获得推理框架`runtime`。 - -输出件中包含以下几类子项，功能不同所含内容也会有所区别。 - -> 编译ARM64默认可获得`arm64-cpu`的推理框架输出件，若添加`-e gpu`则获得`arm64-gpu`的推理框架输出件，编译ARM32同理。 - -| 目录 | 说明 | converter | runtime | -| --- | --- | --- | --- | -| include | 推理框架头文件 | 无 | 有 | -| lib | 推理框架动态库 | 无 | 有 | -| benchmark | 基准测试工具 | 无 | 有 | -| time_profiler | 模型网络层耗时分析工具 | 无 | 有 | -| converter | 模型转换工具 | 有 | 无 | -| third_party | 第三方库头文件和库 | 有 | 有 | - -以0.7.0-beta版本，CPU编译为例，不同包名下，`third party`与`lib`的内容不同： - -- `mindspore-lite-0.7.0-converter-ubuntu`：包含`protobuf`（Protobuf的动态库）。 -- `mindspore-lite-0.7.0-runtime-x86-cpu`：`third party`包含`flatbuffers`（FlatBuffers头文件），`lib`包含`libmindspore-lite.so`（MindSpore Lite的动态库）。 -- `mindspore-lite-0.7.0-runtime-arm64-cpu`：`third party`包含`flatbuffers`（FlatBuffers头文件），`lib`包含`libmindspore-lite.so`（MindSpore Lite的动态库）和`liboptimize.so`。 -TODO：补全文件内容 - -> 运行converter、benchmark或time_profiler目录下的工具前，都需配置环境变量，将MindSpore Lite和Protobuf的动态库所在的路径配置到系统搜索动态库的路径中。以0.7.0-beta版本为例：`export LD_LIBRARY_PATH=./mindspore-lite-0.7.0/lib:./mindspore-lite-0.7.0/third_party/protobuf/lib:${LD_LIBRARY_PATH}`。 - -### 编译示例 - -首先，从MindSpore代码仓下载源码。 - -```bash -git clone https://gitee.com/mindspore/mindspore.git -b r0.7 -``` - -然后，在源码根目录下，执行如下命令，可编译不同版本的MindSpore Lite。 - -- 编译x86_64架构Debug版本。 - ```bash - bash build.sh -I x86_64 -d - ``` - -- 编译x86_64架构Release版本，同时设定线程数。 - ```bash - bash build.sh -I x86_64 -j32 - ``` - -- 增量编译ARM64架构Release版本，同时设定线程数。 - ```bash - bash build.sh -I arm64 -i -j32 - ``` - -- 编译ARM64架构Release版本，同时编译内置的GPU算子。 - ```bash - bash build.sh -I arm64 -e gpu - ``` - -> `build.sh`中会执行`git clone`获取第三方依赖库的代码，请提前确保git的网络设置正确可用。 - -以0.7.0-beta版本为例，x86_64架构Release版本编译完成之后，进入`mindspore/output`目录，执行如下解压缩命令，即可获取输出件`include`、`lib`、`benchmark`、`time_profiler`、`converter`和`third_party`。 - -```bash -tar -xvf mindspore-lite-0.7.0-converter-ubuntu.tar.gz -tar -xvf mindspore-lite-0.7.0-runtime-x86-cpu.tar.gz -``` - -## Windows环境部署 - -### 环境要求 - -- 编译环境仅支持32位或64位Windows系统 - -- 编译依赖（基本项） - - [CMake](https://cmake.org/download/) >= 3.14.1 - - [MinGW GCC](https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/7.3.0/threads-posix/seh/x86_64-7.3.0-release-posix-seh-rt_v5-rev0.7z/download) >= 7.3.0 - - [Python](https://www.python.org/) >= 3.7.5 - - [Git](https://git-scm.com/downloads) >= 2.28.0 - -### 编译选项 - -MindSpore Lite的编译选项如下。 - -| 参数 | 参数说明 | 取值范围 | 是否必选 | -| -------- | ----- | ---- | ---- | -| lite | 设置该参数，则对Mindspore Lite工程进行编译，否则对Mindspore工程进行编译 | - | 是 | -| [n] | 设定编译时所用的线程数，否则默认设定为6线程 | - | 否 | - -### 输出件说明 - -编译完成后，进入源码的`mindspore/output/`目录，可查看编译后生成的文件，命名为`mindspore-lite-{version}-converter-win-{process_unit}.zip`。解压后，即可获得编译后的工具包，名称为`mindspore-lite-{version}`。 - -> version：输出件版本，与所编译的MindSpore版本一致。 -> process_unit：输出件应部署的处理器类型。 - -### 编译示例 - -首先，使用git工具从MindSpore代码仓下载源码。 - -```bash -git clone https://gitee.com/mindspore/mindspore.git -b r0.7 -``` - -然后，使用cmd工具在源码根目录下，执行如下命令即可编译MindSpore Lite。 - -- 以默认线程数（6线程）编译Windows版本。 - ```bash - call build.bat lite - ``` -- 以指定线程数8编译Windows版本。 - ```bash - call build.bat lite 8 - ``` - -> `build.bat`中会执行`git clone`获取第三方依赖库的代码，请提前确保git的网络设置正确可用。 - -编译完成之后，进入`mindspore/output/`目录，解压后即可获取输出件`converter`。 diff --git a/lite/tutorials/source_zh_cn/index.rst b/lite/tutorials/source_zh_cn/index.rst index 52f49f3366457fc17b653fa30071d5cb67c82963..dacac7e5e29e0158579425c43f5a3e375564bfda 100644 --- a/lite/tutorials/source_zh_cn/index.rst +++ b/lite/tutorials/source_zh_cn/index.rst @@ -11,7 +11,7 @@ MindSpore端侧教程 :maxdepth: 1 :caption: 快速入门 - deploy + build quick_start/quick_start .. toctree:: diff --git a/lite/tutorials/source_zh_cn/use/benchmark_tool.md b/lite/tutorials/source_zh_cn/use/benchmark_tool.md index 0f8fc42728093e919b294e5d3d256fc023191f89..0ae2e21309452aa2ecdc1e851cdd6a052d549e60 100644 --- a/lite/tutorials/source_zh_cn/use/benchmark_tool.md +++ b/lite/tutorials/source_zh_cn/use/benchmark_tool.md @@ -16,22 +16,22 @@ ## 概述 -Benchmark工具是一款可以对MindSpore Lite模型进行基准测试的工具，由C++语言编码实现。它不仅可以对MindSpore Lite模型前向推理执行耗时进行定量分析（性能），还可以通过指定模型输出进行可对比的误差分析（精度）。 +Benchmark工具是一款可以对MindSpore Lite模型进行基准测试的工具。它不仅可以对MindSpore Lite模型前向推理执行耗时进行定量分析（性能），还可以通过指定模型输出进行可对比的误差分析（精度）。 ## 环境准备 使用Benchmark工具，需要进行如下环境准备工作。 -- 编译：Benchmark工具代码在MindSpore源码的`mindspore/lite/tools/benchmark`目录中，参考部署文档中的[环境要求](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/deploy.html#id2)和[编译示例](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/deploy.html#id5)，安装编译依赖基本项，并执行编译。 +- 编译：Benchmark工具代码在MindSpore源码的`mindspore/lite/tools/benchmark`目录中，参考构建文档中的[环境要求](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/build.html#id2)和[编译示例](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/build.html#id4)执行编译。 -- 运行：参考部署文档中的[输出件说明](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/deploy.html#id4)，获得`benchmark`工具，并配置环境变量。 +- 运行：参考构建文档中的[编译输出](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/build.html#id5)，获得`benchmark`工具，并配置环境变量。 ## 参数说明 使用编译好的Benchmark工具进行模型的基准测试时，其命令格式如下所示。 ```bash -./benchmark --modelPath= [--accuracyThreshold=] +./benchmark [--modelPath=] [--accuracyThreshold=] [--calibDataPath=] [--cpuBindMode=] [--device=] [--help] [--inDataPath=] [--inDataType=] [--loopCount=] @@ -51,11 +51,8 @@ Benchmark工具是一款可以对MindSpore Lite模型进行基准测试的工具 | `--device=` | 可选 | 指定模型推理程序运行的设备类型。 | String | CPU | CPU、GPU | | `--help` | 可选 | 显示`benchmark`命令的帮助信息。 | - | - | - | | `--inDataPath=` | 可选 | 指定测试模型输入数据的文件路径。如果未设置，则使用随机输入。 | String | null | - | -| `--inDataType=` | 可选 | 指定测试模型输入数据的文件类型。 | String | bin | img：表示输入数据的文件类型为图片
bin：表示输入数据的类型为二进制文件 | | `--loopCount=` | 可选 | 指定Benchmark工具进行基准测试时，测试模型的前向推理运行次数，其值为正整数。 | Integer | 10 | - | | `--numThreads=` | 可选 | 指定模型推理程序运行的线程数。 | Integer | 2 | - | -| `--omModelPath=` | 可选 | 指定OM模型的文件路径，此参数仅当`device`类型为NPU时可选设置。 | String | null | - | -| `--resizeDims=` | 可选 | 指定测试模型输入数据需要调整的尺寸大小。 | String | null | - | | `--warmUpLoopCount=` | 可选 | 指定测试模型在执行基准测试运行轮数前进行的模型预热推理次数。 | Integer | 3 | - | | `--fp16Priority=` | 可选 | 指定是否优先使用float16算子。 | Bool | false | true, false | diff --git a/lite/tutorials/source_zh_cn/use/converter_tool.md b/lite/tutorials/source_zh_cn/use/converter_tool.md index 00262445870bb0e5bbfe0f817c761e1dea35152b..7ea2ce905b3ec28ffc6e6e9d42a47c9ee9fcd2e7 100644 --- a/lite/tutorials/source_zh_cn/use/converter_tool.md +++ b/lite/tutorials/source_zh_cn/use/converter_tool.md @@ -7,13 +7,7 @@ - [Linux环境使用说明](#linux环境使用说明) - [环境准备](#环境准备) - [参数说明](#参数说明) - - [模型可视化](#模型可视化) - [使用示例](#使用示例) - - [Windows环境使用说明](#windows环境使用说明) - - [环境准备](#环境准备-1) - - [参数说明](#参数说明-1) - - [模型可视化](#模型可视化-1) - - [使用示例](#使用示例-1) @@ -21,7 +15,7 @@ ## 概述 -MindSpore Lite提供离线转换模型功能的工具，支持多种类型的模型转换，同时提供转化后模型可视化的功能，转换后的模型可用于推理。命令行参数包含多种个性化选项，为用户提供方便的转换途径。 +MindSpore Lite提供离线转换模型功能的工具，支持多种类型的模型转换，转换后的模型可用于推理。命令行参数包含多种个性化选项，为用户提供方便的转换途径。 目前支持的输入格式有：MindSpore、TensorFlow Lite、Caffe和ONNX。 @@ -31,9 +25,9 @@ MindSpore Lite提供离线转换模型功能的工具，支持多种类型的模 使用MindSpore Lite模型转换工具，需要进行如下环境准备工作。 -- 编译：模型转换工具代码在MindSpore源码的`mindspore/lite/tools/converter`目录中，参考部署文档中的[环境要求](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/deploy.html#id2)和[编译示例](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/deploy.html#id5)，安装编译依赖基本项与模型转换工具所需附加项，并编译x86_64版本。 +- 编译：模型转换工具代码在MindSpore源码的`mindspore/lite/tools/converter`目录中，参考构建文档中的[环境要求](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/build.html#id2)和[编译示例](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/build.html#id4)编译x86_64版本。 -- 运行：参考部署文档中的[输出件说明](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/deploy.html#id4)，获得`converter`工具，并配置环境变量。 +- 运行：参考构建文档中的[编译输出](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/build.html#id5)，获得`converter`工具，并配置环境变量。 ### 参数说明 @@ -49,20 +43,19 @@ MindSpore Lite提供离线转换模型功能的工具，支持多种类型的模 | `--modelFile=` | 是 | 输入模型的路径。 | - | - | | `--outputFile=` | 是 | 输出模型的路径（不存在时将自动创建目录），不需加后缀，可自动生成`.ms`后缀。 | - | - | | `--weightFile=` | 转换Caffe模型时必选 | 输入模型weight文件的路径。 | - | - | -| `--quantType=` | 否 | 设置模型的训练类型 | PostTraining：训练后量化
AwareTraining：感知量化。 | - | +| `--quantType=` | 否 | 设置模型的量化类型。 | PostTraining：训练后量化
AwareTraining：感知量化。 | - | +|` --inputInferenceType=` | 否 | 设置感知量化模型输入数据类型，如果和原模型不一致则转换工具会在模型前插转换算子，使得转换后的模型输入类型和inputInferenceType保持一致。 | FLOAT、INT8 | FLOAT | +| `--inferenceType=` | 否 | 设置感知量化模型输出数据类型，如果和原模型不一致则转换工具会在模型前插转换算子，使得转换后的模型输出类型和inferenceType保持一致。 | FLOAT、INT8 | FLOAT | +| `--stdDev= `| 否 | 感知量化模型转换时用于设置输入数据的标准差。 | （0，+∞） | 128 | +| `--mean=` | 否 | 感知量化模型转换时用于设置输入数据的均值。 | [-128, 127] | -0.5 | > - 参数名和参数值之间用等号连接，中间不能有空格。 > - Caffe模型一般分为两个文件：`*.prototxt`模型结构，对应`--modelFile`参数；`*.caffemodel`模型权值，对应`--weightFile`参数。 -### 模型可视化 - -模型可视化工具提供了一种查验模型转换结果的方法。用户可使用Json命令生成`*.json`文件，与原模型相对比，确定转化效果。 - -TODO: 此功能还在开发中。 ### 使用示例 -首先，在源码根目录下，输入命令进行编译，可参考`deploy.md`。 +首先，在源码根目录下，输入命令进行编译，可参考`build.md`。 ```bash bash build.sh -I x86_64 ``` @@ -106,84 +99,13 @@ bash build.sh -I x86_64 ./converter_lite --fmk=TFLITE --modelFile=model_quant.tflite --outputFile=model --quantType=AwareTraining ``` - 以上几种情况下，均显示如下转换成功提示，且同时获得`model.ms`目标文件。 - ``` - INFO [converter/converter.cc:190] Runconverter] CONVERTER RESULT: SUCCESS! - ``` + - 感知量化模型输入设置为int8，输出设置为int8 -你可以选择使用模型打印工具，可视化查验上述转化后生成的MindSpore Lite模型。本部分功能开发中。 - -## Windows环境使用说明 - -### 环境准备 - -使用MindSpore Lite模型转换工具，需要进行如下环境准备工作。 - -- 编译：模型转换工具代码在MindSpore源码的`mindspore/lite/tools/converter`目录中，参考部署文档中的[环境要求](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id7)和[编译示例](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id10)，安装编译依赖基本项与模型转换工具所需附加项，并编译Windows版本。 - -- 运行：参考部署文档中的[输出件说明](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id9)，获得`converter`工具，并将MinGW/bin目录下的几个依赖文件（libgcc_s_seh-1.dll、libwinpthread-1.dll、libssp-0.dll、libstdc++-6.dll）拷贝至`converter`工具的主目录。 - -### 参数说明 - -参考Linux环境模型转换工具的[参数说明](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/converter_tool.html#id4) - -### 模型可视化 - -参考Linux环境模型转换工具的[模型可视化](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/converter_tool.html#id5) - -### 使用示例 - -首先，使用cmd工具在源码根目录下，输入命令进行编译，可参考`deploy.md`。 -```bash -call build.bat lite -``` - -然后，设置日志打印级别为INFO。 -```bash -set MSLOG=INFO -``` - -下面选取了几个常用示例，说明转换命令的使用方法。 - -- 以Caffe模型LeNet为例，执行转换命令。 - - ```bash - call converter_lite --fmk=CAFFE --modelFile=lenet.prototxt --weightFile=lenet.caffemodel --outputFile=lenet - ``` - - 本例中，因为采用了Caffe模型，所以需要模型结构、模型权值两个输入文件。再加上其他必需的fmk类型和输出路径两个参数，即可成功执行。 - - 结果显示为： - ``` - INFO [converter/converter.cc:190] Runconverter] CONVERTER RESULT: SUCCESS! - ``` - 这表示已经成功将Caffe模型转化为MindSpore Lite模型，获得新文件`lenet.ms`。 - -- 以MindSpore、TensorFlow Lite、ONNX模型格式和感知量化模型为例，执行转换命令。 - - - MindSpore模型`model.mindir` - ```bash - call converter_lite --fmk=MS --modelFile=model.mindir --outputFile=model + ```bash + ./converter_lite --fmk=TFLITE --modelFile=model_quant.tflite --outputFile=model --quantType=AwareTraining --inputInferenceType=INT8 --inferenceType=INT8 ``` - - - TensorFlow Lite模型`model.tflite` - ```bash - call converter_lite --fmk=TFLITE --modelFile=model.tflite --outputFile=model - ``` - - - ONNX模型`model.onnx` - ```bash - call converter_lite --fmk=ONNX --modelFile=model.onnx --outputFile=model - ``` - - - TensorFlow Lite感知量化模型`model_quant.tflite` - ```bash - call converter_lite --fmk=TFLITE --modelFile=model_quant.tflite --outputFile=model --quantType=AwareTraining - ``` - 以上几种情况下，均显示如下转换成功提示，且同时获得`model.ms`目标文件。 + ``` INFO [converter/converter.cc:190] Runconverter] CONVERTER RESULT: SUCCESS! ``` - -你可以选择使用模型打印工具，可视化查验上述转化后生成的MindSpore Lite模型。本部分功能开发中。 diff --git a/lite/tutorials/source_zh_cn/use/runtime.md b/lite/tutorials/source_zh_cn/use/runtime.md index 2e2528b7a370c8128b7bcc88adf88699e5f85385..8d01a18329b7bcd2ed7575e64a662112c5c40839 100644 --- a/lite/tutorials/source_zh_cn/use/runtime.md +++ b/lite/tutorials/source_zh_cn/use/runtime.md @@ -13,20 +13,21 @@ - [可变维度](#可变维度) - [使用示例](#使用示例-1) - [图编译](#图编译-1) + - [使用示例](#使用示例-2) - [输入数据](#输入数据) - [获取输入Tensor](#获取输入tensor) - [数据拷贝](#数据拷贝) - - [使用示例](#使用示例-2) + - [使用示例](#使用示例-3) - [图执行](#图执行) - [执行会话](#执行会话) - [绑核](#绑核) - [回调运行](#回调运行) - - [使用示例](#使用示例-3) + - [使用示例](#使用示例-4) - [获取输出](#获取输出) - [获取输出Tensor](#获取输出tensor) - - [使用示例](#使用示例-4) - - [获取版本号](#获取版本号) - [使用示例](#使用示例-5) + - [获取版本号](#获取版本号) + - [使用示例](#使用示例-6) @@ -119,7 +120,7 @@ if (session == nullptr) { ### 使用示例 -下面代码演示如何对MindSpore Lite的输入进行Resize()： +下面代码演示如何对MindSpore Lite的输入进行Resize： ```cpp // Assume we have created a LiteSession instance named session. auto inputs = session->GetInputs(); @@ -133,6 +134,22 @@ session->Resize(inputs); 在图执行前，需要调用`LiteSession`的`CompileGraph`接口进行图编译，进一步解析从文件中加载的Model实例，主要进行子图切分、算子选型调度。这部分会耗费较多时间，所以建议`ListSession`创建一次，编译一次，多次执行。 +### 使用示例 + +下面代码演示如何进行图编译: +```cpp +// Assume we have created a LiteSession instance named session and a Model instance named model before. +// The methods of creating model and session can refer to "Import Model" and "Create Session" two sections. +auto ret = session->CompileGraph(model); +if (ret != RET_OK) { + std::cerr << "CompileGraph failed" << std::endl; + // session and model need to be released by users manually. + delete (session); + delete (model); + return ret; +} +``` + ## 输入数据 ### 获取输入Tensor @@ -318,7 +335,7 @@ if (out_data == nullptr) { } // Print the first 10 float data or all output data of the output tensor. std::cout << "Output data: "; -for (size_t i = 0; i < 10 & i < out_tensor->ElementsNum(); i++) { +for (size_t i = 0; i < 10 && i < out_tensor->ElementsNum(); i++) { std::cout << " " << out_data[i]; } std::cout << std::endl; @@ -354,6 +371,22 @@ if (out_tensor == nullptr) { } ``` +下面示例代码演示了使用`GetOutputByTensorName`接口获取输出`MSTensor`的方法： + +```cpp +// We can use GetOutputTensorNames method to get all name of output tensor of model which is in order. +auto tensor_names = this->GetOutputTensorNames(); +// Assume we have created a LiteSession instance named session before. +// Use output tensor name returned by GetOutputTensorNames as key +for (auto tensor_name : tensor_names) { + auto out_tensor = this->GetOutputByTensorName(tensor_name); + if (out_tensor == nullptr) { + std::cerr << "Output tensor is nullptr" << std::endl; + return -1; + } +} +``` + ## 获取版本号 MindSpore Lite提供了`Version`方法可以获取版本号，包含在`include/version.h`头文件中，调用该方法可以得到版本号字符串。 diff --git a/lite/tutorials/source_zh_cn/use/timeprofiler_tool.md b/lite/tutorials/source_zh_cn/use/timeprofiler_tool.md index 021101ad009d68ee6ca459eb162b017e5c1df6ac..ccb7820cc9d5a48331da2de3a0e91cc0a4c97f8c 100644 --- a/lite/tutorials/source_zh_cn/use/timeprofiler_tool.md +++ b/lite/tutorials/source_zh_cn/use/timeprofiler_tool.md @@ -20,9 +20,9 @@ TimeProfiler工具可以对MindSpore Lite模型网络层的前向推理进行耗 使用TimeProfiler工具，需要进行如下环境准备工作。 -- 编译：TimeProfiler工具代码在MindSpore源码的`mindspore/lite/tools/time_profiler`目录中，参考部署文档中的[环境要求](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/deploy.html#id2)和[编译示例](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/deploy.html#id5)，安装编译依赖基本项，并执行编译。 +- 编译：TimeProfiler工具代码在MindSpore源码的`mindspore/lite/tools/time_profiler`目录中，参考构建文档中的[环境要求](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/build.html#id2)和[编译示例](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/build.html#id4)执行编译。 -- 运行：参考部署文档中的[输出件说明](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/deploy.html#id4)，获得`time_profiler`工具，并配置环境变量。 +- 运行：参考部署文档中的[输出输出](https://www.mindspore.cn/lite/tutorial/zh-CN/r0.7/build.html#id4)，获得`time_profiler`工具，并配置环境变量。 ## 参数说明