> - To install and use `Android_NDK`, you need to configure environment variables. The command example is `export ANDROID_NDK={$NDK_PATH}/android-ndk-r20b`.
> - In the `build.sh` script, run the `git clone` command to obtain the code in the third-party dependency library. Ensure that the network settings of Git are correct.
### Compilation Options
MindSpore Lite provides a compilation script `build.sh` for one-click compilation, located in the root directory of MindSpore. This script can be used to compile the code of training and inference. The following describes the compilation options of MindSpore Lite.
| Parameter | Parameter Description | Value Range | Mandatory or Not |
| -------- | ----- | ---- | ---- |
| **-I** | **Selects an applicable architecture. This option is required when compile MindSpore Lite.** | **arm64, arm32, or x86_64** | **Yes** |
| -d | If this parameter is set, the debug version is compiled. Otherwise, the release version is compiled. | None | No |
| -i | If this parameter is set, incremental compilation is performed. Otherwise, full compilation is performed. | None | No |
| -j[n] | Sets the number of threads used during compilation. Otherwise, the number of threads is set to 8 by default. | Integer | No |
| -e | In the Arm architecture, select the backend operator and set the `gpu` parameter. The built-in GPU operator of the framework is compiled at the same time. | GPU | No |
| -h | Displays the compilation help information. | None | No |
> When the `-I` parameter changes, such as `-I x86_64` is converted to `-I arm64`, adding `-i` for parameter compilation does not take effect.
### Compilation Example
First, download source code from the MindSpore code repository.
Then, run the following commands in the root directory of the source code to compile MindSpore Lite of different versions:
- Debug version of the x86_64 architecture:
```bash
bash build.sh -I x86_64 -d
```
- Release version of the x86_64 architecture, with the number of threads set:
```bash
bash build.sh -I x86_64 -j32
```
- Release version of the Arm 64-bit architecture in incremental compilation mode, with the number of threads set:
```bash
bash build.sh -I arm64 -i-j32
```
- Release version of the Arm 64-bit architecture in incremental compilation mode, with the built-in GPU operator compiled:
```bash
bash build.sh -I arm64 -e gpu
```
### Output Description
After the compilation is complete, go to the `mindspore/output` directory of the source code to view the file generated after compilation. The file is divided into two parts.
-`mindspore-lite-{version}-converter-{os}.tar.gz`：Contains model conversion tool.
-`mindspore-lite-{version}-runtime-{os}-{device}.tar.gz`：Contains model inference framework、benchmarking tool and performance analysis tool.
> version: version of the output, consistent with that of the MindSpore.
>
> device: Currently divided into cpu (built-in CPU operator) and gpu (built-in CPU and GPU operator).
>
> os: Operating system on which the output will be deployed.
Execute the decompression command to obtain the compiled output:
#### Description of Converter's Directory Structure
The conversion tool is only available under the `-I x86_64` compilation option, and the content includes the following parts:
```
|
├── mindspore-lite-{version}-converter-{os}
│ └── converter # Model conversion Ttool
│ └── third_party # Header files and libraries of third party libraries
│ ├── protobuf # Dynamic library of Protobuf
```
#### Description of Runtime and Other tools' Directory Structure
The inference framework can be obtained under `-I x86_64`, `-I arm64` and `-I arm32` compilation options, and the content includes the following parts:
- When the compilation option is `-I x86_64`:
```
|
├── mindspore-lite-{version}-runtime-x86-cpu
│ └── benchmark # Benchmarking Tool
│ └── lib # Inference framework dynamic library
│ ├── libmindspore-lite.so # Dynamic library of infernece framework in MindSpore Lite
│ └── third_party # Header files and libraries of third party libraries
│ ├── flatbuffers # Header files of FlatBuffers
│ └── include # Header files of inference framework
│ └── time_profiler # Model network layer time-consuming analysis tool
```
- When the compilation option is `-I arm64`:
```
|
├── mindspore-lite-{version}-runtime-arm64-cpu
│ └── benchmark # Benchmarking Tool
│ └── lib # Inference framework dynamic library
│ ├── libmindspore-lite.so # Dynamic library of infernece framework in MindSpore Lite
│ ├── liboptimize.so # Operator performance optimization library in MindSpore Lite
│ └── third_party # Header files and libraries of third party libraries
│ ├── flatbuffers # Header files of FlatBuffers
│ └── include # Header files of inference framework
│ └── time_profiler # Model network layer time-consuming analysis tool
```
- When the compilation option is `-I arm32`:
```
|
├── mindspore-lite-{version}-runtime-arm64-cpu
│ └── benchmark # Benchmarking Tool
│ └── lib # Inference framework dynamic library
│ ├── libmindspore-lite.so # Dynamic library of infernece framework in MindSpore Lite
│ └── third_party # Header files and libraries of third party libraries
│ ├── flatbuffers # Header files of FlatBuffers
│ └── include # Header files of inference framework
│ └── time_profiler # Model network layer time-consuming analysis tool
```
> 1. `liboptimize.so` only exists in the output package of runtime-arm64 and is only used on ARMv8.2 and CPUs that support fp16.
> 2. Compile ARM64 to get the inference framework output of arm64-cpu by default, if you add `-e gpu`, you will get the inference framework output of arm64-gpu, and the package name is `mindspore-lite-{version}-runtime-arm64-gpu.tar.gz`, compiling ARM32 is in the same way.
> 3. Before running the tools in the converter, benchmark or time_profiler directory, you need to configure environment variables, and configure the path where the dynamic libraries of MindSpore Lite and Protobuf are located to the path where the system searches for dynamic libraries. Take the CPU compiled under version 0.7.0-beta as an example: configure converter: `export LD_LIBRARY_PATH=./output/mindspore-lite-0.7.0-converter-ubuntu/third_party/protobuf/lib:${LD_LIBRARY_PATH}`; configure benchmark and time_profiler: `export LD_LIBRARY_PATH= ./output/mindspore-lite-0.7.0-runtime-x86-cpu/lib:${LD_LIBRARY_PATH}`.
> - `Android_NDK` needs to be installed only when the Arm version is compiled. Skip this dependency when the x86_64 version is compiled.
> - To install and use `Android_NDK`, you need to configure environment variables. The command example is `export ANDROID_NDK={$NDK_PATH}/android-ndk-r20b`.
- Compilation dependencies (additional dependencies required by the MindSpore Lite model conversion tool, which is required only for compilation of the x86_64 version)
MindSpore Lite provides multiple compilation options. You can select different compilation options as required.
| Parameter | Parameter Description | Value Range | Mandatory or Not |
| -------- | ----- | ---- | ---- |
| -d | If this parameter is set, the debug version is compiled. Otherwise, the release version is compiled. | - | No |
| -i | If this parameter is set, incremental compilation is performed. Otherwise, full compilation is performed. | - | No |
| -j[n] | Sets the number of threads used during compilation. Otherwise, the number of threads is set to 8 by default. | - | No |
| -I | Selects an applicable architecture. | arm64, arm32, or x86_64 | Yes |
| -e | In the Arm architecture, select the backend operator and set the `gpu` parameter. The built-in GPU operator of the framework is compiled at the same time. | GPU | No |
| -h | Displays the compilation help information. | - | No |
> When the `-I` parameter changes, that is, the applicable architecture is changed, the `-i` parameter cannot be used for incremental compilation.
## Output Description
After the compilation is complete, go to the `mindspore/output` directory of the source code to view the file generated after compilation. The file is named `mindspore-lite-{version}-{function}-{OS}.tar.gz`. After decompression, the tool package named `mindspore-lite-{version}-{function}-{OS}` can be obtained.
> version: version of the output, consistent with that of the MindSpore.
>
> function: function of the output. `convert` indicates the output of the conversion tool and `runtime` indicates the output of the inference framework.
For the x86 architecture, you can obtain the output of the conversion tool and inference framework；But for the ARM architecture, you only get inference framework.
Generally, the compiled output files include the following types. The architecture selection affects the types of output files.
> For the Arm 64-bit architecture, you can obtain the output of the `arm64-cpu` inference framework. If `-e gpu` is added, you can obtain the output of the `arm64-gpu` inference framework. The compilation for arm 64-bit is the same as that for arm 32-bit.
| Directory | Description | converter | runtime |
| --- | --- | --- | --- |
| include | Inference framework header file | No | Yes |
| time_profiler | Time consumption analysis tool at the model network layer| No | Yes |
| converter | Model conversion tool | Yes | No | No |
| third_party | Header file and library of the third-party library | Yes | Yes |
Take the 0.7.0-beta version and CPU as an example. The contents of `third party` and `lib` vary depending on the architecture as follows:
-`mindspore-lite-0.7.0-converter-ubuntu`: include `protobuf` (Protobuf dynamic library).
-`mindspore-lite-0.7.0-runtime-x86-cpu`: include `flatbuffers` (FlatBuffers header file).
TODO: Add document content.
> Before running the tools in the `converter`, `benchmark`, or `time_profiler` directory, you need to configure environment variables and set the paths of the dynamic libraries of MindSpore Lite and Protobuf to the paths of the system dynamic libraries. The following uses the 0.7.0-beta version as an example: `export LD_LIBRARY_PATH=./mindspore-lite-0.7.0/lib:./mindspore-lite-0.7.0/third_party/protobuf/lib:${LD_LIBRARY_PATH}`.
## Compilation Example
First, download source code from the MindSpore code repository.
Then, run the following commands in the root directory of the source code to compile MindSpore Lite of different versions:
- Debug version of the x86_64 architecture:
```bash
bash build.sh -I x86_64 -d
```
- Release version of the x86_64 architecture, with the number of threads set:
```bash
bash build.sh -I x86_64 -j32
```
- Release version of the Arm 64-bit architecture in incremental compilation mode, with the number of threads set:
```bash
bash build.sh -I arm64 -i-j32
```
- Release version of the Arm 64-bit architecture in incremental compilation mode, with the built-in GPU operator compiled:
```bash
bash build.sh -I arm64 -e gpu
```
> - In the `build.sh` script, run the `git clone` command to obtain the code in the third-party dependency library. Ensure that the network settings of Git are correct.
Take the 0.7.0-beta version as an example. After the release version of the x86_64 architecture is compiled, go to the `mindspore/output` directory and run the following decompression command to obtain the output files `include`, `lib`, `benchmark`, `time_profiler`, `converter`, and `third_party`:
The Benchmark tool is used to perform benchmark testing on a MindSpore Lite model and is implemented using the C++ language. It can not only perform quantitative analysis (performance) on the forward inference execution duration of a MindSpore Lite model, but also perform comparative error analysis (accuracy) based on the output of the specified model.
The Benchmark tool is used to perform benchmark testing on a MindSpore Lite model. It can not only perform quantitative analysis (performance) on the forward inference execution duration of a MindSpore Lite model, but also perform comparative error analysis (accuracy) based on the output of the specified model.
## Environment Preparation
To use the Benchmark tool, you need to prepare the environment as follows:
- Compilation: Install compilation dependencies and perform compilation. The code of the Benchmark tool is stored in the `mindspore/lite/tools/benchmark` directory of the MindSpore source code. For details about the compilation operations, see the [Environment Requirements](https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id2) and [Compilation Example](https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id5) in the deployment document.
- Compilation: Install build dependencies and perform build. The code of the Benchmark tool is stored in the `mindspore/lite/tools/benchmark` directory of the MindSpore source code. For details about the build operations, see the [Environment Requirements](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#environment-requirements) and [Compilation Example](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#compilation-example) in the build document.
- Run: Obtain the `Benchmark` tool and configure environment variables. For details, see [Output Description](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id4) in the deployment document.
- Run: Obtain the `Benchmark` tool and configure environment variables. For details, see [Output Description](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#output-description) in the build document.
## Parameter Description
The command used for benchmark testing based on the compiled Benchmark tool is as follows:
@@ -50,12 +50,9 @@ The following describes the parameters in detail.
| `--cpuBindMode=<CPUBINDMODE>` | Optional | Specifies the type of the CPU core bound to the model inference program. | Integer | 1 | −1: medium core<br/>1: large core<br/>0: not bound |
| `--device=<DEVICE>` | Optional | Specifies the type of the device on which the model inference program runs. | String | CPU | CPU or GPU |
| `--help` | Optional | Displays the help information about the `benchmark` command. | - | - | - |
| `--inDataPath=<INDATAPATH>` | Optional | Specifies the file path of the input data of the tested model. If this parameter is not set, a random value will be used. | String | Null | - |
| `--inDataType=<INDATATYPE>` | Optional | Specifies the file type of the input data of the tested model. | String | Bin | Img: The input data is an image. Bin: The input data is a binary file.|
| `--inDataPath=<INDATAPATH>` | Optional | Specifies the file path of the input data of the tested model. If this parameter is not set, a random value will be used. | String | Null | - |
| `--loopCount=<LOOPCOUNT>` | Optional | Specifies the number of forward inference times of the tested model when the Benchmark tool is used for the benchmark testing. The value is a positive integer. | Integer | 10 | - |
| `--numThreads=<NUMTHREADS>` | Optional | Specifies the number of threads for running the model inference program. | Integer | 2 | - |
| `--omModelPath=<OMMODELPATH>` | Optional | Specifies the file path of the OM model. This parameter is optional only when the `device` type is NPU. | String | Null | - |
| `--resizeDims=<RESIZEDIMS>` | Optional | Specifies the size to be adjusted for the input data of the tested model. | String | Null | - |
| `--warmUpLoopCount=<WARMUPLOOPCOUNT>` | Optional | Specifies the number of preheating inference times of the tested model before multiple rounds of the benchmark test are executed. | Integer | 3 | - |
| `--fp16Priority=<FP16PIORITY>` | Optional | Specifies whether the float16 operator is preferred. | Bool | false | true, false |
MindSpore Lite provides a tool for offline model conversion. It supports conversion of multiple types of models and visualization of converted models. The converted models can be used for inference. The command line parameters contain multiple personalized options, providing a convenient conversion method for users.
MindSpore Lite provides a tool for offline model conversion. It supports conversion of multiple types of models. The converted models can be used for inference. The command line parameters contain multiple personalized options, providing a convenient conversion method for users.
Currently, the following input formats are supported: MindSpore, TensorFlow Lite, Caffe, and ONNX.
## Environment Preparation
## Linux Environment Instructions
### Environment Preparation
To use the MindSpore Lite model conversion tool, you need to prepare the environment as follows:
- Compilation: Install basic and additional compilation dependencies and perform compilation. The compilation version is x86_64. The code of the model conversion tool is stored in the `mindspore/lite/tools/converter` directory of the MindSpore source code. For details about the compilation operations, see the [Environment Requirements] (https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id2) and [Compilation Example] (https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id5) in the deployment document.
- Compilation: Install basic and additional build dependencies and perform build. The build version is x86_64. The code of the model conversion tool is stored in the `mindspore/lite/tools/converter` directory of the MindSpore source code. For details about the build operations, see the [Environment Requirements](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#environment-requirements) and [Compilation Example](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#compilation-example) in the build document.
- Run: Obtain the `converter` tool and configure environment variables by referring to [Output Description](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id4) in the deployment document.
- Run: Obtain the `converter` tool and configure environment variables by referring to [Output Description](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#output-description) in the build document.
## Parameter Description
### Parameter Description
You can use `./converter_lite ` to complete the conversion. In addition, you can set multiple parameters as required.
You can enter `./converter_lite --help` to obtain help information in real time.
The following describes the parameters in detail.
| Parameter | Mandatory or Not | Parameter Description | Value Range | Default Value |
| -------- | ------- | ----- | --- | ---- |
| `--help` | No | Prints all help information. | - | - |
...
...
@@ -42,20 +44,19 @@ The following describes the parameters in detail.
| `--modelFile=<MODELFILE>` | Yes | Path of the input model. | - | - |
| `--outputFile=<OUTPUTFILE>` | Yes | Path of the output model. (If the path does not exist, a directory will be automatically created.) The suffix `.ms` can be automatically generated. | - | - |
| `--weightFile=<WEIGHTFILE>` | Yes (for Caffe models only) | Path of the weight file of the input model. | - | - |
| `--quantType=<QUANTTYPE>` | No | Sets the training type of the model. | PostTraining: quantization after training <br>AwareTraining: perceptual quantization | - |
| `--quantType=<QUANTTYPE>` | No | Sets the quant type of the model. | PostTraining: quantization after training <br>AwareTraining: perceptual quantization | - |
|`--inputInferenceType=<INPUTRINFERENCETYPE>` | No(supported by aware quant models only) | Sets the input data type of the converted model. If the type is different from the origin model, the convert tool will insert data type convert op before the model to make sure the input data type is same as the input of origin model. | FLOAT or INT8 | FLOAT |
|`--inferenceType=<INFERENCETYPE> `| No(supported by aware quant models only) | Sets the output data type of the converted model. If the type is different from the origin model, the convert tool will insert data type convert op before the model to make sure the output data type is same as the input of origin model. | FLOAT or INT8 | FLOAT |
|`--stdDev=<STDDEV>`| No(supported by aware quant models only) | Sets the standard deviation of the input data. | （0，+∞） | 128 |
|`--mean=<MEAN>`| No(supported by aware quant models only) | Sets the mean value of the input data. | [-128, 127] | -0.5 |
> - The parameter name and parameter value are separated by an equal sign (=) and no space is allowed between them.
> - The Caffe model is divided into two files: model structure `*.prototxt`, corresponding to the `--modelFile` parameter; model weight `*.caffemodel`, corresponding to the `--weightFile` parameter
## Model Visualization
The model visualization tool provides a method for checking the model conversion result. You can run the JSON command to generate a `*.json` file and compare it with the original model to determine the conversion effect.
TODO: This function is under development now.
## Example
### Example
First, in the root directory of the source code, run the following command to perform compilation. For details, see `deploy.md`.
First, in the root directory of the source code, run the following command to perform compilation. For details, see `build.md`.
```bash
bash build.sh -I x86_64
```
...
...
@@ -94,15 +95,17 @@ The following describes how to use the conversion command by using several commo
In the preceding scenarios, the following information is displayed, indicating that the conversion is successful. In addition, the target file `model.ms` is obtained.
```
INFO [converter/converter.cc:190] Runconverter] CONVERTER RESULT: SUCCESS!
```
You can use the model visualization tool to visually check the converted MindSpore Lite model. This function is under development.
@@ -20,9 +20,9 @@ The TimeProfiler tool can be used to analyze the time consumption of forward inf
To use the TimeProfiler tool, you need to prepare the environment as follows:
- Compilation: Install compilation dependencies and perform compilation. The code of the TimeProfiler tool is stored in the `mindspore/lite/tools/time_profiler` directory of the MindSpore source code. For details about the compilation operations, see the [Environment Requirements](https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id2) and [Compilation Example](https://www.mindspore.cn/lite/docs/en/r0.7/deploy.html#id5) in the deployment document.
- Compilation: Install build dependencies and perform build. The code of the TimeProfiler tool is stored in the `mindspore/lite/tools/time_profiler` directory of the MindSpore source code. For details about the build operations, see the [Environment Requirements](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#environment-requirements) and [Compilation Example](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#compilation-example) in the build document.
- Run: Obtain the `time_profiler` tool and configure environment variables by referring to [Output Description](https://www.mindspore.cn/lite/docs/zh-CN/r0.7/deploy.html#id4) in the deployment document.
- Run: Obtain the `time_profiler` tool and configure environment variables by referring to [Output Description](https://www.mindspore.cn/lite/tutorial/en/r0.7/build.html#output-description) in the build document.
