MindSpore Lite is a lightweight deep neural network inference engine that provides the inference function for models trained by MindSpore on the device side. This tutorial describes how to use and compile MindSpore Lite.
MindSpore Lite is a lightweight deep neural network inference engine that provides the inference function for models on the device side. Models can be trained by MindSpore or imported from third-party, such as TensorFlow Lite, ONNX and Caffe. This tutorial describes how to use and compile MindSpore Lite for its own model.
...
@@ -41,49 +41,44 @@ The environment requirements are as follows:
...
@@ -41,49 +41,44 @@ The environment requirements are as follows:
- Hard disk space: 10 GB or above
- Hard disk space: 10 GB or above
- System requirements
- System requirements
- System is limited on Linux
- System is limited on Linux: Ubuntu = 18.04.02LTS
3.Run the following command in the root directory of the source code to compile MindSpore Lite.
3.Go to the `mindspore/output` directory of the source code to obtain the compilation result. Unzip `MSLite-0.6.0-linux_x86_64.tar.gz` to get the result after building.
```bash
```bash
cd mindspore/lite
tar xvf MSLite-0.6.0-linux_x86_64.tar.gz
sh build.sh
```
```
4. Obtain the compilation result.
Go to the `lite/build` directory of the source code to view the generated documents. Then you can use various tools after changing directory.
## Use of On-Device Inference
## Use of On-Device Inference
When MindSpore is used to perform model inference in the APK project of an app, preprocessing input is required before model inference. For example, before an image is converted into the tensor format required by MindSpore inference, the image needs to be resized. After MindSpore completes model inference, postprocess the model inference result and sends the processed output to the app.
When MindSpore is used to perform model inference in the APK project of an app, preprocessing input is required before model inference. For example, before an image is converted into the tensor format required by MindSpore inference, the image needs to be resized. After MindSpore completes model inference, postprocess the model inference result and sends the processed output to the app.
...
@@ -159,7 +154,7 @@ To perform on-device model inference using MindSpore, perform the following step
...
@@ -159,7 +154,7 @@ To perform on-device model inference using MindSpore, perform the following step
else:
else:
print("checkpoint file does not exist.")
print("checkpoint file does not exist.")
```
```
3.Calling MindSpore convert tool named `converter_lite`, convert model file (`.pb`) to on_device inference model file (`.ms`).
3.In `mindspore/output/MSLite-0.6.0-linux_x86_64/converter` directory, calling MindSpore convert tool named `converter_lite`, convert model file (`.pb`) to on_device inference model file (`.ms`).
@@ -175,54 +170,172 @@ Use the `.ms` model file and image data as input to create a session and impleme
...
@@ -175,54 +170,172 @@ Use the `.ms` model file and image data as input to create a session and impleme
Take the `lenet.ms` model as an example to implement on-device inference, the specific steps are:
1. Load the `.ms` model file to the memory buffer. The ReadFile function needs to be implemented by users, according to the [C++ tutorial](http://www.cplusplus.com/doc/tutorial/files/).
1. Load the `.ms` model file to the memory buffer. The ReadFile function needs to be implemented by users, according to the [C++ tutorial](http://www.cplusplus.com/doc/tutorial/files/).
```cpp
2. Call the `CreateSession` API to get a session.
// Read Model File
3. Call the `CompileGraph` API in previous `Session` and transport model.
4. Call the `GetInputs` API to get input `tensor`, then set graph information as `data`, `data` will be used in to perform model inference.
5. Call the `RunGraph` API in the `Session` to perform inference.
6. Call the `GetOutputs` API to obtain the output.
The complete sample code is as follows:
```CPP
#include <iostream>
#include <fstream>
#include "schema/model_generated.h"
#include "include/model.h"
#include "include/lite_session.h"
#include "include/errorcode.h"
#include "ir/dtype/type_id.h"
char *ReadFile(const char *file, size_t *size) {
if (file == nullptr) {
std::cerr << "file is nullptr" << std::endl;
return nullptr;
}
if (size == nullptr) {
std::cerr << "size is nullptr" << std::endl;
return nullptr;
}
std::ifstream ifs(file);
if (!ifs.good()) {
std::cerr << "file: " << file << " is not exist" << std::endl;
