diff --git a/docs/source_en/glossary.md b/docs/source_en/glossary.md index 0d2d6bccac6de31cad4bc17077bfcf018fb021c5..65bbf149fc348813d044c9fa177bcf368caf2b0b 100644 --- a/docs/source_en/glossary.md +++ b/docs/source_en/glossary.md @@ -40,7 +40,7 @@ | MindData | MindSpore data framework, which provides data loading, enhancement, dataset management, and visualization. | | MindInsight | MindSpore visualization component, which visualizes information such as scalars, images, computational graphs, and model hyperparameters. | | MindSpore | Huawei-leaded open-source deep learning framework. | -| MindSpore Predict | A lightweight deep neural network inference engine that provides the inference function for models trained by MindSpore on the device side. | +| MindSpore Lite | A lightweight deep neural network inference engine that provides the inference function for models trained by MindSpore on the device side. | | MNIST database | Modified National Handwriting of Images and Technology database, a large handwritten digit database, which is usually used to train various image processing systems. | | ONNX | Open Neural Network Exchange, is an open format built to represent machine learning models.| | PyNative Mode | MindSpore dynamic graph mode. In this mode, operators in the neural network are delivered and executed one by one, facilitating the compilation and debugging of the neural network model. | diff --git a/docs/source_zh_cn/glossary.md b/docs/source_zh_cn/glossary.md index a21a46799cf800ec6bcf38503d93e04f4ef30850..ceda67fe3f19b91ad2ac664e0cb353493fdc92f0 100644 --- a/docs/source_zh_cn/glossary.md +++ b/docs/source_zh_cn/glossary.md @@ -40,7 +40,7 @@ | MindData | MindSpore数据框架,提供数据加载、增强、数据集管理以及可视化。 | | MindInsight | MindSpore可视化组件,可视化标量、图像、计算图以及模型超参等信息。 | | MindSpore | 华为主导开源的深度学习框架。 | -| MindSpore Predict | 一个轻量级的深度神经网络推理引擎,提供了将MindSpore训练出的模型在端侧进行推理的功能。 | +| MindSpore Lite | 一个轻量级的深度神经网络推理引擎,提供了将MindSpore训练出的模型在端侧进行推理的功能。 | | MNIST database | Modified National Institute of Standards and Technology database,一个大型手写数字数据库,通常用于训练各种图像处理系统。 | | ONNX | Open Neural Network Exchange,是一种针对机器学习所设计的开放式的文件格式,用于存储训练好的模型。| | PyNative Mode | MindSpore的动态图模式,将神经网络中的各个算子逐一下发执行,方便用户编写和调试神经网络模型。 | diff --git a/tutorials/source_en/advanced_use/on_device_inference.md b/tutorials/source_en/advanced_use/on_device_inference.md index a4b6d5c2cd7ae5c93e667d59001f8d9bc0b693e7..4e444c623118590019222d6e181fb4f6fc9a9fee 100644 --- a/tutorials/source_en/advanced_use/on_device_inference.md +++ b/tutorials/source_en/advanced_use/on_device_inference.md @@ -110,11 +110,11 @@ To perform on-device model inference using MindSpore, perform the following step param_dict = load_checkpoint(ckpt_file_name=ckpt_file_path) load_param_into_net(net, param_dict) ``` -2. Call the `export` API to export the `.pb` model file on the device. +2. Call the `export` API to export the `.mindir` model file on the device. ```python export(net, input_data, file_name="./lenet.mindir", file_format='MINDIR') ``` - Take the LeNet network as an example. The generated on-device model file is `lenet.pb`. The complete sample code `lenet.py` is as follows: + Take the LeNet network as an example. The generated on-device model file is `lenet.mindir`. The complete sample code `lenet.py` is as follows: ```python import os import numpy as np @@ -171,9 +171,9 @@ To perform on-device model inference using MindSpore, perform the following step else: print("checkpoint file does not exist.") ``` -3. In `mindspore/output/MSLite-0.6.0-linux_x86_64/converter` directory, calling MindSpore convert tool named `converter_lite`, convert model file (`.pb`) to on_device inference model file (`.ms`). +3. In `mindspore/output/MSLite-0.6.0-linux_x86_64/converter` directory, calling MindSpore convert tool named `converter_lite`, convert model file (`.mindir`) to on_device inference model file (`.ms`). ``` - ./converter_lite --fmk=MS --modelFile=./lenet.pb --outputFile=lenet + ./converter_lite --fmk=MS --modelFile=./lenet.mindir --outputFile=lenet ``` Result: ``` diff --git a/tutorials/source_en/use/multi_platform_inference.md b/tutorials/source_en/use/multi_platform_inference.md index fc9cb7da340a93b52510a30856a7142e2d004127..080dc3588985fe8fc6dc6ea7e551edc709668625 100644 --- a/tutorials/source_en/use/multi_platform_inference.md +++ b/tutorials/source_en/use/multi_platform_inference.md @@ -114,7 +114,7 @@ MindSpore supports the following inference scenarios based on the hardware platf The Ascend 310 AI processor is equipped with the ACL framework and supports the OM format which needs to be converted from the model in ONNX or AIR format. For inference on the Ascend 310 AI processor, perform the following steps: -1. Generate a model in ONNX or AIR format on the training platform. For details, see [Export AIR Model and ONNX Model](https://www.mindspore.cn/tutorial/en/master/use/saving_and_loading_model_parameters.html#aironnx). +1. Generate a model in ONNX or AIR format on the training platform. For details, see [Export AIR Model](https://www.mindspore.cn/tutorial/en/master/use/saving_and_loading_model_parameters.html#export-air-model) and [Export ONNX Model](https://www.mindspore.cn/tutorial/en/master/use/saving_and_loading_model_parameters.html#export-onnx-model). 2. Convert the ONNX or AIR model file into an OM model file and perform inference. - For performing inference in the cloud environment (ModelArt), see the [Ascend 910 training and Ascend 310 inference samples](https://support.huaweicloud.com/bestpractice-modelarts/modelarts_10_0026.html). @@ -128,7 +128,7 @@ The inference is the same as that on the Ascend 910 AI processor. ### Inference Using an ONNX File -1. Generate a model in ONNX format on the training platform. For details, see [Export AIR Model and ONNX Model](https://www.mindspore.cn/tutorial/en/master/use/saving_and_loading_model_parameters.html#aironnx). +1. Generate a model in ONNX format on the training platform. For details, see [Export ONNX Model](https://www.mindspore.cn/tutorial/en/master/use/saving_and_loading_model_parameters.html#export-onnx-model). 2. Perform inference on a GPU by referring to the runtime or SDK document. For example, use TensorRT to perform inference on the NVIDIA GPU. For details, see [TensorRT backend for ONNX](https://github.com/onnx/onnx-tensorrt). @@ -140,10 +140,10 @@ The inference is the same as that on the Ascend 910 AI processor. ### Inference Using an ONNX File Similar to the inference on a GPU, the following steps are required: -1. Generate a model in ONNX format on the training platform. For details, see [Export AIR Model and ONNX Model](https://www.mindspore.cn/tutorial/en/master/use/saving_and_loading_model_parameters.html#aironnx). +1. Generate a model in ONNX format on the training platform. For details, see [Export ONNX Model](https://www.mindspore.cn/tutorial/en/master/use/saving_and_loading_model_parameters.html#export-onnx-model). 2. Perform inference on a CPU by referring to the runtime or SDK document. For details about how to use the ONNX Runtime, see the [ONNX Runtime document](https://github.com/microsoft/onnxruntime). ## On-Device Inference -MindSpore Predict is an inference engine for on-device inference. For details, see [On-Device Inference](https://www.mindspore.cn/tutorial/en/master/advanced_use/on_device_inference.html). +MindSpore Lite is an inference engine for on-device inference. For details, see [Export MINDIR Model](https://www.mindspore.cn/tutorial/en/master/use/saving_and_loading_model_parameters.html#export-mindir-model) and [On-Device Inference](https://www.mindspore.cn/tutorial/en/master/advanced_use/on_device_inference.html). diff --git a/tutorials/source_en/use/saving_and_loading_model_parameters.md b/tutorials/source_en/use/saving_and_loading_model_parameters.md index 33a1b397019f1b7da23e0d4c719e130ec3aa2544..34389472531155fa080aa7d0975357de50788e3e 100644 --- a/tutorials/source_en/use/saving_and_loading_model_parameters.md +++ b/tutorials/source_en/use/saving_and_loading_model_parameters.md @@ -9,7 +9,10 @@ - [Loading Model Parameters](#loading-model-parameters) - [For Inference Validation](#for-inference-validation) - [For Retraining](#for-retraining) - - [Export AIR Model and ONNX Model](#export-air-model-and-onnx-model) + - [Export Model](#export-model) + - [Export AIR Model](#export-air-model) + - [Export ONNX Model](#export-onnx-model) + - [Export MINDIR Model](#export-mindir-model) @@ -140,10 +143,14 @@ model.train(epoch, dataset) The `load_checkpoint` method returns a parameter dictionary and then the `load_param_into_net` method loads parameters in the parameter dictionary to the network or optimizer. -## Export AIR Model and ONNX Model -When you have a CheckPoint file, if you want to do inference, you need to generate corresponding models based on the network and CheckPoint. -Currently we support the export of AIR models based on Ascend AI processor and the export of ONNX models. Taking the export of AIR model as an example to illustrate the implementation of model export, -the code is as follows: +## Export Model + +When you have a CheckPoint file, if you want to do inference, you need to generate corresponding models based on the network and CheckPoint. The `export` interface supports exporting multiple types of model file formats for inference on different hardware platforms. + +### Export AIR Model + +AIR format file only supports Ascend AI processor. The code example of exporting this format file is as follows: + ```python from mindspore.train.serialization import export import numpy as np @@ -153,13 +160,37 @@ param_dict = load_checkpoint("resnet50-2_32.ckpt") # load the parameter into net load_param_into_net(resnet, param_dict) input = np.random.uniform(0.0, 1.0, size = [32, 3, 224, 224]).astype(np.float32) -export(resnet, Tensor(input), file_name = 'resnet50-2_32.pb', file_format = 'AIR') +export(resnet, Tensor(input), file_name = 'resnet50-2_32.air', file_format = 'AIR') ``` + Before using the `export` interface, you need to import` mindspore.train.serialization`. + The `input` parameter is used to specify the input shape and data type of the exported model. -If you want to export the ONNX model, you only need to specify the `file_format` parameter in the` export` interface as ONNX: `file_format = 'ONNX'`. -## Export MINDIR Model +It is recommended to use '.air' as the suffix of AIR format files. + +### Export ONNX Model + +ONNX format file is a general model file, which can be applied to many kinds of hardware, such as Ascend AI processor, GPU, CPU, etc. The code example of exporting this format file is as follows: + +```python +from mindspore.train.serialization import export +import numpy as np +resnet = ResNet50() +# return a parameter dict for model +param_dict = load_checkpoint("resnet50-2_32.ckpt") +# load the parameter into net +load_param_into_net(resnet, param_dict) +input = np.random.uniform(0.0, 1.0, size = [32, 3, 224, 224]).astype(np.float32) +export(resnet, Tensor(input), file_name = 'resnet50-2_32.onnx', file_format = 'ONNX') +``` + +It is recommended to use '.onnx' as the suffix of ONNX format files. + +### Export MINDIR Model + +MINDIR format file can be applied to MindSpore Lite and MindSpore Serving. Currently, it supports inference network based on static graph without control flow semantics. + If you want to do inference on the device, then you need to generate corresponding MINDIR models based on the network and CheckPoint. Currently we support the export of MINDIR models for inference based on graph mode, which don't contain control flow. Taking the export of MINDIR model as an example to illustrate the implementation of model export, the code is as follows: @@ -174,5 +205,5 @@ load_param_into_net(resnet, param_dict) input = np.random.uniform(0.0, 1.0, size = [32, 3, 224, 224]).astype(np.float32) export(resnet, Tensor(input), file_name = 'resnet50-2_32.mindir', file_format = 'MINDIR') ``` -Before using the `export` interface, you need to import` mindspore.train.serialization`. -The `input` parameter is used to specify the input shape and data type of the exported model. + +It is recommended to use '.mindir' as the suffix of MINDIR format files. diff --git a/tutorials/source_zh_cn/advanced_use/on_device_inference.md b/tutorials/source_zh_cn/advanced_use/on_device_inference.md index 3ee60e955eec3fee18c99caa377444b61102b0bc..f52aa0f6b4b85476724a32b47108be86949a568f 100644 --- a/tutorials/source_zh_cn/advanced_use/on_device_inference.md +++ b/tutorials/source_zh_cn/advanced_use/on_device_inference.md @@ -110,12 +110,12 @@ MindSpore进行端侧模型推理的步骤如下。 param_dict = load_checkpoint(ckpt_file_name=ckpt_file_path) load_param_into_net(net, param_dict) ``` -2. 调用`export`接口,导出模型文件(`.pb`)。 +2. 调用`export`接口,导出模型文件(`.mindir`)。 ```python export(net, input_data, file_name="./lenet.mindir", file_format='MINDIR') ``` - 以LeNet网络为例,生成的端侧模型文件为`lenet.pb`,完整示例代码`lenet.py`如下。 + 以LeNet网络为例,生成的端侧模型文件为`lenet.mindir`,完整示例代码`lenet.py`如下。 ```python import os import numpy as np @@ -172,9 +172,9 @@ MindSpore进行端侧模型推理的步骤如下。 else: print("checkpoint file does not exist.") ``` -3. 在`mindspore/output/MSLite-0.6.0-linux_x86_64/converter`路径下,调用MindSpore端侧转换工具`converter_lite`,将模型文件(`.pb`)转换为端侧模型文件(`.ms`)。 +3. 在`mindspore/output/MSLite-0.6.0-linux_x86_64/converter`路径下,调用MindSpore端侧转换工具`converter_lite`,将模型文件(`.mindir`)转换为端侧模型文件(`.ms`)。 ``` - ./converter_lite --fmk=MS --modelFile=./lenet.pb --outputFile=lenet + ./converter_lite --fmk=MS --modelFile=./lenet.mindir --outputFile=lenet ``` 结果显示为: ``` diff --git a/tutorials/source_zh_cn/use/multi_platform_inference.md b/tutorials/source_zh_cn/use/multi_platform_inference.md index 7f41f946bb7d4c8bbda362c0746bc1aebee3f9a5..c5cc88762d4fce4cb5332d6c9f46a2a92d5c96a7 100644 --- a/tutorials/source_zh_cn/use/multi_platform_inference.md +++ b/tutorials/source_zh_cn/use/multi_platform_inference.md @@ -111,7 +111,7 @@ CPU | ONNX格式 | 支持ONNX推理的runtime/SDK,如TensorRT。 Ascend 310 AI处理器上搭载了ACL框架,他支持OM格式,而OM格式需要从ONNX或者AIR模型进行转换。所以在Ascend 310 AI处理器上推理,需要下述两个步骤: -1. 在训练平台上生成ONNX或AIR格式模型,具体步骤请参考[模型导出-导出AIR模型和ONNX模型](https://www.mindspore.cn/tutorial/zh-CN/master/use/saving_and_loading_model_parameters.html#aironnx)。 +1. 在训练平台上生成ONNX或AIR格式模型,具体步骤请参考[导出AIR格式文件](https://www.mindspore.cn/tutorial/zh-CN/master/use/saving_and_loading_model_parameters.html#air)和[导出ONNX格式文件](https://www.mindspore.cn/tutorial/zh-CN/master/use/saving_and_loading_model_parameters.html#onnx)。 2. 将ONNX/AIR格式模型文件,转化为OM格式模型,并进行推理。 - 云上(ModelArt环境),请参考[Ascend910训练和Ascend310推理的样例](https://support.huaweicloud.com/bestpractice-modelarts/modelarts_10_0026.html)完成推理操作。 @@ -125,7 +125,7 @@ Ascend 310 AI处理器上搭载了ACL框架,他支持OM格式,而OM格式需 ### 使用ONNX格式文件推理 -1. 在训练平台上生成ONNX格式模型,具体步骤请参考[模型导出-导出AIR模型和ONNX模型](https://www.mindspore.cn/tutorial/zh-CN/master/use/saving_and_loading_model_parameters.html#aironnx)。 +1. 在训练平台上生成ONNX格式模型,具体步骤请参考[导出ONNX格式文件](https://www.mindspore.cn/tutorial/zh-CN/master/use/saving_and_loading_model_parameters.html#onnx)。 2. 在GPU上进行推理,具体可以参考推理使用runtime/SDK的文档。如在Nvidia GPU上进行推理,使用常用的TensorRT,可参考[TensorRT backend for ONNX](https://github.com/onnx/onnx-tensorrt)。 @@ -137,10 +137,10 @@ Ascend 310 AI处理器上搭载了ACL框架,他支持OM格式,而OM格式需 ### 使用ONNX格式文件推理 与在GPU上进行推理类似,需要以下几个步骤: -1. 在训练平台上生成ONNX格式模型,具体步骤请参考[模型导出-导出AIR模型和ONNX模型](https://www.mindspore.cn/tutorial/zh-CN/master/use/saving_and_loading_model_parameters.html#aironnx)。 +1. 在训练平台上生成ONNX格式模型,具体步骤请参考[导出ONNX格式文件](https://www.mindspore.cn/tutorial/zh-CN/master/use/saving_and_loading_model_parameters.html#onnx)。 2. 在CPU上进行推理,具体可以参考推理使用runtime/SDK的文档。如使用ONNX Runtime,可以参考[ONNX Runtime说明文档](https://github.com/microsoft/onnxruntime)。 ## 端侧推理 -端侧推理需使用MindSpore Predict推理引擎,详细操作请参考[端侧推理教程](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/on_device_inference.html)。 +端侧推理需使用MindSpore Lite推理引擎,详细操作请参考[导出MINDIR格式文件](https://www.mindspore.cn/tutorial/zh-CN/master/use/saving_and_loading_model_parameters.html#mindir)和[端侧推理教程](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/on_device_inference.html)。 diff --git a/tutorials/source_zh_cn/use/saving_and_loading_model_parameters.md b/tutorials/source_zh_cn/use/saving_and_loading_model_parameters.md index 48daa9302af574b966a225ffecee5c8dfc4bc919..2853ccb29168729b1e2315bb762f5e26d080dd2a 100644 --- a/tutorials/source_zh_cn/use/saving_and_loading_model_parameters.md +++ b/tutorials/source_zh_cn/use/saving_and_loading_model_parameters.md @@ -9,7 +9,10 @@ - [模型参数加载](#模型参数加载) - [用于推理验证](#用于推理验证) - [用于再训练场景](#用于再训练场景) - - [导出AIR模型和ONNX模型](#导出air模型和onnx模型) + - [导出模型](#导出模型) + - [导出AIR格式文件](#导出air格式文件) + - [导出ONNX格式文件](#导出onnx格式文件) + - [导出MINDIR格式文件](#导出mindir格式文件) @@ -141,9 +144,14 @@ model.train(epoch, dataset) `load_checkpoint`方法会返回一个参数字典,`load_param_into_net`会把参数字典中相应的参数加载到网络或优化器中。 -## 导出AIR模型和ONNX模型 -当有了CheckPoint文件后,如果想继续做推理,就需要根据网络和CheckPoint生成对应的模型,当前我们支持基于昇腾AI处理器的AIR模型导出和通用ONNX模型的导出。 -下面以AIR为例说明模型导出的实现,代码如下: +## 导出模型 + +当有了CheckPoint文件后,如果想继续做推理,需要通过网络和CheckPoint生成对应的模型。`export`接口支持导出多种类型的模型文件格式,用于不同硬件平台的推理。 + +### 导出AIR格式文件 + +AIR格式文件仅支持昇腾AI处理器,导出该格式文件的代码样例如下: + ```python from mindspore.train.serialization import export import numpy as np @@ -153,15 +161,37 @@ param_dict = load_checkpoint("resnet50-2_32.ckpt") # load the parameter into net load_param_into_net(resnet, param_dict) input = np.random.uniform(0.0, 1.0, size = [32, 3, 224, 224]).astype(np.float32) -export(resnet, Tensor(input), file_name = 'resnet50-2_32.pb', file_format = 'AIR') +export(resnet, Tensor(input), file_name = 'resnet50-2_32.air', file_format = 'AIR') ``` + 使用`export`接口之前,需要先导入`mindspore.train.serialization`。 + `input`用来指定导出模型的输入shape以及数据类型。 -如果要导出ONNX模型,只需要将`export`接口中的`file_format`参数指定为ONNX即可:`file_format = 'ONNX'`。 -## 导出MINDIR模型 -如果想将训练好的模型用于端测推理,就需要将网络和CheckPoint生成对应的MINDIR模型,当前我们支持基于静态图,不包含控制流语义的推理网络导出。 -下面以MINDIR为例说明模型导出的实现,代码如下: +建议使用`.air`作为AIR格式文件的后缀名。 + +### 导出ONNX格式文件 + +ONNX格式文件是一种较为通用的模型文件,可适用于昇腾AI处理器、GPU、CPU等多种硬件。导出该格式文件的代码样例如下: + +```python +from mindspore.train.serialization import export +import numpy as np +resnet = ResNet50() +# return a parameter dict for model +param_dict = load_checkpoint("resnet50-2_32.ckpt") +# load the parameter into net +load_param_into_net(resnet, param_dict) +input = np.random.uniform(0.0, 1.0, size = [32, 3, 224, 224]).astype(np.float32) +export(resnet, Tensor(input), file_name = 'resnet50-2_32.onnx', file_format = 'ONNX') +``` + +建议使用`.onnx`作为ONNX格式文件的后缀名。 + +### 导出MINDIR格式文件 + +MINDIR格式文件可用于MindSpore Lite端侧推理和MindSpore Serving推理服务。当前支持基于静态图,且不包含控制流语义的推理网络导出。导出该格式文件的代码样例如下: + ```python from mindspore.train.serialization import export import numpy as np @@ -173,5 +203,5 @@ load_param_into_net(resnet, param_dict) input = np.random.uniform(0.0, 1.0, size = [32, 3, 224, 224]).astype(np.float32) export(resnet, Tensor(input), file_name = 'resnet50-2_32.mindir', file_format = 'MINDIR') ``` -使用`export`接口之前,需要先导入`mindspore.train.serialization`。 -`input`用来指定导出模型的输入shape以及数据类型。 + +建议使用`.mindir`作为MINDIR格式文件的后缀名。