Provides APIs related to MindSpore Lite model inference.
Copyright 2021 Huawei Technologies Co., Ltd
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
\@Syscap SystemCapability.Ai.MindSpore
**Since:**
9
## Summary
### Files
| Name | Description |
| -------- | -------- |
| [context.h](context_8h.md) | Provides **Context** APIs for configuring runtime information. <br>File to Include: <mindspore/context.h> |
| [data_type.h](data__type_8h.md) | Declares tensor data types. <br>File to Include: <mindspore/data_type.h> |
| [format.h](format_8h.md) | Declares tensor data formats. <br>File to Include: <mindspore/format.h> |
| [model.h](model_8h.md) | Provides model-related APIs for model creation and inference. <br>File to Include: <mindspore/model.h> |
| [status.h](status_8h.md) | Provides the status codes of MindSpore Lite. <br>File to Include: <mindspore/status.h> |
| [tensor.h](tensor_8h.md) | Provides APIs for creating and modifying tensor information. <br>File to Include: <mindspore/tensor.h> |
| [types.h](types_8h.md) | Provides the model file types and device types supported by MindSpore Lite. <br>File to Include: <mindspore/types.h> |
### Structs
| Name | Description |
| -------- | -------- |
| [OH_AI_TensorHandleArray](_o_h___a_i___tensor_handle_array.md) | Defines the tensor array structure, which is used to store the tensor array pointer and tensor array length. |
| [OH_AI_ShapeInfo](_o_h___a_i___shape_info.md) | Defines dimension information. The maximum dimension is set by **MS_MAX_SHAPE_NUM**. |
| [OH_AI_CallBackParam](_o_h___a_i___call_back_param.md) | Defines the operator information passed in a callback. |
### Macros
| Name | Description |
| -------- | -------- |
| [OH_AI_MAX_SHAPE_NUM](#oh_ai_max_shape_num) 32 | Defines dimension information. The maximum dimension is set by **MS_MAX_SHAPE_NUM**. |
### Types
| Name | Description |
| -------- | -------- |
| [OH_AI_ContextHandle](#oh_ai_contexthandle) | Defines the pointer to the MindSpore context. |
| [OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) | Defines the pointer to the MindSpore device information. |
| [OH_AI_DataType](#oh_ai_datatype) | Declares data types supported by MSTensor. |
| [OH_AI_Format](#oh_ai_format) | Declares data formats supported by MSTensor. |
| [OH_AI_ModelHandle](#oh_ai_modelhandle) | Defines the pointer to a model object. |
| [OH_AI_TensorHandleArray](#oh_ai_tensorhandlearray) | Defines the tensor array structure, which is used to store the tensor array pointer and tensor array length. |
| **OH_AI_ShapeInfo** | Defines dimension information. The maximum dimension is set by **MS_MAX_SHAPE_NUM**. |
| [OH_AI_CallBackParam](#oh_ai_callbackparam) | Defines the operator information passed in a callback. |
| [OH_AI_KernelCallBack](#oh_ai_kernelcallback)) (const [OH_AI_TensorHandleArray](_o_h___a_i___tensor_handle_array.md) inputs, const [OH_AI_TensorHandleArray](_o_h___a_i___tensor_handle_array.md) outputs, const [OH_AI_CallBackParam](_o_h___a_i___call_back_param.md) kernel_Info) | Defines the pointer to a callback. |
| [OH_AI_Status](#oh_ai_status) | Defines MindSpore status codes. |
| [OH_AI_TensorHandle](#oh_ai_tensorhandle) | Defines the handle of a tensor object. |
| [OH_AI_ModelType](#oh_ai_modeltype) | Defines model file types. |
| [OH_AI_DeviceType](#oh_ai_devicetype) | Defines the supported device types. |
| [OH_AI_ContextCreate](#oh_ai_contextcreate)() | Creates a context object. |
| [OH_AI_ContextDestroy](#oh_ai_contextdestroy)([OH_AI_ContextHandle](#oh_ai_contexthandle)\*context) | Destroys a context object. |
| [OH_AI_ContextSetThreadNum](#oh_ai_contextsetthreadnum)([OH_AI_ContextHandle](#oh_ai_contexthandle) context, int32_t thread_num) | Sets the number of runtime threads. |
| [OH_AI_ContextGetThreadNum](#oh_ai_contextgetthreadnum)(const[OH_AI_ContextHandle](#oh_ai_contexthandle) context) | Obtains the number of threads. |
| [OH_AI_ContextSetThreadAffinityMode](#oh_ai_contextsetthreadaffinitymode)([OH_AI_ContextHandle](#oh_ai_contexthandle) context, int mode) | Sets the affinity mode for binding runtime threads to CPU cores, which are categorized into little cores and big cores depending on the CPU frequency. |
| [OH_AI_ContextGetThreadAffinityMode](#oh_ai_contextgetthreadaffinitymode)(const[OH_AI_ContextHandle](#oh_ai_contexthandle) context) | Obtains the affinity mode for binding runtime threads to CPU cores. |
| [OH_AI_ContextSetThreadAffinityCoreList](#oh_ai_contextsetthreadaffinitycorelist)([OH_AI_ContextHandle](#oh_ai_contexthandle) context, const int32_t \*core_list, size_t core_num) | Sets the list of CPU cores bound to a runtime thread. |
| [OH_AI_ContextGetThreadAffinityCoreList](#oh_ai_contextgetthreadaffinitycorelist)(const[OH_AI_ContextHandle](#oh_ai_contexthandle) context, size_t \*core_num) | Obtains the list of bound CPU cores. |
| [OH_AI_ContextSetEnableParallel](#oh_ai_contextsetenableparallel)([OH_AI_ContextHandle](#oh_ai_contexthandle) context, bool is_parallel) | Sets whether to enable parallelism between operators. |
| [OH_AI_ContextGetEnableParallel](#oh_ai_contextgetenableparallel)(const[OH_AI_ContextHandle](#oh_ai_contexthandle) context) | Checks whether parallelism between operators is supported. |
| [OH_AI_ContextAddDeviceInfo](#oh_ai_contextadddeviceinfo)([OH_AI_ContextHandle](#oh_ai_contexthandle) context, [OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) device_info) | Adds information about a running device. |
| [OH_AI_DeviceInfoCreate](#oh_ai_deviceinfocreate)([OH_AI_DeviceType](#oh_ai_devicetype) device_type) | Creates a device information object. |
| [OH_AI_DeviceInfoDestroy](#oh_ai_deviceinfodestroy)([OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle)\*device_info) | Destroys a device information instance. |
| [OH_AI_DeviceInfoSetProvider](#oh_ai_deviceinfosetprovider)([OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) device_info, const char \*provider) | Sets the name of a provider. |
| [OH_AI_DeviceInfoGetProvider](#oh_ai_deviceinfogetprovider)(const[OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) device_info) | Obtains the provider name. |
| [OH_AI_DeviceInfoSetProviderDevice](#oh_ai_deviceinfosetproviderdevice)([OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) device_info, const char \*device) | Sets the name of a provider device. |
| [OH_AI_DeviceInfoGetProviderDevice](#oh_ai_deviceinfogetproviderdevice)(const[OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) device_info) | Obtains the name of a provider device. |
| [OH_AI_DeviceInfoGetDeviceType](#oh_ai_deviceinfogetdevicetype)(const[OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) device_info) | Obtains the type of a provider device. |
| [OH_AI_DeviceInfoSetEnableFP16](#oh_ai_deviceinfosetenablefp16)([OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) device_info, bool is_fp16) | Sets whether to enable float16 inference. This function is available only for CPU/GPU devices. |
| [OH_AI_DeviceInfoGetEnableFP16](#oh_ai_deviceinfogetenablefp16)(const[OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) device_info) | Checks whether float16 inference is enabled. This function is available only for CPU/GPU devices. |
| [OH_AI_DeviceInfoSetFrequency](#oh_ai_deviceinfosetfrequency)([OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) device_info, int frequency) | Sets the NPU frequency type. This function is available only for NPU devices. |
| [OH_AI_DeviceInfoGetFrequency](#oh_ai_deviceinfogetfrequency)(const[OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) device_info) | Obtains the NPU frequency type. This function is available only for NPU devices. |
| [OH_AI_ModelCreate](#oh_ai_modelcreate)() | Creates a model object. |
| [OH_AI_ModelDestroy](#oh_ai_modeldestroy)([OH_AI_ModelHandle](#oh_ai_modelhandle)\*model) | Destroys a model object. |
| [OH_AI_ModelBuild](#oh_ai_modelbuild)([OH_AI_ModelHandle](#oh_ai_modelhandle) model, const void \*model_data, size_t data_size, [OH_AI_ModelType](#oh_ai_modeltype) model_type, const [OH_AI_ContextHandle](#oh_ai_contexthandle) model_context) | Loads and builds a MindSpore model from the memory buffer. |
| [OH_AI_ModelBuildFromFile](#oh_ai_modelbuildfromfile)([OH_AI_ModelHandle](#oh_ai_modelhandle) model, const char \*model_path, [OH_AI_ModelType](#oh_ai_modeltype) model_type, const [OH_AI_ContextHandle](#oh_ai_contexthandle) model_context) | Loads and builds a MindSpore model from a model file. |
| [OH_AI_ModelResize](#oh_ai_modelresize)([OH_AI_ModelHandle](#oh_ai_modelhandle) model, const [OH_AI_TensorHandleArray](_o_h___a_i___tensor_handle_array.md) inputs, [OH_AI_ShapeInfo](_o_h___a_i___shape_info.md)\*shape_infos, size_t shape_info_num) | Adjusts the input tensor shapes of a built model. |
| [OH_AI_ModelGetInputs](#oh_ai_modelgetinputs)(const[OH_AI_ModelHandle](#oh_ai_modelhandle) model) | Obtains the input tensor array structure of a model. |
| [OH_AI_ModelGetOutputs](#oh_ai_modelgetoutputs)(const[OH_AI_ModelHandle](#oh_ai_modelhandle) model) | Obtains the output tensor array structure of a model. |
| [OH_AI_ModelGetInputByTensorName](#oh_ai_modelgetinputbytensorname)(const[OH_AI_ModelHandle](#oh_ai_modelhandle) model, const char \*tensor_name) | Obtains the input tensor of a model by tensor name. |
| [OH_AI_ModelGetOutputByTensorName](#oh_ai_modelgetoutputbytensorname)(const[OH_AI_ModelHandle](#oh_ai_modelhandle) model, const char \*tensor_name) | Obtains the output tensor of a model by tensor name. |
| [OH_AI_TensorDestroy](#oh_ai_tensordestroy)([OH_AI_TensorHandle](#oh_ai_tensorhandle)\*tensor) | Destroys a tensor object. |
| [OH_AI_TensorClone](#oh_ai_tensorclone)([OH_AI_TensorHandle](#oh_ai_tensorhandle) tensor) | Clones a tensor. |
| [OH_AI_TensorSetName](#oh_ai_tensorsetname)([OH_AI_TensorHandle](#oh_ai_tensorhandle) tensor, const char \*name) | Sets the name of a tensor. |
| [OH_AI_TensorGetName](#oh_ai_tensorgetname)(const[OH_AI_TensorHandle](#oh_ai_tensorhandle) tensor) | Obtains the name of a tensor. |
| [OH_AI_TensorSetDataType](#oh_ai_tensorsetdatatype)([OH_AI_TensorHandle](#oh_ai_tensorhandle) tensor, [OH_AI_DataType](#oh_ai_datatype) type) | Sets the data type of a tensor. |
| [OH_AI_TensorGetDataType](#oh_ai_tensorgetdatatype)(const[OH_AI_TensorHandle](#oh_ai_tensorhandle) tensor) | Obtains the data type of a tensor. |
| [OH_AI_TensorSetShape](#oh_ai_tensorsetshape)([OH_AI_TensorHandle](#oh_ai_tensorhandle) tensor, const int64_t \*shape, size_t shape_num) | Sets the shape of a tensor. |
| [OH_AI_TensorGetShape](#oh_ai_tensorgetshape)(const[OH_AI_TensorHandle](#oh_ai_tensorhandle) tensor, size_t \*shape_num) | Obtains the shape of a tensor. |
| [OH_AI_TensorSetFormat](#oh_ai_tensorsetformat)([OH_AI_TensorHandle](#oh_ai_tensorhandle) tensor, [OH_AI_Format](#oh_ai_format) format) | Sets the tensor data format. |
| [OH_AI_TensorGetFormat](#oh_ai_tensorgetformat)(const[OH_AI_TensorHandle](#oh_ai_tensorhandle) tensor) | Obtains the tensor data format. |
| [OH_AI_TensorGetData](#oh_ai_tensorgetdata)(const[OH_AI_TensorHandle](#oh_ai_tensorhandle) tensor) | Obtains the pointer to tensor data. |
| [OH_AI_TensorGetMutableData](#oh_ai_tensorgetmutabledata)(const[OH_AI_TensorHandle](#oh_ai_tensorhandle) tensor) | Obtains the pointer to variable tensor data. If the data is empty, memory will be allocated. |
| [OH_AI_TensorGetElementNum](#oh_ai_tensorgetelementnum)(const[OH_AI_TensorHandle](#oh_ai_tensorhandle) tensor) | Obtains the number of tensor elements. |
| [OH_AI_TensorGetDataSize](#oh_ai_tensorgetdatasize)(const[OH_AI_TensorHandle](#oh_ai_tensorhandle) tensor) | Obtains the number of bytes of the tensor data. |
## Macro Description
### OH_AI_MAX_SHAPE_NUM
```
#define OH_AI_MAX_SHAPE_NUM 32
```
**Description**<br>
Defines dimension information. The maximum dimension is set by **MS_MAX_SHAPE_NUM**.
This pointer is used to set the two callback functions in [OH_AI_ModelPredict](#oh_ai_modelpredict). Each callback function must contain three parameters, where **inputs** and **outputs** indicate the input and output tensors of the operator, and **kernel_Info** indicates information about the current operator. You can use the callback functions to monitor the operator execution status, for example, operator execution time and the operator correctness.
Checks whether parallelism between operators is supported.
**Parameters**
| Name | Description |
| -------- | -------- |
| context | [OH_AI_ContextHandle](#oh_ai_contexthandle) that points to the context instance. |
**Returns**
Whether parallelism between operators is supported. The value **true** means that parallelism between operators is supported, and the value **false** means the opposite.
Sets whether to enable parallelism between operators.
**Parameters**
| Name | Description |
| -------- | -------- |
| context | [OH_AI_ContextHandle](#oh_ai_contexthandle) that points to the context instance. |
| is_parallel | Whether to enable parallelism between operators. The value **true** means to enable parallelism between operators, and the value **false** means the opposite. |
Sets the list of CPU cores bound to a runtime thread.
For example, if **core_list** is set to **[2,6,8]**, threads run on the 2nd, 6th, and 8th CPU cores. If [OH_AI_ContextSetThreadAffinityMode](#oh_ai_contextsetthreadaffinitymode) and [OH_AI_ContextSetThreadAffinityCoreList](#oh_ai_contextsetthreadaffinitycorelist) are called for the same context object, the **core_list** parameter of [OH_AI_ContextSetThreadAffinityCoreList](#oh_ai_contextsetthreadaffinitycorelist) takes effect, but the **mode** parameter of [OH_AI_ContextSetThreadAffinityMode](#oh_ai_contextsetthreadaffinitymode) does not.
**Parameters**
| Name | Description |
| -------- | -------- |
| context | [OH_AI_ContextHandle](#oh_ai_contexthandle) that points to the context instance. |
| core_list | List of bound CPU cores. |
| core_num | Number of cores, which indicates the length of **core_list**. |
### OH_AI_ContextSetThreadAffinityMode()
```
OH_AI_API void OH_AI_ContextSetThreadAffinityMode (OH_AI_ContextHandle context, int mode )
```
**Description**<br>
Sets the affinity mode for binding runtime threads to CPU cores, which are categorized into little cores and big cores depending on the CPU frequency.
**Parameters**
| Name | Description |
| -------- | -------- |
| context | [OH_AI_ContextHandle](#oh_ai_contexthandle) that points to the context instance. |
| mode | Affinity mode. **0**: no affinities; **1**: big cores first; **2**: little cores first |
Checks whether float16 inference is enabled. This function is available only for CPU/GPU devices.
**Parameters**
| Name | Description |
| -------- | -------- |
| device_info | [OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) that points to a device information instance. |
**Returns**
Whether float16 inference is enabled.
### OH_AI_DeviceInfoGetFrequency()
```
OH_AI_API int OH_AI_DeviceInfoGetFrequency (const OH_AI_DeviceInfoHandle device_info)
```
**Description**<br>
Obtains the NPU frequency type. This function is available only for NPU devices.
**Parameters**
| Name | Description |
| -------- | -------- |
| device_info | [OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) that points to a device information instance. |
**Returns**
Frequency type of the NPU. The value ranges from **0** to **4**. **1**: low power consumption; **2**: balanced; **3**: high performance; **4**: ultra-high performance
Sets whether to enable float16 inference. This function is available only for CPU/GPU devices.
**Parameters**
| Name | Description |
| -------- | -------- |
| device_info | [OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) that points to a device information instance. |
| is_fp16 | Whether to enable float16 inference. |
### OH_AI_DeviceInfoSetFrequency()
```
OH_AI_API void OH_AI_DeviceInfoSetFrequency (OH_AI_DeviceInfoHandle device_info, int frequency )
```
**Description**<br>
Sets the NPU frequency type. This function is available only for NPU devices.
**Parameters**
| Name | Description |
| -------- | -------- |
| device_info | [OH_AI_DeviceInfoHandle](#oh_ai_deviceinfohandle) that points to a device information instance. |
| frequency | NPU frequency type. The value ranges from **0** to **4**. The default value is **3**. **1**: low power consumption; **2**: balanced; **3**: high performance; **4**: ultra-high performance |
Loads and builds a MindSpore model from the memory buffer.
Note that the same {\@Link OH_AI_ContextHandle} object can only be passed to {\@Link OH_AI_ModelBuild} or {\@Link OH_AI_ModelBuildFromFile} once. If you call this function multiple times, make sure that you create multiple {\@Link OH_AI_ContextHandle} objects accordingly.
**Parameters**
| Name | Description |
| -------- | -------- |
| model | Pointer to the model object. |
| model_data | Address of the loaded model data in the memory. |
| data_size | Length of the model data. |
| model_type | Type of the model file. For details, see [OH_AI_ModelType](#oh_ai_modeltype). |
| model_context | Context for model running. For details, see [OH_AI_ContextHandle](#oh_ai_contexthandle). |
**Returns**
Status code enumerated by [OH_AI_Status](#oh_ai_status). The value **MSStatus::kMSStatusSuccess** indicates that the operation is successful.
Loads and builds a MindSpore model from a model file.
Note that the same {\@Link OH_AI_ContextHandle} object can only be passed to {\@Link OH_AI_ModelBuild} or {\@Link OH_AI_ModelBuildFromFile} once. If you call this function multiple times, make sure that you create multiple {\@Link OH_AI_ContextHandle} objects accordingly.
**Parameters**
| Name | Description |
| -------- | -------- |
| model | Pointer to the model object. |
| model_path | Path of the model file. |
| model_type | Type of the model file. For details, see [OH_AI_ModelType](#oh_ai_modeltype). |
| model_context | Context for model running. For details, see [OH_AI_ContextHandle](#oh_ai_contexthandle). |
**Returns**
Status code enumerated by [OH_AI_Status](#oh_ai_status). The value **MSStatus::kMSStatusSuccess** indicates that the operation is successful.
| inputs | Tensor array structure corresponding to the model input. |
| shape_infos | Input shape array, which consists of tensor shapes arranged in the model input sequence. The model adjusts the tensor shapes in sequence. |
| shape_info_num | Length of the input shape array. |
**Returns**
Status code enumerated by [OH_AI_Status](#oh_ai_status). The value **MSStatus::kMSStatusSuccess** indicates that the operation is successful.
