Generally, the compiled output files include the following types. The architecture selection affects the types of output files.
> For the x86 architecture, you can obtain the output of the conversion tool; for the Arm 64-bit architecture, you can obtain the output of the `arm64-cpu` inference framework. If `-e gpu` is added, you can obtain the output of the `arm64-cpu` inference framework. The compilation for arm 64-bit is the same as that for arm 32-bit.
For the x86 architecture, you can obtain the output of the conversion tool and inference framework；But for the ARM architecture, you only get inference framework.
| Directory | Description | x86_64 | Arm 64-bit | Arm 32-bit |
| --- | --- | --- | --- | --- |
| include | Inference framework header file | No | Yes | Yes |
| time_profiler | Time consumption analysis tool at the model network layer | Yes | Yes | Yes |
| converter | Model conversion tool | Yes | No | No |
| third_party | Header file and library of the third-party library | Yes | Yes | Yes |
Generally, the compiled output files include the following types. The architecture selection affects the types of output files.
The contents of `third party` vary depending on the architecture as follows:
- x86_64: `protobuf` (Protobuf dynamic library).
- arm: `flatbuffers` (FlatBuffers header file).
> For the Arm 64-bit architecture, you can obtain the output of the `arm64-cpu` inference framework. If `-e gpu` is added, you can obtain the output of the `arm64-gpu` inference framework. The compilation for arm 64-bit is the same as that for arm 32-bit.
| Directory | Description | converter | runtime |
| --- | --- | --- | --- |
| include | Inference framework header file | No | Yes |
| time_profiler | Time consumption analysis tool at the model network layer| No | Yes |
| converter | Model conversion tool | Yes | No | No |
| third_party | Header file and library of the third-party library | Yes | Yes |
Take the 0.7.0-beta version and CPU as an example. The contents of `third party` and `lib` vary depending on the architecture as follows:
-`mindspore-lite-0.7.0-converter-ubuntu`: include `protobuf` (Protobuf dynamic library).
-`mindspore-lite-0.7.0-runtime-x86-cpu`: include `flatbuffers` (FlatBuffers header file).
TODO: Add document content.
> Before running the tools in the `converter`, `benchmark`, or `time_profiler` directory, you need to configure environment variables and set the paths of the dynamic libraries of MindSpore Lite and Protobuf to the paths of the system dynamic libraries. The following uses the 0.7.0-beta version as an example: `export LD_LIBRARY_PATH=./mindspore-lite-0.7.0/lib:./mindspore-lite-0.7.0/third_party/protobuf/lib:${LD_LIBRARY_PATH}`.
MS_LOG(ERROR)<<"New context failed while running %s",modelName.c_str();
returnRET_ERROR;
}
// The preferred backend is GPU, which means, if there is a GPU operator, it will run on the GPU first, otherwise it will run on the CPU.
context->device_ctx_.type=lite::DT_GPU;
// The medium core takes priority in thread and core binding methods. This parameter will work in the BindThread interface. For specific binding effect, see the "Run Graph" section.
context->cpu_bind_mode_=MID_CPU;
// Configure the number of worker threads in the thread pool to 2, including the main thread.
context->thread_num_=2;
// Allocators can be shared across multiple Contexts.
// Assume we have created a LiteSession instance named session.
autoinputs=session->GetInputs();
// Assume that the model has only one input tensor.
autoin_tensor=inputs.front();
if(in_tensor==nullptr){
std::cerr<<"Input tensor is nullptr"<<std::endl;
return-1;
}
// It is omitted that users have read the model input file and generated a section of memory buffer: input_buf, as well as the byte size of input_buf: data_size.
if(in_tensor->Size()!=data_size){
std::cerr<<"Input data size is not suit for model input"<<std::endl;
return-1;
}
auto*in_data=in_tensor->MutableData();
if(in_data==nullptr){
std::cerr<<"Data of in_tensor is nullptr"<<std::endl;
return-1;
}
memcpy(in_data,input_buf,data_size);
// Users need to free input_buf.
// The elements in the inputs are managed by MindSpore Lite so that users do not need to free inputs.
// Assume we have created a LiteSession instance named session and a Model instance named model before.
// The methods of creating model and session can refer to "Import Model" and "Create Session" two sections.
autoret=session->CompileGraph(model);
if(ret!=RET_OK){
std::cerr<<"CompileGraph failed"<<std::endl;
// session and model need to be released by users manually.
delete(session);
delete(model);
returnret;
}
// Copy input data into the input tensor. Users can refer to the "Input Data" section. We uses random data here.
autoinputs=session->GetInputs();
for(autoin_tensor:inputs){
in_tensor=inputs.front();
if(in_tensor==nullptr){
std::cerr<<"Input tensor is nullptr"<<std::endl;
return-1;
}
// When calling the MutableData method, if the data in MSTensor is not allocated, it will be malloced. After allocation, the data in MSTensor can be considered as random data.
(void)in_tensor->MutableData();
}
// Definition of callback function before forwarding operator.
