提交 a7d0c053 编写于 作者: L Liangliang He

Merge branch 'update_docs' into 'master'

Update docs

See merge request !651
...@@ -4,9 +4,9 @@ ...@@ -4,9 +4,9 @@
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
[![Build Status](https://travis-ci.org/travis-ci/travis-web.svg?branch=master)](https://travis-ci.org/travis-ci/travis-web)
[![pipeline status](https://gitlab.com/llhe/mace/badges/master/pipeline.svg)](https://gitlab.com/llhe/mace/pipelines) [![pipeline status](https://gitlab.com/llhe/mace/badges/master/pipeline.svg)](https://gitlab.com/llhe/mace/pipelines)
[![doc build status](https://readthedocs.org/projects/mace/badge/?version=latest)](https://readthedocs.org/projects/mace/badge/?version=latest) [![doc build status](https://readthedocs.org/projects/mace/badge/?version=latest)](https://readthedocs.org/projects/mace/badge/?version=latest)
[![Build Status](https://travis-ci.org/travis-ci/travis-web.svg?branch=master)](https://travis-ci.org/travis-ci/travis-web)
[Documentation](https://mace.readthedocs.io) | [Documentation](https://mace.readthedocs.io) |
[FAQ](https://mace.readthedocs.io/en/latest/faq.html) | [FAQ](https://mace.readthedocs.io/en/latest/faq.html) |
......
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
import recommonmark.parser import recommonmark.parser
import sphinx_rtd_theme import sphinx_rtd_theme
project = u'Mobile AI Compute Engine (MACE)' project = u'MACE'
author = u'%s Developers' % project author = u'%s Developers' % project
copyright = u'2018, %s' % author copyright = u'2018, %s' % author
......
Environment Requirement Environment requirement
======================== ========================
MACE requires the following dependencies: MACE requires the following dependencies:
Necessary Dependencies: Required dependencies
------------------------ ---------------------
.. list-table:: .. list-table::
:header-rows: 1 :header-rows: 1
* - software * - Software
- version - Installation command
- install command - Tested version
* - bazel * - Python
- >= 0.13.0 -
- 2.7
* - Bazel
- `bazel installation guide <https://docs.bazel.build/versions/master/install.html>`__ - `bazel installation guide <https://docs.bazel.build/versions/master/install.html>`__
* - android-ndk - 0.13.0
- r15c/r16b * - CMake
- `NDK installation guide <https://developer.android.com/ndk/guides/setup#install>`__
* - adb
- >= 1.0.32
- apt-get install android-tools-adb
* - cmake
- >= 3.11.3
- apt-get install cmake - apt-get install cmake
* - numpy - >= 3.11.3
- >= 1.14.0 * - Jinja2
- pip install -I numpy==1.14.0
* - scipy
- >= 1.0.0
- pip install -I scipy==1.0.0
* - jinja2
- >= 2.10
- pip install -I jinja2==2.10 - pip install -I jinja2==2.10
- 2.10
* - PyYaml * - PyYaml
- >= 3.12.0
- pip install -I pyyaml==3.12 - pip install -I pyyaml==3.12
- 3.12.0
* - sh * - sh
- >= 1.12.14
- pip install -I sh==1.12.14 - pip install -I sh==1.12.14
* - filelock - 1.12.14
- >= 3.0.0
- pip install -I filelock==3.0.0
.. note::
``export ANDROID_NDK_HOME=/path/to/ndk`` to specify ANDROID_NDK_HOME Optional dependencies
---------------------
Optional Dependencies:
-----------------------
.. list-table:: .. list-table::
:header-rows: 1 :header-rows: 1
* - software * - Software
- version - Installation command
- install command - Remark
* - tensorflow * - Android NDK
- >= 1.6.0 - `NDK installation guide <https://developer.android.com/ndk/guides/setup#install>`__
- pip install -I tensorflow==1.6.0 (if you use tensorflow model) - Required by Android build, r15b, r15c, r16b
* - docker (for caffe) * - ADB
- >= 17.09.0-ce - apt-get install android-tools-adb
- Required by Android run, >= 1.0.32
* - TensorFlow
- pip install -I tensorflow==1.6.0
- Required by TensorFlow model
* - Docker
- `docker installation guide <https://docs.docker.com/install/linux/docker-ce/ubuntu/#set-up-the-repository>`__ - `docker installation guide <https://docs.docker.com/install/linux/docker-ce/ubuntu/#set-up-the-repository>`__
- Required by docker mode for Caffe model
* - Numpy
- pip install -I numpy==1.14.0
- Required by model validation
* - Scipy
- pip install -I scipy==1.0.0
- Required by model validation
* - FileLock
- pip install -I filelock==3.0.0
- Required by Android run
.. note::
For Android build, `ANDROID_NDK_HOME` must be confifigured by using ``export ANDROID_NDK_HOME=/path/to/ndk``
Manual setup Manual setup
============= =============
The setup steps are based on ``Ubuntu``. And dependencies to install can refer to :doc:`env_requirement`. The setup steps are based on ``Ubuntu``, you can change the commands
correspondingly for other systems.
Install Necessary Dependencies For the detailed installation dependencies, please refer to :doc:`env_requirement`.
-------------------------------
Install Bazel Install Bazel
~~~~~~~~~~~~~~ -------------
Recommend bazel with version larger than ``0.13.0`` (Refer to `Bazel documentation <https://docs.bazel.build/versions/master/install.html>`__). Recommend bazel with version larger than ``0.13.0`` (Refer to `Bazel documentation <https://docs.bazel.build/versions/master/install.html>`__).
...@@ -22,10 +21,11 @@ Recommend bazel with version larger than ``0.13.0`` (Refer to `Bazel documentati ...@@ -22,10 +21,11 @@ Recommend bazel with version larger than ``0.13.0`` (Refer to `Bazel documentati
cd / && \ cd / && \
rm -f /bazel/bazel-$BAZEL_VERSION-installer-linux-x86_64.sh rm -f /bazel/bazel-$BAZEL_VERSION-installer-linux-x86_64.sh
Install NDK Install Android NDK
~~~~~~~~~~~~ --------------------
Recommend NDK with version r15c or r16 (Refer to `NDK installation guide <https://developer.android.com/ndk/guides/setup#install>`__). The recommended Android NDK versions includes r15b, r15c and r16b (Refers to
`NDK installation guide <https://developer.android.com/ndk/guides/setup#install>`__).
.. code:: sh .. code:: sh
...@@ -43,7 +43,7 @@ Recommend NDK with version r15c or r16 (Refer to `NDK installation guide <https: ...@@ -43,7 +43,7 @@ Recommend NDK with version r15c or r16 (Refer to `NDK installation guide <https:
export PATH=${PATH}:${ANDROID_NDK_HOME} export PATH=${PATH}:${ANDROID_NDK_HOME}
Install extra tools Install extra tools
~~~~~~~~~~~~~~~~~~~~ --------------------
.. code:: sh .. code:: sh
...@@ -60,8 +60,8 @@ Install extra tools ...@@ -60,8 +60,8 @@ Install extra tools
pycodestyle==2.4.0 \ pycodestyle==2.4.0 \
filelock filelock
Install Optional Dependencies Install TensorFlow (Optional)
------------------------------ -----------------------------
.. code:: sh .. code:: sh
......
Using docker Using docker
============= =============
Pull or Build docker image Pull or build docker image
--------------------------- ---------------------------
MACE provides docker images with necessary dependencies installed and also Dockerfiles for images building, MACE provides docker images with dependencies installed and also Dockerfiles for images building,
you can pull the existing ones directly or build them from the Dockerfiles. you can pull the existing ones directly or build them from the Dockerfiles.
In most cases, the ``lite edition`` image can satify developer's basic needs. In most cases, the ``lite edition`` image can satisfy developer's basic needs.
.. note:: .. note::
It's highly recommended to pull built images. It's highly recommended to pull built images.
......
Introduction Introduction
============ ============
Mobile AI Compute Engine (MACE) is a deep learning inference framework optimized for MACE (Mobile AI Compute Engine) is a deep learning inference framework optimized for
mobile heterogeneous computing platforms. MACE covers common mobile computing devices (CPU, GPU and DSP), mobile heterogeneous computing platforms.
and supplies tools and document to help users to deploy neural network model to mobile devices. MACE provides tools and documents to help users to deploy deep learning models
MACE has been widely used in Xiaomi and proved with industry leading performance and stability. to mobile phones, tablets, personal computers and IoT devices.
Framework Architecture
--------- -------------
The following figure shows the overall architecture. The following figure shows the overall architecture.
.. image:: mace-arch.png .. image:: mace-arch.png
:scale: 40 % :scale: 40 %
:align: center :align: center
==========
MACE Model MACE Model
========== ~~~~~~~~~~
MACE defines a customized model format which is similar to MACE defines a customized model format which is similar to
Caffe2. The MACE model can be converted from exported models by TensorFlow Caffe2. The MACE model can be converted from exported models by TensorFlow
and Caffe. and Caffe.
================
MACE Interpreter MACE Interpreter
================ ~~~~~~~~~~~~~~~~~
Mace Interpreter mainly parses the NN graph and manages the tensors in the graph. Mace Interpreter mainly parses the NN graph and manages the tensors in the graph.
=======
Runtime Runtime
======= ~~~~~~~
CPU/GPU/DSP runtime correspond to the Ops for different devices. CPU/GPU/DSP runtime correspond to the Ops for different devices.
...@@ -42,36 +39,31 @@ The following figure shows the basic work flow of MACE. ...@@ -42,36 +39,31 @@ The following figure shows the basic work flow of MACE.
:scale: 60 % :scale: 60 %
:align: center :align: center
==================================
1. Configure model deployment file 1. Configure model deployment file
================================== ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Model deploy configuration file (.yml) describe the information of the model and library, Model deploy configuration file (.yml) describes the information of the model and library,
MACE will build the library based on the file. MACE will build the library based on the file.
==================
2. Build libraries 2. Build libraries
================== ~~~~~~~~~~~~~~~~~~
Build MACE dynamic or static libraries. Build MACE dynamic or static libraries.
==================
3. Convert model 3. Convert model
================== ~~~~~~~~~~~~~~~~~~
Convert TensorFlow or Caffe model to MACE model. Convert TensorFlow or Caffe model to MACE model.
===========
4.1. Deploy 4.1. Deploy
=========== ~~~~~~~~~~~~~~~~~~
Integrate the MACE library to your application and run with MACE API. Integrate the MACE library into your application and run with MACE API.
==============
4.2. Run (CLI) 4.2. Run (CLI)
============== ~~~~~~~~~~~~~~~~~~
There are command line tools to run models, which could be used for testing time, memory usage and correctness. MACE provides `mace_run` command line tool, which could be used to run model
and validate model correctness against original TensorFlow or Caffe results.
==============
4.3. Benchmark 4.3. Benchmark
============== ~~~~~~~~~~~~~~~~~~
MACE supplies Benchmark tool to look up the run time of every Operation in the model. MACE provides benchmark tool to get the Op level profiling result of the model.
...@@ -84,29 +76,26 @@ MACE覆盖了常见的移动端计算设备(CPU,GPU和DSP),并且提供 ...@@ -84,29 +76,26 @@ MACE覆盖了常见的移动端计算设备(CPU,GPU和DSP),并且提供
框架 框架
---- ----
下图描述了MACE的基本框架. 下图描述了MACE的基本框架
.. image:: mace-arch.png .. image:: mace-arch.png
:scale: 60 % :scale: 40 %
:align: center :align: center
==============
MACE Model MACE Model
============== ~~~~~~~~~~~~~~~~~~
MACE定义了自有的模型格式(类似于Caffe2),通过MACE提供的工具可以将Caffe和TensorFlow的模型 MACE定义了自有的模型格式(类似于Caffe2),通过MACE提供的工具可以将Caffe和TensorFlow的模型
转为MACE模型。 转为MACE模型。
=================
MACE Interpreter MACE Interpreter
================= ~~~~~~~~~~~~~~~~~~
MACE Interpreter主要负责解析运行神经网络图(DAG)并管理网络中的Tensors。 MACE Interpreter主要负责解析运行神经网络图(DAG)并管理网络中的Tensors。
=======
Runtime Runtime
======= ~~~~~~~~~~~~~~~~~~
CPU/GPU/DSP Runtime对应于各个计算设备的算子实现。 CPU/GPU/DSP Runtime对应于各个计算设备的算子实现。
...@@ -119,33 +108,27 @@ CPU/GPU/DSP Runtime对应于各个计算设备的算子实现。 ...@@ -119,33 +108,27 @@ CPU/GPU/DSP Runtime对应于各个计算设备的算子实现。
:align: center :align: center
==================================
1. 配置模型部署文件(.yml) 1. 配置模型部署文件(.yml)
================================== ~~~~~~~~~~~~~~~~~~~~~~~~~~
模型部署文件详细描述了需要部署的模型以及生成库的信息,MACE根据该文件最终生成对应的库文件。 模型部署文件详细描述了需要部署的模型以及生成库的信息,MACE根据该文件最终生成对应的库文件。
==================================
2. 编译MACE库 2. 编译MACE库
================================== ~~~~~~~~~~~~~~~~~~
编译MACE的静态库或者动态库。 编译MACE的静态库或者动态库。
==================
3. 转换模型 3. 转换模型
================== ~~~~~~~~~~~~~~~~~~
将TensorFlow 或者 Caffe的模型转为MACE的模型. 将TensorFlow 或者 Caffe的模型转为MACE的模型
==================================
4.1. 部署 4.1. 部署
================================== ~~~~~~~~~~~~~~~~~~
根据不同使用目的集成Build阶段生成的库文件,然后调用MACE相应的接口执行模型。 根据不同使用目的集成Build阶段生成的库文件,然后调用MACE相应的接口执行模型。
==================================
4.2. 命令行运行 4.2. 命令行运行
================================== ~~~~~~~~~~~~~~~~~~
MACE提供了命令行工具,可以在命令行运行模型,可以用来测试模型运行时间,内存占用和正确性。 MACE提供了命令行工具,可以在命令行运行模型,可以用来测试模型运行时间,内存占用和正确性。
==================================
4.3. Benchmark 4.3. Benchmark
================================== ~~~~~~~~~~~~~~~~~~
MACE提供了命令行benchmark工具,可以细粒度的查看模型中所涉及的所有算子的运行时间。 MACE提供了命令行benchmark工具,可以细粒度的查看模型中所涉及的所有算子的运行时间。
===============
Advanced usage Advanced usage
=============== ===============
This part contains the full usage of MACE. This part contains the full usage of MACE.
---------
Overview Overview
--------- ---------
As mentioned in the previous part, a model deployment file defines a case of model deployment. As mentioned in the previous part, a model deployment file defines a case of model deployment.
The whole building process is loading a deployment file, converting models, building MACE and packing generated files. The building process includes parsing model deployment file, converting models,
building MACE core library and packing generated model libraries.
----------------
Deployment file Deployment file
---------------- ---------------
One deployment file will generate one library normally, but if more than one ABIs are specified, One deployment file will generate one library normally, but if more than one ABIs are specified,
...@@ -24,7 +22,8 @@ in one deployment file. ...@@ -24,7 +22,8 @@ in one deployment file.
* **Example** * **Example**
Here is an example deployment file with two models. Here is an example deployment file with two models used by
`MACE Android demo application <https://github.com/XiaoMi/mace/blob/master/mace/examples/android>`__.
.. literalinclude:: models/demo_app_models.yml .. literalinclude:: models/demo_app_models.yml
:language: yaml :language: yaml
...@@ -84,8 +83,6 @@ in one deployment file. ...@@ -84,8 +83,6 @@ in one deployment file.
- [optional] The data type used for specified runtime. [fp16_fp32, fp32_fp32] for GPU, default is fp16_fp32, [fp32] for CPU and [uint8] for DSP. - [optional] The data type used for specified runtime. [fp16_fp32, fp32_fp32] for GPU, default is fp16_fp32, [fp32] for CPU and [uint8] for DSP.
* - limit_opencl_kernel_time * - limit_opencl_kernel_time
- [optional] Whether splitting the OpenCL kernel within 1 ms to keep UI responsiveness, default is 0. - [optional] Whether splitting the OpenCL kernel within 1 ms to keep UI responsiveness, default is 0.
* - nnlib_graph_mode
- [optional] Control the DSP precision and performance, default to 0 usually works for most cases.
* - obfuscate * - obfuscate
- [optional] Whether to obfuscate the model operator name, default to 0. - [optional] Whether to obfuscate the model operator name, default to 0.
* - winograd * - winograd
...@@ -105,13 +102,14 @@ in one deployment file. ...@@ -105,13 +102,14 @@ in one deployment file.
sha256sum /path/to/your/file sha256sum /path/to/your/file
--------------- Advanced usage
Advanced Usage --------------
---------------
There are two common advanced use cases: 1. convert a model to CPP code. 2. tuning for specific SOC if use GPU. There are two common advanced use cases:
- converting model to C++ code.
- tuning GPU kernels for a specific SoC.
* **Convert model(s) to CPP code** * **Convert model(s) to C++ code**
.. warning:: .. warning::
...@@ -119,7 +117,7 @@ There are two common advanced use cases: 1. convert a model to CPP code. 2. tuni ...@@ -119,7 +117,7 @@ There are two common advanced use cases: 1. convert a model to CPP code. 2. tuni
* **1. Change the model deployment file(.yml)** * **1. Change the model deployment file(.yml)**
If you want to protect your model, you can convert model to CPP code. there are also two cases: If you want to protect your model, you can convert model to C++ code. there are also two cases:
* convert model graph to code and model weight to file with below model configuration. * convert model graph to code and model weight to file with below model configuration.
...@@ -197,10 +195,10 @@ There are two common advanced use cases: 1. convert a model to CPP code. 2. tuni ...@@ -197,10 +195,10 @@ There are two common advanced use cases: 1. convert a model to CPP code. 2. tuni
// ... Same with the code in basic usage // ... Same with the code in basic usage
* **Tuning for specific SOC's GPU** * **Tuning for specific SoC's GPU**
If you want to use the GPU of a specific device, you can just specify the ``target_socs`` in your YAML file and If you want to use the GPU of a specific device, you can just specify the ``target_socs`` in your YAML file and
then tune the MACE lib for it, which may get 1~10% performance improvement. then tune the MACE lib for it (OpenCL kernels), which may get 1~10% performance improvement.
* **1. Change the model deployment file(.yml)** * **1. Change the model deployment file(.yml)**
...@@ -253,7 +251,7 @@ There are two common advanced use cases: 1. convert a model to CPP code. 2. tuni ...@@ -253,7 +251,7 @@ There are two common advanced use cases: 1. convert a model to CPP code. 2. tuni
used for your models, which could accelerate the initialization stage. used for your models, which could accelerate the initialization stage.
Details please refer to `OpenCL Specification <https://www.khronos.org/registry/OpenCL/sdk/1.0/docs/man/xhtml/clCreateProgramWithBinary.html>`__. Details please refer to `OpenCL Specification <https://www.khronos.org/registry/OpenCL/sdk/1.0/docs/man/xhtml/clCreateProgramWithBinary.html>`__.
* **mobilenet-v2-tuned_opencl_parameter.MI6.msm8998.bin** stands for the tuned OpenCL parameters * **mobilenet-v2-tuned_opencl_parameter.MI6.msm8998.bin** stands for the tuned OpenCL parameters
for the SOC. for the SoC.
* **4. Deployment** * **4. Deployment**
* Change the names of files generated above for not collision and push them to **your own device's directory**. * Change the names of files generated above for not collision and push them to **your own device's directory**.
...@@ -274,9 +272,8 @@ There are two common advanced use cases: 1. convert a model to CPP code. 2. tuni ...@@ -274,9 +272,8 @@ There are two common advanced use cases: 1. convert a model to CPP code. 2. tuni
// ... Same with the code in basic usage. // ... Same with the code in basic usage.
----------------
Useful Commands Useful Commands
---------------- ---------------
* **run the model** * **run the model**
.. code:: sh .. code:: sh
......
...@@ -7,13 +7,14 @@ Build and run an example model ...@@ -7,13 +7,14 @@ Build and run an example model
At first, make sure the environment has been set up correctly already (refer to :doc:`../installation/env_requirement`). At first, make sure the environment has been set up correctly already (refer to :doc:`../installation/env_requirement`).
The followings are instructions about how to quickly build and run a provided model in *MACE Model Zoo*. The followings are instructions about how to quickly build and run a provided model in
`MACE Model Zoo <https://github.com/XiaoMi/mace-models>`__.
Here we use the mobilenet-v2 model as an example. Here we use the mobilenet-v2 model as an example.
**Commands** **Commands**
1. Pull *MACE* project. 1. Pull `MACE <https://github.com/XiaoMi/mace>`__ project.
.. code:: sh .. code:: sh
...@@ -29,14 +30,14 @@ Here we use the mobilenet-v2 model as an example. ...@@ -29,14 +30,14 @@ Here we use the mobilenet-v2 model as an example.
It's highly recommanded to use a release version instead of master branch. It's highly recommanded to use a release version instead of master branch.
2. Pull *MACE Model Zoo* project. 2. Pull `MACE Model Zoo <https://github.com/XiaoMi/mace-models>`__ project.
.. code:: sh .. code:: sh
git clone https://github.com/XiaoMi/mace-models.git git clone https://github.com/XiaoMi/mace-models.git
3. Build a general MACE library. 3. Build a generic MACE library.
.. code:: sh .. code:: sh
...@@ -46,7 +47,7 @@ Here we use the mobilenet-v2 model as an example. ...@@ -46,7 +47,7 @@ Here we use the mobilenet-v2 model as an example.
bash tools/build-standalone-lib.sh bash tools/build-standalone-lib.sh
4. Convert the model to MACE format model. 4. Convert the pre-trained mobilenet-v2 model to MACE format model.
.. code:: sh .. code:: sh
...@@ -57,7 +58,7 @@ Here we use the mobilenet-v2 model as an example. ...@@ -57,7 +58,7 @@ Here we use the mobilenet-v2 model as an example.
5. Run the model. 5. Run the model.
.. warning:: .. note::
If you want to run on device/phone, please plug in at least one device/phone. If you want to run on device/phone, please plug in at least one device/phone.
...@@ -77,13 +78,13 @@ Here we use the mobilenet-v2 model as an example. ...@@ -77,13 +78,13 @@ Here we use the mobilenet-v2 model as an example.
Build your own model Build your own model
--------------------- ---------------------
This part will show you how to use your pre-trained model in MACE. This part will show you how to use your own pre-trained model in MACE.
====================== ======================
1. Prepare your model 1. Prepare your model
====================== ======================
Mace now supports models from TensorFlow and Caffe (more frameworks will be supported). MACE now supports models from TensorFlow and Caffe (more frameworks will be supported).
- TensorFlow - TensorFlow
...@@ -338,4 +339,4 @@ Please refer to \ ``mace/examples/example.cc``\ for full usage. The following li ...@@ -338,4 +339,4 @@ Please refer to \ ``mace/examples/example.cc``\ for full usage. The following li
// 6. Run the model // 6. Run the model
MaceStatus status = engine.Run(inputs, &outputs); MaceStatus status = engine.Run(inputs, &outputs);
More details are in :doc:`advanced_usage`. More details are in :doc:`advanced_usage`.
\ No newline at end of file
...@@ -26,7 +26,6 @@ models: ...@@ -26,7 +26,6 @@ models:
- https://cnbj1.fds.api.xiaomi.com/mace/inputs/dog.npy - https://cnbj1.fds.api.xiaomi.com/mace/inputs/dog.npy
runtime: cpu+gpu runtime: cpu+gpu
limit_opencl_kernel_time: 0 limit_opencl_kernel_time: 0
nnlib_graph_mode: 0
obfuscate: 0 obfuscate: 0
winograd: 0 winograd: 0
squeezenet_v11: squeezenet_v11:
...@@ -46,6 +45,5 @@ models: ...@@ -46,6 +45,5 @@ models:
- 1,1,1,1000 - 1,1,1,1000
runtime: cpu+gpu runtime: cpu+gpu
limit_opencl_kernel_time: 0 limit_opencl_kernel_time: 0
nnlib_graph_mode: 0
obfuscate: 0 obfuscate: 0
winograd: 0 winograd: 0
\ No newline at end of file
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册