diff --git a/lite/docs/Makefile b/lite/docs/Makefile new file mode 100644 index 0000000000000000000000000000000000000000..1eff8952707bdfa503c8d60c1e9a903053170ba2 --- /dev/null +++ b/lite/docs/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = source_zh_cn +BUILDDIR = build_zh_cn + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/lite/docs/requirements.txt b/lite/docs/requirements.txt new file mode 100644 index 0000000000000000000000000000000000000000..8788e57530c50ff0656148a12b6d3a4480f4e430 --- /dev/null +++ b/lite/docs/requirements.txt @@ -0,0 +1,5 @@ +sphinx +recommonmark +sphinx-markdown-tables +sphinx_rtd_theme +numpy \ No newline at end of file diff --git a/lite/docs/source_en/_static/logo_source.png b/lite/docs/source_en/_static/logo_source.png new file mode 100644 index 0000000000000000000000000000000000000000..fc347d271abe082ae8d16242328551648766b6fb Binary files /dev/null and b/lite/docs/source_en/_static/logo_source.png differ diff --git a/lite/docs/source_en/architecture.md b/lite/docs/source_en/architecture.md new file mode 100644 index 0000000000000000000000000000000000000000..a70c96987d8df7c2724b51e83244928a3fdf055e --- /dev/null +++ b/lite/docs/source_en/architecture.md @@ -0,0 +1 @@ +# Overall Architecture diff --git a/lite/docs/source_en/conf.py b/lite/docs/source_en/conf.py new file mode 100644 index 0000000000000000000000000000000000000000..fd89055b9bf9e2a889d23de6fa7395072650db42 --- /dev/null +++ b/lite/docs/source_en/conf.py @@ -0,0 +1,60 @@ +# Configuration file for the Sphinx documentation builder. +# +# This file only contains a selection of the most common options. For a full +# list see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +import os + + +# -- Project information ----------------------------------------------------- + +project = 'MindSpore Lite' +copyright = '2020, MindSpore Lite' +author = 'MindSpore Lite' + +# The full version, including alpha/beta/rc tags +release = 'master' + + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + 'sphinx_markdown_tables', + 'recommonmark', +] + +source_suffix = { + '.rst': 'restructuredtext', + '.md': 'markdown', +} + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = [] + +pygments_style = 'sphinx' + +autodoc_inherit_docstrings = False + +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'sphinx_rtd_theme' + +html_static_path = ['_static'] \ No newline at end of file diff --git a/lite/docs/source_en/glossary.md b/lite/docs/source_en/glossary.md new file mode 100644 index 0000000000000000000000000000000000000000..ca3c8276922faf3e1b50bf298ec40d8507b9aff4 --- /dev/null +++ b/lite/docs/source_en/glossary.md @@ -0,0 +1 @@ +# Glossary diff --git a/lite/docs/source_en/index.rst b/lite/docs/source_en/index.rst new file mode 100644 index 0000000000000000000000000000000000000000..293149dbc33f0281ccbb1d3ee2cc3904f3ed1022 --- /dev/null +++ b/lite/docs/source_en/index.rst @@ -0,0 +1,14 @@ +.. MindSpore documentation master file, created by + sphinx-quickstart on Thu Aug 17 10:00:00 2020. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +MindSpore Lite Documentation +============================ + +.. toctree:: + :glob: + :maxdepth: 1 + + architecture + glossary diff --git a/lite/docs/source_zh_cn/_static/logo_source.png b/lite/docs/source_zh_cn/_static/logo_source.png new file mode 100644 index 0000000000000000000000000000000000000000..fc347d271abe082ae8d16242328551648766b6fb Binary files /dev/null and b/lite/docs/source_zh_cn/_static/logo_source.png differ diff --git a/lite/docs/source_zh_cn/architecture.md b/lite/docs/source_zh_cn/architecture.md new file mode 100644 index 0000000000000000000000000000000000000000..bb5332f3d143de3183b2b10cecd7fedf579bf5f2 --- /dev/null +++ b/lite/docs/source_zh_cn/architecture.md @@ -0,0 +1,56 @@ +# 总体架构 + + + +- [总体架构](#总体架构) + - [MindSpore Lite的架构](#mindspore-lite的架构) + - [MindSpore Lite的技术特点](#mindspore-lite的技术特点) + + + + + +## MindSpore Lite的架构 + +![architecture](images/architecture.jpg) + +MindSpore Lite框架包括如下几部分： + +- 前端（Frontend）：负责模型生成，用户可以通过模型构建接口构建模型，将第三方模型和MindSpore训练的模型转换为MindSpore Lite模型，其中第三方模型包括TensorFlow Lite、Caffe 1.0和ONNX模型。 + +- IR：负责MindSpore的Tensor定义、算子定义和图定义。 + +- Backend：基于IR进行图优化，包括GHLO、GLLO和量化三部分。其中，GHLO负责和硬件无关的优化，如算子融合、常量折叠等；GLLO负责与硬件相关的优化；量化Quantizer支持权重量化、激活值量化等训练后量化手段。 + +- Runtime：智能终端的推理运行时，其中session负责会话管理，提供对外接口；线程池和并行原语负责图执行使用的线程池管理，内存分配负责图执行中各个算子的内存复用，算子库提供CPU和GPU算子。 + +- Micro：IoT设备的运行时，包括模型生成.c文件、线程池、内存复用和算子库。 + +其中，Runtime和Micro共享底层的算子库、内存分配、线程池、并行原语等基础设施层。 + +## MindSpore Lite的技术特点 + +MindSpore已经在HMS和华为智能终端的图像分类、目标识别、人脸识别、文字识别等应用中广泛使用。 + +MindSpore Lite具备如下特征： + +1. 超轻量 + + - 智能手机部署的MindSpore包大小为800k左右。 + - 智能手表、耳机等IoT设备部署的MindSpore Micro包大小低至几十K级别。 + +2. 高性能 + + - 时延低：通过图优化、算子优化、并行处理、充分利用硬件能力等各种措施，优化推理时延。 + - 功耗低：通过量化、内存复用等，降低功耗。 + +3. 广覆盖 + + - 支持Android和iOS。 + - 支持智能手机、手表、耳机以及各种IoT设备。 + - 支持CV和NLP多种神经网络模型，支持200多CPU算子、60多GPU算子。 + +4. 端云协同提供一站式训练和推理 + + - 提供模型训练，模型转换优化，部署和推理端到端流程。 + - 统一的IR实现端云AI应用一体化。 diff --git a/lite/docs/source_zh_cn/conf.py b/lite/docs/source_zh_cn/conf.py new file mode 100644 index 0000000000000000000000000000000000000000..dd8e0482bb14ee1ec4242dcd8e550cbb12eb0712 --- /dev/null +++ b/lite/docs/source_zh_cn/conf.py @@ -0,0 +1,62 @@ +# Configuration file for the Sphinx documentation builder. +# +# This file only contains a selection of the most common options. For a full +# list see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +import os + + +# -- Project information ----------------------------------------------------- + +project = 'MindSpore Lite' +copyright = '2020, MindSpore Lite' +author = 'MindSpore Lite' + +# The full version, including alpha/beta/rc tags +release = 'master' + + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + 'sphinx_markdown_tables', + 'recommonmark', +] + +source_suffix = { + '.rst': 'restructuredtext', + '.md': 'markdown', +} + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = [] + +pygments_style = 'sphinx' + +autodoc_inherit_docstrings = False + +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'sphinx_rtd_theme' + +html_search_language = 'zh' + +html_static_path = ['_static'] \ No newline at end of file diff --git a/lite/docs/source_zh_cn/glossary.md b/lite/docs/source_zh_cn/glossary.md new file mode 100644 index 0000000000000000000000000000000000000000..2104f4270883b3828646fbbcf27fdb8150075704 --- /dev/null +++ b/lite/docs/source_zh_cn/glossary.md @@ -0,0 +1,12 @@ +# 术语 + + + +| 术语/缩略语 | 说明 | +| ----- | ----- | +| MindSpore Lite | 应用在智能终端，边缘册资源受限场景的MindSpore AI 引擎。 | +| MindSpore Micro | 应用在IoT设备的，包大小更小的MindSpore AI引擎。 | +| GHLO | Graph high-level optimization，图高层优化。 | +| GLLO | Graph low-level optimization，图底层优化。 | +| RT | Runtime运行时。 | + diff --git a/lite/docs/source_zh_cn/images/architecture.jpg b/lite/docs/source_zh_cn/images/architecture.jpg new file mode 100644 index 0000000000000000000000000000000000000000..6006c845e8002831ef79b39ce9d68b8afd85e0f2 Binary files /dev/null and b/lite/docs/source_zh_cn/images/architecture.jpg differ diff --git a/lite/docs/source_zh_cn/index.rst b/lite/docs/source_zh_cn/index.rst new file mode 100644 index 0000000000000000000000000000000000000000..300d20dfa712466366460818e5ab5b057702bd01 --- /dev/null +++ b/lite/docs/source_zh_cn/index.rst @@ -0,0 +1,14 @@ +.. MindSpore documentation master file, created by + sphinx-quickstart on Thu Aug 17 10:00:00 2020. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +MindSpore端侧文档 +================== + +.. toctree:: + :glob: + :maxdepth: 1 + + architecture + glossary \ No newline at end of file diff --git a/lite/tutorials/Makefile b/lite/tutorials/Makefile new file mode 100644 index 0000000000000000000000000000000000000000..1eff8952707bdfa503c8d60c1e9a903053170ba2 --- /dev/null +++ b/lite/tutorials/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = source_zh_cn +BUILDDIR = build_zh_cn + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/lite/tutorials/requirements.txt b/lite/tutorials/requirements.txt new file mode 100644 index 0000000000000000000000000000000000000000..595e6d8df76bf21707fa108959ad7d8e91c20b52 --- /dev/null +++ b/lite/tutorials/requirements.txt @@ -0,0 +1,4 @@ +sphinx +recommonmark +sphinx-markdown-tables +sphinx_rtd_theme \ No newline at end of file diff --git a/lite/tutorials/source_en/_static/logo_source.png b/lite/tutorials/source_en/_static/logo_source.png new file mode 100644 index 0000000000000000000000000000000000000000..fc347d271abe082ae8d16242328551648766b6fb Binary files /dev/null and b/lite/tutorials/source_en/_static/logo_source.png differ diff --git a/lite/tutorials/source_en/conf.py b/lite/tutorials/source_en/conf.py new file mode 100644 index 0000000000000000000000000000000000000000..a341838e9f6e67d53b005bd741fcc22fff7dec32 --- /dev/null +++ b/lite/tutorials/source_en/conf.py @@ -0,0 +1,59 @@ +# Configuration file for the Sphinx documentation builder. +# +# This file only contains a selection of the most common options. For a full +# list see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +import os +import sys + + +# -- Project information ----------------------------------------------------- + +project = 'MindSpore Lite' +copyright = '2020, MindSpore Lite' +author = 'MindSpore Lite' + +# The full version, including alpha/beta/rc tags +release = 'master' + + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + 'recommonmark', + 'sphinx_markdown_tables', +] + +source_suffix = { + '.rst': 'restructuredtext', + '.md': 'markdown', +} + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = [] + +pygments_style = 'sphinx' + +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'sphinx_rtd_theme' + +html_static_path = ['_static'] \ No newline at end of file diff --git a/lite/tutorials/source_en/deploy.md b/lite/tutorials/source_en/deploy.md new file mode 100644 index 0000000000000000000000000000000000000000..168a970080b88a1236046e040b24ea03dc2b4b64 --- /dev/null +++ b/lite/tutorials/source_en/deploy.md @@ -0,0 +1 @@ +# Deploy diff --git a/lite/tutorials/source_en/index.rst b/lite/tutorials/source_en/index.rst new file mode 100644 index 0000000000000000000000000000000000000000..ac48c19eeb0dc9ee7e406a857c3f3bd7d89a31f1 --- /dev/null +++ b/lite/tutorials/source_en/index.rst @@ -0,0 +1,23 @@ +.. MindSpore documentation master file, created by + sphinx-quickstart on Thu Aug 17 09:00:00 2020. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +MindSpore Lite Tutorials +======================== + +.. toctree:: + :glob: + :maxdepth: 1 + :caption: Quick Start + + deploy + quick_start/quick_start_lite + +.. toctree:: + :glob: + :maxdepth: 1 + :caption: Use + + use/converter_tool + use/tools diff --git a/lite/tutorials/source_en/quick_start/quick_start_lite.md b/lite/tutorials/source_en/quick_start/quick_start_lite.md new file mode 100644 index 0000000000000000000000000000000000000000..789d1707ef665c86880480b878455b3702aab957 --- /dev/null +++ b/lite/tutorials/source_en/quick_start/quick_start_lite.md @@ -0,0 +1 @@ +# Quick Start (Lite) diff --git a/lite/tutorials/source_en/use/benchmark_tool.md b/lite/tutorials/source_en/use/benchmark_tool.md new file mode 100644 index 0000000000000000000000000000000000000000..76d564f5987aab0e9ff1e407274e55338af94cf6 --- /dev/null +++ b/lite/tutorials/source_en/use/benchmark_tool.md @@ -0,0 +1 @@ +# Benchmark Tool diff --git a/lite/tutorials/source_en/use/converter_tool.md b/lite/tutorials/source_en/use/converter_tool.md new file mode 100644 index 0000000000000000000000000000000000000000..bf2fe497db621697c622716edfce5e58f9b343c9 --- /dev/null +++ b/lite/tutorials/source_en/use/converter_tool.md @@ -0,0 +1 @@ +# Converter Tool diff --git a/lite/tutorials/source_en/use/runtime_lite.md b/lite/tutorials/source_en/use/runtime_lite.md new file mode 100644 index 0000000000000000000000000000000000000000..32aff0082c988a67cdf26700e0b6fc9e4433cfb5 --- /dev/null +++ b/lite/tutorials/source_en/use/runtime_lite.md @@ -0,0 +1 @@ +# Runtime (Lite) diff --git a/lite/tutorials/source_en/use/timeprofiler_tool.md b/lite/tutorials/source_en/use/timeprofiler_tool.md new file mode 100644 index 0000000000000000000000000000000000000000..eb29fa545d56a621b87fcd5c180d754ed54d6ab0 --- /dev/null +++ b/lite/tutorials/source_en/use/timeprofiler_tool.md @@ -0,0 +1 @@ +# TimeProfiler Tool diff --git a/lite/tutorials/source_en/use/tools.rst b/lite/tutorials/source_en/use/tools.rst new file mode 100644 index 0000000000000000000000000000000000000000..722a938d6916890fc2e03545f926c6092aff85cd --- /dev/null +++ b/lite/tutorials/source_en/use/tools.rst @@ -0,0 +1,8 @@ +Other Tools +=========== + +.. toctree:: + :maxdepth: 1 + + benchmark_tool + timeprofiler_tool \ No newline at end of file diff --git a/lite/tutorials/source_zh_cn/_static/logo_source.png b/lite/tutorials/source_zh_cn/_static/logo_source.png new file mode 100644 index 0000000000000000000000000000000000000000..fc347d271abe082ae8d16242328551648766b6fb Binary files /dev/null and b/lite/tutorials/source_zh_cn/_static/logo_source.png differ diff --git a/lite/tutorials/source_zh_cn/conf.py b/lite/tutorials/source_zh_cn/conf.py new file mode 100644 index 0000000000000000000000000000000000000000..c2ba20320a75ee8ae812f03dbc21cb39f30827a8 --- /dev/null +++ b/lite/tutorials/source_zh_cn/conf.py @@ -0,0 +1,61 @@ +# Configuration file for the Sphinx documentation builder. +# +# This file only contains a selection of the most common options. For a full +# list see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +import os +import sys + + +# -- Project information ----------------------------------------------------- + +project = 'MindSpore Lite' +copyright = '2020, MindSpore Lite' +author = 'MindSpore Lite' + +# The full version, including alpha/beta/rc tags +release = 'master' + + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + 'recommonmark', + 'sphinx_markdown_tables', +] + +source_suffix = { + '.rst': 'restructuredtext', + '.md': 'markdown', +} + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = [] + +pygments_style = 'sphinx' + +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'sphinx_rtd_theme' + +html_search_language = 'zh' + +html_static_path = ['_static'] \ No newline at end of file diff --git a/lite/tutorials/source_zh_cn/deploy.md b/lite/tutorials/source_zh_cn/deploy.md new file mode 100644 index 0000000000000000000000000000000000000000..081b7d4565d2eb629b6f4abfcf789bdf9c599474 --- /dev/null +++ b/lite/tutorials/source_zh_cn/deploy.md @@ -0,0 +1,116 @@ +# 部署 + +本文档介绍如何在Ubuntu系统上快速安装MindSpore Lite。 + + + +- [部署](#部署) + - [环境要求](#环境要求) + - [编译选项](#编译选项) + - [输出件说明](#输出件说明) + - [编译示例](#编译示例) + + + + + +## 环境要求 + +- 编译环境仅支持x86_64版本的Linux：推荐使用Ubuntu 18.04.02LTS + +- 编译依赖（基本项） + - [CMake](https://cmake.org/download/) >= 3.14.1 + - [GCC](https://gcc.gnu.org/releases.html) >= 7.3.0 + - [Python](https://www.python.org/) >= 3.7 + - [Android_NDK r20b](https://dl.google.com/android/repository/android-ndk-r20b-linux-x86_64.zip) + + > - 仅在编译ARM版本时需要安装`Android_NDK`，编译x86_64版本可跳过此项。 + > - 如果安装并使用`Android_NDK`，需配置环境变量，命令参考：`export ANDROID_NDK={$NDK_PATH}/android-ndk-r20b`。 + +- 编译依赖（MindSpore Lite模型转换工具所需附加项，仅编译x86_64版本时需要） + - [Autoconf](http://ftp.gnu.org/gnu/autoconf/) >= 2.69 + - [Libtool](https://www.gnu.org/software/libtool/) >= 2.4.6 + - [LibreSSL](http://www.libressl.org/) >= 3.1.3 + - [Automake](https://www.gnu.org/software/automake/) >= 1.11.6 + - [Libevent](https://libevent.org) >= 2.0 + - [M4](https://www.gnu.org/software/m4/m4.html) >= 1.4.18 + - [OpenSSL](https://www.openssl.org/) >= 1.1.1 + + +## 编译选项 + +MindSpore Lite提供多种编译方式，用户可根据需要选择不同的编译选项。 + +| 参数 | 参数说明 | 取值范围 | 是否必选 | +| -------- | ----- | ---- | ---- | +| -d | 设置该参数，则编译Debug版本，否则编译Release版本 | - | 否 | +| -i | 设置该参数，则进行增量编译，否则进行全量编译 | - | 否 | +| -j[n] | 设定编译时所用的线程数，否则默认设定为8线程 | - | 否 | +| -I | 选择适用架构 | arm64、arm32、x86_64 | 是 | +| -e | 在ARM架构下，选择后端算子，设置`gpu`参数，会同时编译框架内置的GPU算子 | gpu | 否 | +| -h | 设置该参数，显示编译帮助信息 | - | 否 | + +> 在`-I`参数变动时，即切换适用架构时，无法使用`-i`参数进行增量编译。 + +## 输出件说明 + +编译完成后，进入源码的`mindspore/output`目录，可查看编译后生成的文件，命名为`MSLite-{version}-{platform}.tar.gz`。解压后，即可获得编译后的工具包。 + +```bash +tar -xvf MSLite-{version}-{platform}.tar.gz +``` + +编译后的输出件一般包含以下几种，架构的选择会影响输出件的种类。 + +| 目录 | 说明 | x86_64 | arm64 | arm32 | +| --- | --- | --- | --- | --- | +| include | 推理框架头文件 | 有 | 有 | 有 | +| lib | 推理框架动态库 | 有 | 有 | 有 | +| benchmark | 基准测试工具 | 有 | 有 | 有 | +| time_profiler | 模型网络层耗时分析工具 | 有 | 有 | 有 | +| converter | 模型转换工具 | 有 | 无 | 无 | +| third_party | 第三方库头文件和库 | 有 | 有 | 有 | + +在x86_64、ARM两种架构下，`third party`的内容不同。其中： +- x86_64：包含`flatbuffers`（FlatBuffers头文件）和`protobuf`（Protobuf头文件与动态库）。 +- ARM：包含`flatbuffers`（FlatBuffers头文件）。 + +> 运行converter、benchmark或time_profiler目录下的工具前，都需配置环境变量，将MindSpore Lite和Protobuf的动态库所在的路径配置到系统搜索动态库的路径中。以0.6.0-beta版本为例：`export LD_LIBRARY_PATH=./MSLite-0.6.0-linux_x86_64/lib:./MSLite-0.6.0-linux_x86_64/third_party/protobuf/lib:${LD_LIBRARY_PATH}`。 + +## 编译示例 + +首先，从MindSpore代码仓下载源码。 + +```bash +git clone https://gitee.com/mindspore/mindspore.git +``` + +然后，在源码根目录下，执行如下命令，可编译不同版本的MindSpore Lite。 + +- 编译x86_64架构Debug版本。 + ```bash + bash build.sh -I x86_64 -d + ``` + +- 编译x86_64架构Release版本，同时设定线程数。 + ```bash + bash build.sh -I x86_64 -j32 + ``` + +- 增量编译ARM64架构Release版本，同时设定线程数。 + ```bash + bash build.sh -I arm64 -i -j32 + ``` + +- 编译ARM64架构Release版本，同时编译内置的GPU算子。 + ```bash + bash build.sh -I arm64 -e gpu + ``` + +> `build.sh`中会执行`git clone`获取第三方依赖库的代码，请提前确保git的网络设置正确可用。 + +以0.6.0-beta版本为例，x86_64架构Release版本编译完成之后，进入`mindspore/output`目录，执行如下解压缩命令，即可获取输出件`include`、`lib`、`benchmark`、`time_profiler`、`converter`和`third_party`。 + +```bash +tar -xvf MSLite-0.6.0-linux_x86_64.tar.gz +``` diff --git a/lite/tutorials/source_zh_cn/index.rst b/lite/tutorials/source_zh_cn/index.rst new file mode 100644 index 0000000000000000000000000000000000000000..77d3d333b2373192055526caa82d0928a4a5a24c --- /dev/null +++ b/lite/tutorials/source_zh_cn/index.rst @@ -0,0 +1,23 @@ +.. MindSpore documentation master file, created by + sphinx-quickstart on Thu Aug 17 09:00:00 2020. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +MindSpore端侧教程 +================== + +.. toctree:: + :glob: + :maxdepth: 1 + :caption: 快速入门 + + deploy + quick_start/quick_start_lite + +.. toctree:: + :glob: + :maxdepth: 1 + :caption: 使用指南 + + use/converter_tool + use/tools diff --git a/lite/tutorials/source_zh_cn/quick_start/quick_start_lite.md b/lite/tutorials/source_zh_cn/quick_start/quick_start_lite.md new file mode 100644 index 0000000000000000000000000000000000000000..392ca8c332ca24e6d1f55eadffac74adbf00894d --- /dev/null +++ b/lite/tutorials/source_zh_cn/quick_start/quick_start_lite.md @@ -0,0 +1,11 @@ +# 快速入门（Lite） + + + +- [快速入门（Lite）](#快速入门lite) + + + + + + diff --git a/lite/tutorials/source_zh_cn/use/benchmark_tool.md b/lite/tutorials/source_zh_cn/use/benchmark_tool.md new file mode 100644 index 0000000000000000000000000000000000000000..172026a124d13a5b497bb67e1e6ccb278a0fc66a --- /dev/null +++ b/lite/tutorials/source_zh_cn/use/benchmark_tool.md @@ -0,0 +1,95 @@ +# Benchmark工具 + + + +- [Benchmark工具](#benchmark工具) + - [概述](#概述) + - [环境准备](#环境准备) + - [参数说明](#参数说明) + - [使用示例](#使用示例) + - [性能测试](#性能测试) + - [精度测试](#精度测试) + + + + + +## 概述 + +Benchmark工具是一款可以对MindSpore Lite模型进行基准测试的工具，由C++语言编码实现。它不仅可以对MindSpore Lite模型前向推理执行耗时进行定量分析（性能），还可以通过指定模型输出进行可对比的误差分析（精度）。 + +## 环境准备 + +使用Benchmark工具，需要进行如下环境准备工作。 + +- 编译：Benchmark工具代码在MindSpore源码的`mindspore/lite/tools/benchmark`目录中，参考部署文档中的[环境要求](https://www.mindspore.cn/lite/docs/zh-CN/master/deploy.html#id2)和[编译示例](https://www.mindspore.cn/lite/docs/zh-CN/master/deploy.html#id5)，安装编译依赖基本项，并执行编译。 + +- 运行：参考部署文档中的[输出件说明](https://www.mindspore.cn/lite/docs/zh-CN/master/deploy.html#id4)，获得`benchmark`工具，并配置环境变量。 + +## 参数说明 + +使用编译好的Benchmark工具进行模型的基准测试时，其命令格式如下所示。 + +```bash +./benchmark --modelPath= [--accuracyThreshold=] + [--calibDataPath=] [--cpuBindMode=] + [--device=] [--help] [--inDataPath=] + [--inDataType=] [--loopCount=] + [--numThreads=] [--omModelPath=] + [--resizeDims=] [--warmUpLoopCount=] +``` + +下面提供详细的参数说明。 + +| 参数名 | 属性 | 功能描述 | 参数类型 | 默认值 | 取值范围 | +| ----------------- | ---- | ------------------------------------------------------------ | ------ | -------- | ---------------------------------- | +| `--modelPath=` | 必选 | 指定需要进行基准测试的MindSpore Lite模型文件路径。 | String | null | - | +| `--accuracyThreshold=` | 可选 | 指定准确度阈值。 | Float | 0.5 | - | +| `--calibDataPath=` | 可选 | 指定标杆数据的文件路径。标杆数据作为该测试模型的对比输出，是该测试模型使用相同输入并由其它深度学习框架前向推理而来。 | String | null | - | +| `--cpuBindMode=` | 可选 | 指定模型推理程序运行时绑定的CPU核类型。 | Integer | 1 | -1：表示中核
1：表示大核
0：表示不绑定 | +| `--device=` | 可选 | 指定模型推理程序运行的设备类型。 | String | CPU | CPU、NPU、GPU | +| `--help` | 可选 | 显示`benchmark`命令的帮助信息。 | - | - | - | +| `--inDataPath=` | 可选 | 指定测试模型输入数据的文件路径。如果未设置，则使用随机输入。 | String | null | - | +| `--inDataType=` | 可选 | 指定测试模型输入数据的文件类型。 | String | bin | img：表示输入数据的文件类型为图片
bin：表示输入数据的类型为二进制文件 | +| `--loopCount=` | 可选 | 指定Benchmark工具进行基准测试时，测试模型的前向推理运行次数，其值为正整数。 | Integer | 10 | - | +| `--numThreads=` | 可选 | 指定模型推理程序运行的线程数。 | Integer | 2 | - | +| `--omModelPath=` | 可选 | 指定OM模型的文件路径，此参数仅当`device`类型为NPU时可选设置。 | String | null | - | +| `--resizeDims=` | 可选 | 指定测试模型输入数据需要调整的尺寸大小。 | String | null | - | +| `--warmUpLoopCount=` | 可选 | 指定测试模型在执行基准测试运行轮数前进行的模型预热推理次数。 | Integer | 3 | - | + +## 使用示例 + +对于不同的MindSpore Lite模型，在使用Benchmark工具对其进行基准测试时，可通过设置不同的参数，实现对其不同的测试功能。主要分为性能测试和精度测试。 + +### 性能测试 + +Benchmark工具进行的性能测试主要的测试指标为模型单次前向推理的耗时。在性能测试任务中，不需要设置`calibDataPath`等标杆数据参数。例如： + +```bash +./benchmark --modelPath=./models/face/age/ml_face_age.ms +``` + +这条命令使用随机输入，其他参数使用默认值。该命令执行后会输出如下统计信息，该信息显示了测试模型在运行指定推理轮数后所统计出的单次推理最短耗时、单次推理最长耗时和平均推理耗时。 + +``` +Model = ml_face_age.ms, numThreads = 2, MinRunTime = 72.228996 ms, MaxRuntime = 73.094002 ms, AvgRunTime = 72.556000 ms +``` + +### 精度测试 + +Benchmark工具进行的精度测试主要是通过设置标杆数据来对比验证MinSpore模型输出的精确性。在精确度测试任务中，除了需要设置`modelPath`参数以外，还必须设置`calibDataPath`参数。例如： + +```bash +./benchmark --modelPath=./models/face/age/ml_face_age.ms --inDataPath=./data/input/1_3_120_120_NCHW.bin --device=NPU --accuracyThreshold=3 --calibDataPath=./data/output/face/ml_face_age.out +``` + +这条命令指定了测试模型的输入数据、标杆数据，同时指定了模型推理程序在NPU上运行，并指定了准确度阈值为3%。该命令执行后会输出如下统计信息，该信息显示了测试模型的单条输入数据、输出节点的输出结果和平均偏差率以及所有节点的平均偏差率。 + +``` +InData0: 139.947 182.373 153.705 138.945 108.032 164.703 111.585 227.402 245.734 97.7776 201.89 134.868 144.851 236.027 18.1142 22.218 5.15569 212.318 198.43 221.853 +================ Comparing Output data ================ +Data of node age_out : 5.94584e-08 6.3317e-08 1.94726e-07 1.91809e-07 8.39805e-08 7.66035e-08 1.69285e-07 1.46246e-07 6.03796e-07 1.77631e-07 1.54343e-07 2.04623e-07 8.89609e-07 3.63487e-06 4.86876e-06 1.23939e-05 3.09981e-05 3.37098e-05 0.000107102 0.000213932 0.000533579 0.00062465 0.00296401 0.00993984 0.038227 0.0695085 0.162854 0.123199 0.24272 0.135048 0.169159 0.0221256 0.013892 0.00502971 0.00134921 0.00135701 0.000383242 0.000163475 0.000136294 9.77864e-05 8.00793e-05 5.73874e-05 3.53858e-05 2.18535e-05 2.04467e-05 1.85286e-05 1.05075e-05 9.34751e-06 6.12732e-06 4.55476e-06 +Mean bias of node age_out : 0% +Mean bias of all nodes: 0% +======================================================= +``` diff --git a/lite/tutorials/source_zh_cn/use/converter_tool.md b/lite/tutorials/source_zh_cn/use/converter_tool.md new file mode 100644 index 0000000000000000000000000000000000000000..bd9d7c88b0f181f9791e7e9b83ee75bf99d70089 --- /dev/null +++ b/lite/tutorials/source_zh_cn/use/converter_tool.md @@ -0,0 +1,106 @@ +# 模型转换工具 + + + +- [模型转换工具](#模型转换工具) + - [概述](#概述) + - [环境准备](#环境准备) + - [参数说明](#参数说明) + - [模型可视化](#模型可视化) + - [使用示例](#使用示例) + + + + + +## 概述 + +MindSpore Lite提供离线转换模型功能的工具，支持多种类型的模型转换，同时提供转化后模型可视化的功能，转换后的模型可用于推理。命令行参数包含多种个性化选项，为用户提供方便的转换途径。 + +目前支持的输入格式有：MindSpore、TensorFlow Lite、Caffe和ONNX。 + +## 环境准备 + +使用MindSpore Lite模型转换工具，需要进行如下环境准备工作。 + +- 编译：模型转换工具代码在MindSpore源码的`mindspore/lite/tools/converter`目录中，参考部署文档中的[环境要求](https://www.mindspore.cn/lite/docs/zh-CN/master/deploy.html#id2)和[编译示例](https://www.mindspore.cn/lite/docs/zh-CN/master/deploy.html#id5)，安装编译依赖基本项与模型转换工具所需附加项，并编译x86_64版本。 + +- 运行：参考部署文档中的[输出件说明](https://www.mindspore.cn/lite/docs/zh-CN/master/deploy.html#id4)，获得`converter`工具，并配置环境变量。 + +## 参数说明 + +使用`./converter_lite `即可完成转换，同时提供了多种参数设置，用户可根据需要来选择使用。 +此外，用户可输入`./converter_lite --help`获取实时帮助。 + +下面提供详细的参数说明。 + +| 参数 | 是否必选 | 参数说明 | 取值范围 | 默认值 | +| -------- | ------- | ----- | --- | ---- | +| `--help` | 否 | 打印全部帮助信息。 | - | - | +| `--fmk=` | 是 | 输入模型的原始格式（可从模型文件后缀识别）。 | MS、CAFFE、TFLITE、ONNX | - | +| `--modelFile=` | 是 | 输入模型的路径。 | - | - | +| `--outputFile=` | 是 | 输出模型的路径（不存在时将自动创建目录），不需加后缀，可自动生成`.ms`后缀。 | - | - | +| `--weightFile=` | 转换Caffe模型时必选 | 输入模型weight文件的路径。 | - | - | +| `--quantType=` | 否 | 设置模型的训练类型 | PostTraining：训练后量化
AwareTraining：感知量化。 | - | + +> - 参数名和参数值之间用等号连接，中间不能有空格。 +> - Caffe模型一般分为两个文件：`*.prototxt`模型结构，对应`--modelFile`参数；`*.caffemodel`模型权值，对应`--weightFile`参数。 + +## 模型可视化 + +模型可视化工具提供了一种查验模型转换结果的方法。用户可使用Json命令生成`*.json`文件，与原模型相对比，确定转化效果。 + +TODO: 此功能还在开发中。 + +## 使用示例 + +首先，在源码根目录下，输入命令进行编译，可参考`deploy.md`。 +```bash +bash build.sh -I x86_64 +``` +> 目前模型转换工具仅支持x86_64架构。 + +下面选取了几个常用示例，说明转换命令的使用方法。 + +- 以Caffe模型LeNet为例，执行转换命令。 + + ```bash + ./converter_lite --fmk=CAFFE --modelFile=lenet.prototxt --weightFile=lenet.caffemodel --outputFile=lenet + ``` + + 本例中，因为采用了Caffe模型，所以需要模型结构、模型权值两个输入文件。再加上其他必需的fmk类型和输出路径两个参数，即可成功执行。 + + 结果显示为： + ``` + INFO [converter/converter.cc:190] Runconverter] CONVERTER RESULT: SUCCESS! + ``` + 这表示已经成功将Caffe模型转化为MindSpore Lite模型，获得新文件`lenet.ms`。 + +- 以MindSpore、TensorFlow Lite、ONNX模型格式和感知量化模型为例，执行转换命令。 + + - MindSpore模型`model.mindir` + ```bash + ./converter_lite --fmk=MS --modelFile=model.mindir --outputFile=model + ``` + + - TensorFlow Lite模型`model.tflite` + ```bash + ./converter_lite --fmk=TFLITE --modelFile=model.tflite --outputFile=model + ``` + + - ONNX模型`model.onnx` + ```bash + ./converter_lite --fmk=ONNX --modelFile=model.onnx --outputFile=model + ``` + + - TensorFlow Lite感知量化模型`model_quant.tflite` + ```bash + ./converter_lite --fmk=TFLITE --modelFile=model_quant.tflite --outputFile=model --quantType=AwareTraining + ``` + + 以上几种情况下，均显示如下转换成功提示，且同时获得`model.ms`目标文件。 + ``` + INFO [converter/converter.cc:190] Runconverter] CONVERTER RESULT: SUCCESS! + ``` + +你可以选择使用模型打印工具，可视化查验上述转化后生成的MindSpore Lite模型。本部分功能开发中。 diff --git a/lite/tutorials/source_zh_cn/use/runtime_lite.md b/lite/tutorials/source_zh_cn/use/runtime_lite.md new file mode 100644 index 0000000000000000000000000000000000000000..1daa84ad6643c14b1f5b8f3556be3cee6805f078 --- /dev/null +++ b/lite/tutorials/source_zh_cn/use/runtime_lite.md @@ -0,0 +1,11 @@ +# Runtime使用指南（Lite） + + + +- [Runtime使用指南（Lite）](#runtime使用指南lite) + + + + + + diff --git a/lite/tutorials/source_zh_cn/use/timeprofiler_tool.md b/lite/tutorials/source_zh_cn/use/timeprofiler_tool.md new file mode 100644 index 0000000000000000000000000000000000000000..f3bce7af197beacb782aa4a8a1d7b0188712be6a --- /dev/null +++ b/lite/tutorials/source_zh_cn/use/timeprofiler_tool.md @@ -0,0 +1,92 @@ +# TimeProfiler工具 + + + +- [TimeProfiler工具](#timeprofiler工具) + - [概述](#概述) + - [环境准备](#环境准备) + - [参数说明](#参数说明) + - [使用示例](#使用示例) + + + + + +## 概述 + +TimeProfiler工具可以对MindSpore Lite模型网络层的前向推理进行耗时分析，由C++语言编码实现。 + +## 环境准备 + +使用TimeProfiler工具，需要进行如下环境准备工作。 + +- 编译：TimeProfiler工具代码在MindSpore源码的`mindspore/lite/tools/time_profiler`目录中，参考部署文档中的[环境要求](https://www.mindspore.cn/lite/docs/zh-CN/master/deploy.html#id2)和[编译示例](https://www.mindspore.cn/lite/docs/zh-CN/master/deploy.html#id5)，安装编译依赖基本项，并执行编译。 + +- 运行：参考部署文档中的[输出件说明](https://www.mindspore.cn/lite/docs/zh-CN/master/deploy.html#id4)，获得`time_profiler`工具，并配置环境变量。 + +## 参数说明 + +使用编译好的TimeProfiler工具进行模型网络层耗时分析时，其命令格式如下所示。 + +```bash +./timeprofiler --modelPath= [--help] [--loopCount=] [--numThreads=] [--cpuBindMode=] [--inDataPath=] +``` + +下面提供详细的参数说明。 + +| 参数名 | 属性 | 功能描述 | 参数类型 | 默认值 | 取值范围 | +| ----------------- | ---- | ------------------------------------------------------------ | ------ | -------- | ---------------------------------- | +| `--help` | 可选 | 显示`timeprofiler`命令的帮助信息。 | - | - | - | +| `--modelPath= ` | 必选 | 指定需要进行耗时分析的MindSpore Lite模型的文件路径。 | String | null | - | +| `--loopCount=` | 可选 | 指定TimeProfiler工具进行耗时分析时，模型推理的运行次数，其值为正整数。 | Integer | 100 | - | +| `--numThreads=` | 可选 | 指定模型推理程序运行的线程数。 | Integer | 4 | - | +| `--cpuBindMode=` | 可选 | 指定模型推理程序运行时绑定的CPU核类型。 | Integer | 1 | -1：表示中核
1：表示大核
0：表示不绑定 | +| `--inDataPath=` | 可选 | 指定模型输入数据的文件路径。如果未设置，则使用随机输入。 | String | null | - | + +## 使用示例 + +使用TimeProfiler对`tcpclassify.ms`模型的网络层进行耗时分析，并且设置模型推理循环运行次数为10，则其命令代码如下： + +```bash +./timeprofiler --modelPath=./models/siteAI/tcpclassify.ms --loopCount=10 +``` + +该条命令执行后，TimeProfiler工具会输出模型网络层运行耗时的相关统计信息。对于本例命令，输出的统计信息如下。其中统计信息按照`opName`和`optype`两种划分方式分别显示，`opName`表示算子名，`optype`表示算子类别，`avg`表示该算子的平均单次运行时间，`percent`表示该算子运行耗时占所有算子运行总耗时的比例，`calledTimess`表示该算子的运行次数，`opTotalTime`表示该算子运行指定次数的总耗时。最后，`total time`和`kernel cost`分别显示了该模型单次推理的平均耗时和模型推理中所有算子的平均耗时之和。 + +``` +----------------------------------------------------------------------------------------- +opName avg(ms) percent calledTimess opTotalTime +conv2d_1/convolution 2.264800 0.824012 10 22.648003 +conv2d_2/convolution 0.223700 0.081390 10 2.237000 +dense_1/BiasAdd 0.007500 0.002729 10 0.075000 +dense_1/MatMul 0.126000 0.045843 10 1.260000 +dense_1/Relu 0.006900 0.002510 10 0.069000 +max_pooling2d_1/MaxPool 0.035100 0.012771 10 0.351000 +max_pooling2d_2/MaxPool 0.014300 0.005203 10 0.143000 +max_pooling2d_2/MaxPool_nchw2nhwc_reshape_1/Reshape_0 0.006500 0.002365 10 0.065000 +max_pooling2d_2/MaxPool_nchw2nhwc_reshape_1/Shape_0 0.010900 0.003966 10 0.109000 +output/BiasAdd 0.005300 0.001928 10 0.053000 +output/MatMul 0.011400 0.004148 10 0.114000 +output/Softmax 0.013300 0.004839 10 0.133000 +reshape_1/Reshape 0.000900 0.000327 10 0.009000 +reshape_1/Reshape/shape 0.009900 0.003602 10 0.099000 +reshape_1/Shape 0.002300 0.000837 10 0.023000 +reshape_1/strided_slice 0.009700 0.003529 10 0.097000 +----------------------------------------------------------------------------------------- +opType avg(ms) percent calledTimess opTotalTime +Activation 0.006900 0.002510 10 0.069000 +BiasAdd 0.012800 0.004657 20 0.128000 +Conv2D 2.488500 0.905401 20 24.885004 +MatMul 0.137400 0.049991 20 1.374000 +Nchw2Nhwc 0.017400 0.006331 20 0.174000 +Pooling 0.049400 0.017973 20 0.494000 +Reshape 0.000900 0.000327 10 0.009000 +Shape 0.002300 0.000837 10 0.023000 +SoftMax 0.013300 0.004839 10 0.133000 +Stack 0.009900 0.003602 10 0.099000 +StridedSlice 0.009700 0.003529 10 0.097000 + +total time : 2.90800 ms, kernel cost : 2.74851 ms + +----------------------------------------------------------------------------------------- +``` \ No newline at end of file diff --git a/lite/tutorials/source_zh_cn/use/tools.rst b/lite/tutorials/source_zh_cn/use/tools.rst new file mode 100644 index 0000000000000000000000000000000000000000..c9b9e2a8401e7f71da7f3e10b5946d02fa7ff58c --- /dev/null +++ b/lite/tutorials/source_zh_cn/use/tools.rst @@ -0,0 +1,8 @@ +其他工具 +========== + +.. toctree:: + :maxdepth: 1 + + benchmark_tool + timeprofiler_tool \ No newline at end of file