提交 3703474d 编写于 作者: L liangyongxiong

add mindinsight supports for gpu

上级 4a303b09
...@@ -13,6 +13,7 @@ ...@@ -13,6 +13,7 @@
- [通过可执行文件安装](#通过可执行文件安装) - [通过可执行文件安装](#通过可执行文件安装)
- [从源码编译安装](#从源码编译安装) - [从源码编译安装](#从源码编译安装)
- [安装验证](#安装验证) - [安装验证](#安装验证)
- [安装MindInsight](#安装mindinsight)
- [安装MindArmour](#安装mindarmour) - [安装MindArmour](#安装mindarmour)
<!-- /TOC --> <!-- /TOC -->
...@@ -112,6 +113,76 @@ ...@@ -112,6 +113,76 @@
[ 2. 2. 2. 2.]]] [ 2. 2. 2. 2.]]]
``` ```
# 安装MindInsight
当您需要查看训练过程中的标量、图像、计算图以及模型超参等信息时,可以选装MindInsight。
## 环境要求
### 系统要求和软件依赖
| 版本号 | 操作系统 | 可执行文件安装依赖 | 源码编译安装依赖 |
| ---- | :--- | :--- | :--- |
| MindInsight master | - Ubuntu 16.04(及以上) x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5 <br> - MindSpore master <br> - 其他依赖项参见[requirements.txt](https://gitee.com/mindspore/mindinsight/blob/master/requirements.txt) | **编译依赖:**<br> - [Python](https://www.python.org/downloads/) 3.7.5 <br> - [CMake](https://cmake.org/download/) >= 3.14.1 <br> - [GCC](https://gcc.gnu.org/releases.html) 7.3.0 <br> - [node.js](https://nodejs.org/en/download/) >= 10.19.0 <br> - [wheel](https://pypi.org/project/wheel/) >= 0.32.0 <br> - [pybind11](https://pypi.org/project/pybind11/) >= 2.4.3 <br> **安装依赖:**<br> 与可执行文件安装依赖相同 |
- 在联网状态下,安装whl包时会自动下载requirements.txt中的依赖项,其余情况需自行安装。
## 安装指南
### 通过可执行文件安装
1.[MindSpore网站下载地址](https://www.mindspore.cn/versions)下载whl包,建议先进行SHA-256完整性校验,执行如下命令安装MindInsight。
```bash
pip install mindinsight-{version}-cp37-cp37m-linux_{arch}.whl
```
2. 执行如下命令,如果提示`web address: http://127.0.0.1:8080`,则说明安装成功。
```bash
mindinsight start
```
### 从源码编译安装
1. 从代码仓下载源码。
```bash
git clone https://gitee.com/mindspore/mindinsight.git
```
2. 可选择以下任意一种安装方式:
(1) 进入源码的根目录,执行安装命令。
```bash
cd mindinsight
pip install -r requirements.txt
python setup.py install
```
(2) 构建whl包进行安装。
进入源码的build目录,执行MindInsight编译脚本。
```bash
cd mindinsight/build
bash build.sh
```
进入源码的output目录,即可查看生成的MindInsight安装包,执行安装命令。
```bash
cd mindinsight/output
pip install mindinsight-{version}-cp37-cp37m-linux_{arch}.whl
```
3. 执行如下命令,如果提示`web address: http://127.0.0.1:8080`,则说明安装成功。
```bash
mindinsight start
```
# 安装MindArmour # 安装MindArmour
当您进行AI模型安全研究或想要增强AI应用模型的防护能力时,可以选装MindArmour。 当您进行AI模型安全研究或想要增强AI应用模型的防护能力时,可以选装MindArmour。
......
...@@ -13,6 +13,7 @@ This document describes how to quickly install MindSpore on a NVIDIA GPU environ ...@@ -13,6 +13,7 @@ This document describes how to quickly install MindSpore on a NVIDIA GPU environ
- [Installing Using Executable Files](#installing-using-executable-files) - [Installing Using Executable Files](#installing-using-executable-files)
- [Installing Using the Source Code](#installing-using-the-source-code) - [Installing Using the Source Code](#installing-using-the-source-code)
- [Installation Verification](#installation-verification) - [Installation Verification](#installation-verification)
- [Installing MindInsight](#installing-mindinsight)
- [Installing MindArmour](#installing-mindarmour) - [Installing MindArmour](#installing-mindarmour)
<!-- /TOC --> <!-- /TOC -->
...@@ -112,6 +113,76 @@ This document describes how to quickly install MindSpore on a NVIDIA GPU environ ...@@ -112,6 +113,76 @@ This document describes how to quickly install MindSpore on a NVIDIA GPU environ
[ 2. 2. 2. 2.]]] [ 2. 2. 2. 2.]]]
``` ```
# Installing MindInsight
If you need to analyze information such as model scalars, graphs, and model traceback, you can install MindInsight.
## Environment Requirements
### System Requirements and Software Dependencies
| Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies |
| ---- | :--- | :--- | :--- |
| MindInsight master | - Ubuntu 16.04 or later x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5 <br> - MindSpore master <br> - For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindinsight/blob/master/requirements.txt). | **Compilation dependencies:**<br> - [Python](https://www.python.org/downloads/) 3.7.5 <br> - [CMake](https://cmake.org/download/) >= 3.14.1 <br> - [GCC](https://gcc.gnu.org/releases.html) 7.3.0 <br> - [node.js](https://nodejs.org/en/download/) >= 10.19.0 <br> - [wheel](https://pypi.org/project/wheel/) >= 0.32.0 <br> - [pybind11](https://pypi.org/project/pybind11/) >= 2.4.3 <br> **Installation dependencies:**<br> same as the executable file installation dependencies. |
- When the network is connected, dependency items in the requirements.txt file are automatically downloaded during .whl package installation. In other cases, you need to manually install dependency items.
## Installation Guide
### Installing Using Executable Files
1. Download the .whl package from the [MindSpore website](https://www.mindspore.cn/versions/en). It is recommended to perform SHA-256 integrity verification first and run the following command to install MindInsight:
```bash
pip install mindinsight-{version}-cp37-cp37m-linux_{arch}.whl
```
2. Run the following command. If `web address: http://127.0.0.1:8080` is displayed, the installation is successful.
```bash
mindinsight start
```
### Installing Using the Source Code
1. Download the source code from the code repository.
```bash
git clone https://gitee.com/mindspore/mindinsight.git
```
2. Install MindInsight by using either of the following installation methods:
(1) Access the root directory of the source code and run the following installation command:
```bash
cd mindinsight
pip install -r requirements.txt
python setup.py install
```
(2) Create a .whl package to install MindInsight.
Access the build directory of the source code and run the MindInsight compilation script.
```bash
cd mindinsight/build
bash build.sh
```
Access the output directory of the source code, where the generated MindInsight installation package is stored, and run the installation command.
```bash
cd mindinsight/output
pip install mindinsight-{version}-cp37-cp37m-linux_{arch}.whl
```
3. Run the following command. If `web address: http://127.0.0.1:8080` is displayed, the installation is successful.
```bash
mindinsight start
```
# Installing MindArmour # Installing MindArmour
If you need to conduct AI model security research or enhance the security of the model in you applications, you can install MindArmour. If you need to conduct AI model security research or enhance the security of the model in you applications, you can install MindArmour.
......
...@@ -6,8 +6,6 @@ ...@@ -6,8 +6,6 @@
- [Overview](#overview) - [Overview](#overview)
- [Operation Process](#operation-process) - [Operation Process](#operation-process)
- [Preparing the Training Script](#preparing-the-training-script) - [Preparing the Training Script](#preparing-the-training-script)
- [Basic Script Editing](#basic-script-editing)
- [Recording the Computational Graph After Operator Fusion](#recording-the-computational-graph-after-operator-fusion)
- [MindInsight Commands](#mindinsight-commands) - [MindInsight Commands](#mindinsight-commands)
- [Visualization Components](#visualization-components) - [Visualization Components](#visualization-components)
- [Computational Graph Visualization](#computational-graph-visualization) - [Computational Graph Visualization](#computational-graph-visualization)
...@@ -35,9 +33,6 @@ Currently, MindSpore uses the `Callback` mechanism to save scalars, images, comp ...@@ -35,9 +33,6 @@ Currently, MindSpore uses the `Callback` mechanism to save scalars, images, comp
Scalar and image data is recorded by using the `Summary` operator. A computational graph is saved to the summary log file by using `SummaryRecord` after network compilation is complete. Scalar and image data is recorded by using the `Summary` operator. A computational graph is saved to the summary log file by using `SummaryRecord` after network compilation is complete.
Model parameters are saved to the summary log file by using `TrainLineage` or `EvalLineage`. Model parameters are saved to the summary log file by using `TrainLineage` or `EvalLineage`.
### Basic Script Editing
Step 1: Call the `Summary` operator in the `construct` function of the derived class that inherits `nn.Cell` to collect image or scalar data. Step 1: Call the `Summary` operator in the `construct` function of the derived class that inherits `nn.Cell` to collect image or scalar data.
For example, when a network is defined, image data is recorded in `construct` of the network. When the loss function is defined, the loss value is recorded in `construct` of the loss function. For example, when a network is defined, image data is recorded in `construct` of the network. When the loss function is defined, the loss value is recorded in `construct` of the loss function.
...@@ -160,11 +155,11 @@ def test_summary(): ...@@ -160,11 +155,11 @@ def test_summary():
summary_writer.close() summary_writer.close()
``` ```
### Recording the Computational Graph After Operator Fusion After completing the script, use the `save_graphs` option of `context` to record the computational graph after operator fusion.
After completing the script by referring to "Basic Writing", use the `save_graphs` option of `context` to record the computational graph after operator fusion.
`ms_output_after_hwopt.pb` is the computational graph after operator fusion. `ms_output_after_hwopt.pb` is the computational graph after operator fusion.
> Currently MindSpore supports recording computational graph after operator fusion for Ascend 910 AI processor only.
## MindInsight Commands ## MindInsight Commands
### View the command help information. ### View the command help information.
...@@ -232,7 +227,7 @@ gunicorn <PID> <USER> <FD> <TYPE> <DEVICE> <SIZE/OFF> <NODE> <WORKSPACE> ...@@ -232,7 +227,7 @@ gunicorn <PID> <USER> <FD> <TYPE> <DEVICE> <SIZE/OFF> <NODE> <WORKSPACE>
## Visualization Components ## Visualization Components
### Computational Graph Visualization ### Computational Graph Visualization
Computational graph visualization is used to display the graph structure, data flow direction, and control flow direction of a computational graph. Computational graph visualization is used to display the graph structure, data flow direction, and control flow direction of a computational graph. It supports visualization of summary log files and pb files generated by `save_graphs` configuration in `context`.
![graph.png](./images/graph.png) ![graph.png](./images/graph.png)
......
...@@ -6,8 +6,6 @@ ...@@ -6,8 +6,6 @@
- [概述](#概述) - [概述](#概述)
- [操作流程](#操作流程) - [操作流程](#操作流程)
- [准备训练脚本](#准备训练脚本) - [准备训练脚本](#准备训练脚本)
- [基础写法](#基础写法)
- [记录算子融合后的计算图](#记录算子融合后的计算图)
- [MindInsight相关命令](#mindinsight相关命令) - [MindInsight相关命令](#mindinsight相关命令)
- [查看命令帮助信息](#查看命令帮助信息) - [查看命令帮助信息](#查看命令帮助信息)
- [查看版本信息](#查看版本信息) - [查看版本信息](#查看版本信息)
...@@ -40,9 +38,6 @@ ...@@ -40,9 +38,6 @@
其中标量、图像是通过Summary算子实现记录数据,计算图是在网络编译完成后,通过 `SummaryRecord` 将其保存到summary日志文件中, 其中标量、图像是通过Summary算子实现记录数据,计算图是在网络编译完成后,通过 `SummaryRecord` 将其保存到summary日志文件中,
模型参数是通过 `TrainLineage``EvalLineage` 保存到summary日志文件中。 模型参数是通过 `TrainLineage``EvalLineage` 保存到summary日志文件中。
### 基础写法
步骤一:在继承 `nn.Cell` 的衍生类的 `construct` 函数中调用Summary算子来采集图像或标量数据。 步骤一:在继承 `nn.Cell` 的衍生类的 `construct` 函数中调用Summary算子来采集图像或标量数据。
比如,在定义网络时,在网络的 `construct` 中记录图像数据;在定义损失函数时,在损失函数的 `construct`中记录损失值。 比如,在定义网络时,在网络的 `construct` 中记录图像数据;在定义损失函数时,在损失函数的 `construct`中记录损失值。
...@@ -165,11 +160,11 @@ def test_summary(): ...@@ -165,11 +160,11 @@ def test_summary():
summary_writer.close() summary_writer.close()
``` ```
### 记录算子融合后的计算图 完成脚本后,可以通过`context``save_graphs`选项配置记录算子融合后的计算图。
参照“基础写法”完成脚本后,可以通过`context``save_graphs`选项配置记录算子融合后的计算图。
其中`ms_output_after_hwopt.pb`为算子融合后的计算图。 其中`ms_output_after_hwopt.pb`为算子融合后的计算图。
> 目前MindSpore仅支持在Ascend 910 AI处理器上导出算子融合后的计算图。
## MindInsight相关命令 ## MindInsight相关命令
### 查看命令帮助信息 ### 查看命令帮助信息
...@@ -237,7 +232,7 @@ gunicorn <PID> <USER> <FD> <TYPE> <DEVICE> <SIZE/OFF> <NODE> <WORKSPACE> ...@@ -237,7 +232,7 @@ gunicorn <PID> <USER> <FD> <TYPE> <DEVICE> <SIZE/OFF> <NODE> <WORKSPACE>
## 可视化组件 ## 可视化组件
### 计算图可视化 ### 计算图可视化
计算图可视化用于展示计算图的图结构,数据流以及控制流的走向。 计算图可视化用于展示计算图的图结构,数据流以及控制流的走向,支持展示summary日志文件与通过`context``save_graphs`参数导出的`pb`文件
![graph.png](./images/graph.png) ![graph.png](./images/graph.png)
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册