mindspore_d_install_en.md 14.2 KB
Newer Older
L
leiyuning 已提交
1 2 3 4 5 6 7 8 9 10 11 12 13
# MindSpore Installation Guide

This document describes how to quickly install MindSpore on an Ascend AI processor environment.

<!-- TOC -->

- [MindSpore Installation Guide](#mindspore-installation-guide)
    - [Environment Requirements](#environment-requirements)
        - [Hardware Requirements](#hardware-requirements)
        - [System Requirements and Software Dependencies](#system-requirements-and-software-dependencies)
        - [(Optional) Installing Conda](#optional-installing-conda)
        - [Configuring software package Dependencies](#configuring-software-package-dependencies)
    - [Installation Guide](#installation-guide)
14
        - [Installing Using Executable Files](#installing-using-executable-files)
L
leiyuning 已提交
15 16 17 18 19 20 21 22 23 24 25 26 27
        - [Installing Using the Source Code](#installing-using-the-source-code)
    - [Configuring Environment Variables](#configuring-environment-variables)
    - [Installation Verification](#installation-verification)
- [Installing MindInsight](#installing-mindinsight)
- [Installing MindArmour](#installing-mindarmour)

<!-- /TOC -->

## Environment Requirements

### Hardware Requirements

- Ascend 910 AI processor
L
leonwanghui 已提交
28 29

  > - Reserve at least 32 GB memory for each card.
L
leiyuning 已提交
30 31 32 33 34

### System Requirements and Software Dependencies

| Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies |
| ---- | :--- | :--- | :--- |
J
jonyguo 已提交
35
| MindSpore master | - Ubuntu 18.04 aarch64 <br> - Ubuntu 18.04 x86_64 <br> - EulerOS 2.8 aarch64 <br> - EulerOS 2.5 x86_64 | - [Python](https://www.python.org/downloads/) 3.7.5 <br> - Ascend 910 AI processor software package(Version:[Atlas Data Center Solution V100R020C00RC1](https://support.huawei.com/enterprise/zh/ascend-computing/atlas-data-center-solution-pid-251167910/software/251638361)) <br> - [gmp](https://gmplib.org/download/gmp/) 6.1.2 <br> - For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindspore/blob/master/requirements.txt). | **Compilation dependencies:**<br> - [Python](https://www.python.org/downloads/) 3.7.5 <br> - Ascend 910 AI processor software package(Version:[Atlas Data Center Solution V100R020C00RC1](https://support.huawei.com/enterprise/zh/ascend-computing/atlas-data-center-solution-pid-251167910/software/251638361)) <br> - [wheel](https://pypi.org/project/wheel/) >= 0.32.0 <br> - [GCC](https://gcc.gnu.org/releases.html) 7.3.0 <br> - [CMake](https://cmake.org/download/) >= 3.14.1 <br> - [patch](http://ftp.gnu.org/gnu/patch/) >= 2.5 <br> - [gmp](https://gmplib.org/download/gmp/) 6.1.2 <br> **Installation dependencies:**<br> same as the executable file installation dependencies. |
L
leiyuning 已提交
36

J
jonyguo 已提交
37
- Confirm that the current user has the right to access the installation path `/usr/local/Ascend `of Ascend 910 AI processor software package(Version:[Atlas Data Center Solution V100R020C00RC1](https://support.huawei.com/enterprise/zh/ascend-computing/atlas-data-center-solution-pid-251167910/software/251638361)). If not, the root user needs to add the current user to the user group where `/usr/local/Ascend` is located. For the specific configuration, please refer to the software package instruction document.
38
- GCC 7.3.0 can be installed by using apt command.
L
lvmingfu 已提交
39
- When the network is connected, dependency items in the `requirements.txt` file are automatically downloaded during .whl package installation. In other cases, you need to manually install dependency items.
L
leiyuning 已提交
40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58

### (Optional) Installing Conda

1. Download the Conda installation package from the following path:

   - [X86 Anaconda](https://www.anaconda.com/distribution/) or [X86 Miniconda](https://docs.conda.io/en/latest/miniconda.html)
   - [ARM Anaconda](https://github.com/Archiconda/build-tools/releases/download/0.2.3/Archiconda3-0.2.3-Linux-aarch64.sh)

2. Create and activate the Python environment.

    ```bash
    conda create -n {your_env_name} python=3.7.5
    conda activate {your_env_name}
    ```

> Conda is a powerful Python environment management tool. It is recommended that a beginner read related information on the Internet first.

### Configuring software package Dependencies

J
jonyguo 已提交
59
 - Install the .whl package provided in Ascend 910 AI processor software package(Version:[Atlas Data Center Solution V100R020C00RC1](https://support.huawei.com/enterprise/zh/ascend-computing/atlas-data-center-solution-pid-251167910/software/251638361)). The .whl package is released with the software package. After software package is upgraded, reinstall the .whl package.
L
leiyuning 已提交
60 61

    ```bash
C
caifubi 已提交
62 63 64
    pip install /usr/local/Ascend/fwkacllib/lib64/topi-{version}-py3-none-any.whl
    pip install /usr/local/Ascend/fwkacllib/lib64/te-{version}-py3-none-any.whl
    pip install /usr/local/Ascend/fwkacllib/lib64/hccl-{version}-py3-none-any.whl
L
leiyuning 已提交
65 66 67 68
    ```

## Installation Guide

69 70 71 72 73
### Installing Using Executable Files

- Download the .whl package from the [MindSpore website](https://www.mindspore.cn/versions/en). It is recommended to perform SHA-256 integrity verification first and run the following command to install MindSpore:

    ```bash
W
wukesong 已提交
74
    pip install mindspore_ascend-{version}-cp37-cp37m-linux_{arch}.whl
75 76
    ```

L
leiyuning 已提交
77 78 79 80 81 82 83 84 85
### Installing Using the Source Code

The compilation and installation must be performed on the Ascend 910 AI processor environment.

1. Download the source code from the code repository.

    ```bash
    git clone https://gitee.com/mindspore/mindspore.git
    ```
L
leonwanghui 已提交
86

L
leiyuning 已提交
87 88 89
2. Run the following command in the root directory of the source code to compile MindSpore:

    ```bash
90
    bash build.sh -e ascend
L
leiyuning 已提交
91
    ```
L
lvmingfu 已提交
92 93
    > - Before running the preceding command, ensure that the paths where the executable files `cmake` and `patch` store have been added to the environment variable PATH.
    > - In the `build.sh` script, the `git clone` command will be executed to obtain the code in the third-party dependency database. Ensure that the network settings of Git are correct.
94
    > - In the `build.sh` script, the default number of compilation threads is 8. If the compiler performance is poor, compilation errors may occur. You can add -j{Number of threads} in to script to reduce the number of threads. For example, `bash build.sh -e ascend -j4`.
L
leiyuning 已提交
95 96 97 98

3. Run the following command to install MindSpore:

    ```bash
W
wukesong 已提交
99 100
    chmod +x build/package/mindspore_ascend-{version}-cp37-cp37m-linux_{arch}.whl
    pip install build/package/mindspore_ascend-{version}-cp37-cp37m-linux_{arch}.whl
L
leiyuning 已提交
101 102 103 104
    ```

## Configuring Environment Variables

105
- In EulerOS, after MindSpore is installed, export runtime-related environment variables.
L
leonwanghui 已提交
106

L
leiyuning 已提交
107 108 109
    ```bash
    # control log level. 0-DEBUG, 1-INFO, 2-WARNING, 3-ERROR, default level is WARNING.
    export GLOG_v=2
110

W
wukesong 已提交
111 112
    # Conda environmental options
    LOCAL_ASCEND=/usr/local/Ascend # the root directory of run package
113

L
leiyuning 已提交
114
    # lib libraries that the run package depends on
C
caifubi 已提交
115
    export LD_LIBRARY_PATH=${LOCAL_ASCEND}/add-ons/:${LOCAL_ASCEND}/fwkacllib/lib64:${LD_LIBRARY_PATH}
116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134

    # Environment variables that must be configured
    export TBE_IMPL_PATH=${LOCAL_ASCEND}/opp/op_impl/built-in/ai_core/tbe  # TBE operator implementation tool path
    export PATH=${LOCAL_ASCEND}/fwkacllib/ccec_compiler/bin/:${PATH}       # TBE operator compilation tool path
    export PYTHONPATH=${TBE_IMPL_PATH}:${PYTHONPATH}                       # Python library that TBE implementation depends on
    ```

- In Ubuntu, after MindSpore is installed, export runtime-related environment variables. Note: you need to replace {version} in the following configuration with the actual version number on the environment.

    ```bash
    # control log level. 0-DEBUG, 1-INFO, 2-WARNING, 3-ERROR, default level is WARNING.
    export GLOG_v=2

    # Conda environmental options
    LOCAL_ASCEND=/usr/local/Ascend # the root directory of run package

    # lib libraries that the run package depends on
    export LD_LIBRARY_PATH=${LOCAL_ASCEND}/add-ons/:${LOCAL_ASCEND}/ascend-toolkit/{version}/fwkacllib/lib64:${LOCAL_ASCEND}/driver/lib64:${LD_LIBRARY_PATH}

W
wukesong 已提交
135
    # Environment variables that must be configured
136 137 138
    export TBE_IMPL_PATH=${LOCAL_ASCEND}/ascend-toolkit/{version}/opp/op_impl/built-in/ai_core/tbe  # TBE operator implementation tool path
    export PATH=${LOCAL_ASCEND}/ascend-toolkit/{version}/fwkacllib/ccec_compiler/bin/:${PATH}       # TBE operator compilation tool path
    export PYTHONPATH=${TBE_IMPL_PATH}:${PYTHONPATH}                                                # Python library that TBE implementation depends on
L
leiyuning 已提交
139 140 141 142 143
    ```

## Installation Verification

- After configuring the environment variables, execute the following Python script:
L
leonwanghui 已提交
144

L
leiyuning 已提交
145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182
    ```bash
    import numpy as np
    from mindspore import Tensor
    from mindspore.ops import functional as F
    import mindspore.context as context

    context.set_context(device_target="Ascend")
    x = Tensor(np.ones([1,3,3,4]).astype(np.float32))
    y = Tensor(np.ones([1,3,3,4]).astype(np.float32))
    print(F.tensor_add(x, y))
    ```

- The outputs should be same as:

    ```
    [[[ 2.  2.  2.  2.],
      [ 2.  2.  2.  2.],
      [ 2.  2.  2.  2.]],

     [[ 2.  2.  2.  2.],
      [ 2.  2.  2.  2.],
      [ 2.  2.  2.  2.]],

     [[ 2.  2.  2.  2.],
      [ 2.  2.  2.  2.],
      [ 2.  2.  2.  2.]]]
    ```

# Installing MindInsight

If you need to analyze information such as model scalars, graphs, and model traceback, you can install MindInsight.

## Environment Requirements

### System Requirements and Software Dependencies

| Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies |
| ---- | :--- | :--- | :--- |
183
| MindInsight master | - Ubuntu 18.04 aarch64 <br> - Ubuntu 18.04 x86_64 <br> - EulerOS 2.8 aarch64 <br> - EulerOS 2.5 x86_64 <br> | - [Python](https://www.python.org/downloads/) 3.7.5 <br> - MindSpore master <br> - For details about other dependency items, see [requirements.txt](https://gitee.com/mindspore/mindinsight/blob/master/requirements.txt). | **Compilation dependencies:**<br> - [Python](https://www.python.org/downloads/) 3.7.5 <br> - [CMake](https://cmake.org/download/) >= 3.14.1 <br> - [GCC](https://gcc.gnu.org/releases.html) 7.3.0 <br> - [node.js](https://nodejs.org/en/download/) >= 10.19.0 <br> - [wheel](https://pypi.org/project/wheel/) >= 0.32.0 <br> - [pybind11](https://pypi.org/project/pybind11/) >= 2.4.3 <br> **Installation dependencies:**<br> same as the executable file installation dependencies. |
L
leiyuning 已提交
184

L
lvmingfu 已提交
185
- When the network is connected, dependency items in the `requirements.txt` file are automatically downloaded during .whl package installation. In other cases, you need to manually install dependency items.
L
leiyuning 已提交
186 187 188

## Installation Guide

189 190 191 192 193 194 195 196 197 198 199 200 201 202
### Installing Using Executable Files

1. Download the .whl package from the [MindSpore website](https://www.mindspore.cn/versions/en). It is recommended to perform SHA-256 integrity verification first  and run the following command to install MindInsight:

    ```bash
    pip install mindinsight-{version}-cp37-cp37m-linux_{arch}.whl
    ```

2. Run the following command. If `web address: http://127.0.0.1:8080` is displayed, the installation is successful.

    ```bash
    mindinsight start
    ```

L
leiyuning 已提交
203 204 205 206 207 208 209
### Installing Using the Source Code

1. Download the source code from the code repository.

    ```bash
    git clone https://gitee.com/mindspore/mindinsight.git
    ```
李鸿章 已提交
210
    > You are **not** supposed to obtain the source code from the zip package downloaded from the repository homepage.
L
leiyuning 已提交
211 212 213 214 215 216 217 218 219 220

2. Install MindInsight by using either of the following installation methods:

   (1) Access the root directory of the source code and run the following installation command:

      ```bash
      cd mindinsight
      pip install -r requirements.txt
      python setup.py install
      ```
L
leonwanghui 已提交
221

L
leiyuning 已提交
222
   (2) Create a .whl package to install MindInsight.
L
leonwanghui 已提交
223

李鸿章 已提交
224
      Access the root directory of the source code.
L
lvmingfu 已提交
225 226
      First run the MindInsight compilation script under the `build` directory of the source code.
      Then run the command to install the .whl package generated into the `output` directory of the source code.
L
leiyuning 已提交
227 228

      ```bash
李鸿章 已提交
229 230 231
      cd mindinsight
      bash build/build.sh
      pip install output/mindinsight-{version}-cp37-cp37m-linux_{arch}.whl
L
leiyuning 已提交
232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249
      ```

3. Run the following command. If `web address: http://127.0.0.1:8080` is displayed, the installation is successful.

    ```bash
    mindinsight start
    ```

# Installing MindArmour

If you need to conduct AI model security research or enhance the security of the model in you applications, you can install MindArmour.

## Environment Requirements

### System Requirements and Software Dependencies

| Version | Operating System | Executable File Installation Dependencies | Source Code Compilation and Installation Dependencies |
| ---- | :--- | :--- | :--- |
250
| MindArmour master | - Ubuntu 18.04 aarch64 <br> - Ubuntu 18.04 x86_64 <br> - EulerOS 2.8 aarch64 <br> - EulerOS 2.5 x86_64 <br> | - [Python](https://www.python.org/downloads/) 3.7.5 <br> - MindSpore master <br> - For details about other dependency items, see [setup.py](https://gitee.com/mindspore/mindarmour/blob/master/setup.py). | Same as the executable file installation dependencies. |
L
leiyuning 已提交
251

L
lvmingfu 已提交
252
- When the network is connected, dependency items in the `setup.py` file are automatically downloaded during .whl package installation. In other cases, you need to manually install dependency items.
L
leiyuning 已提交
253 254 255

## Installation Guide

256 257 258 259 260 261 262 263 264 265 266 267 268 269
### Installing Using Executable Files

1. Download the .whl package from the [MindSpore website](https://www.mindspore.cn/versions/en). It is recommended to perform SHA-256 integrity verification first  and run the following command to install MindArmour:

   ```bash
   pip install mindarmour-{version}-cp37-cp37m-linux_{arch}.whl
   ```

2. Run the following command. If no loading error message such as `No module named 'mindarmour'` is displayed, the installation is successful.

   ```bash
   python -c 'import mindarmour'
   ```

L
leiyuning 已提交
270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290
### Installing Using the Source Code

1. Download the source code from the code repository.

   ```bash
   git clone https://gitee.com/mindspore/mindarmour.git
   ```

2. Run the following command in the root directory of the source code to compile and install MindArmour:

   ```bash
   cd mindarmour
   python setup.py install
   ```


3. Run the following command. If no loading error message such as `No module named 'mindarmour'` is displayed, the installation is successful.

   ```bash
   python -c 'import mindarmour'
   ```