initial version

.gitee/PULL_REQUEST_TEMPLATE.md

+<!--  Thanks for sending a pull request!  Here are some tips for you:
+
+If this is your first time, please read our contributor guidelines: https://gitee.com/mindspore/mindspore/blob/master/CONTRIBUTING.md
+-->
+
+**What type of PR is this?**
+> Uncomment only one ` /kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line:
+>
+> /kind bug
+> /kind task
+> /kind feature
+
+
+**What this PR does / why we need it**:
+
+
+**Which issue(s) this PR fixes**:
+<!-- 
+*Automatically closes linked issue when PR is merged.
+Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
+-->
+Fixes #
+
+**Special notes for your reviewer**:
+

.gitignore

+*.dot
+*.ir
+*.dat
+*.pyc
+*.csv
+*.gz
+*.tar
+*.zip
+*.rar
+*.ipynb
+.idea/
+build/
+dist/
+local_script/
+example/dataset/
+example/mnist_demo/MNIST_unzip/
+example/mnist_demo/trained_ckpt_file/
+example/mnist_demo/model/
+example/cifar_demo/model/
+example/dog_cat_demo/model/
+mindarmour.egg-info/
+*model/
+*MNIST/
+*out.data/
+*defensed_model/
+*pre_trained_model/
+*__pycache__/
+*kernel_meta

LICENSE

+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

NOTICE

+MindSpore MindArmour
+Copyright 2019-2020 Huawei Technologies Co., Ltd
\ No newline at end of file

README.md

+# MindArmour
+
+- [What is MindArmour](#what-is-mindarmour)
+- [Setting up](#setting-up-mindarmour)
+- [Docs](#docs)
+- [Community](#community)
+- [Contributing](#contributing)
+- [Release Notes](#release-notes)
+- [License](#license)
+
+## What is MindArmour
+
+A tool box for MindSpore users to enhance model security and trustworthiness.
+
+MindArmour is designed for adversarial examples, including four submodule: adversarial examples generation, adversarial example detection, model defense and evaluation. The architecture is shown as follow：
+
+![mindarmour_architecture](docs/mindarmour_architecture.png)
+
+## Setting up MindArmour
+
+### Dependencies
+
+This library uses MindSpore to accelerate graph computations performed by many machine learning models. Therefore, installing MindSpore is a pre-requisite.  All other dependencies are included in `setup.py`.
+
+### Installation
+
+#### Installation for development
+
+1. Download source code from Gitee.
+
+```bash
+git clone https://gitee.com/mindspore/mindarmour.git
+```
+
+2. Compile and install in MindArmour directory. 
+
+```bash
+$ cd mindarmour
+$ python setup.py install
+```
+
+#### `Pip` installation
+
+1. Download whl package from [MindSpore website](https://www.mindspore.cn/versions/en), then run the following command:
+
+```
+pip install mindarmour-{version}-cp37-cp37m-linux_{arch}.whl
+```
+
+2. Successfully installed, if there is no error message such as `No module named 'mindarmour'` when execute the following command:
+
+```bash
+python -c 'import mindarmour'
+```
+
+## Docs
+
+Guidance on installation, tutorials, API, see our [User Documentation](https://gitee.com/mindspore/docs).
+
+## Community
+
+- [MindSpore Slack](https://join.slack.com/t/mindspore/shared_invite/enQtOTcwMTIxMDI3NjM0LTNkMWM2MzI5NjIyZWU5ZWQ5M2EwMTQ5MWNiYzMxOGM4OWFhZjI4M2E5OGI2YTg3ODU1ODE2Njg1MThiNWI3YmQ) - Ask questions and find answers.
+
+## Contributing
+
+Welcome contributions. See our [Contributor Wiki](https://gitee.com/mindspore/mindspore/blob/master/CONTRIBUTING.md) for more details.
+
+## Release Notes
+
+The release notes, see our [RELEASE](RELEASE.md).
+
+## License
+
+[Apache License 2.0](LICENSE)

RELEASE.md

+# Release 0.1.0-alpha
+
+Initial release of MindArmour.
+
+## Major Features
+
+- Support adversarial attack and defense on the platform of MindSpore.
+- Include 13 white-box and 7 black-box attack methods.
+- Provide 5 detection algorithms to detect attacking in multiple way.
+- Provide adversarial training to enhance model security.
+- Provide 6 evaluation metrics for attack methods and 9 evaluation metrics for defense methods.
\ No newline at end of file

docs/README.md

+# MindArmour Documentation
+
+The MindArmour documentation is in the [MindSpore Docs](https://gitee.com/mindspore/docs) repository.

example/data_processing.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import mindspore.dataset as ds
+import mindspore.dataset.transforms.vision.c_transforms as CV
+import mindspore.dataset.transforms.c_transforms as C
+from mindspore.dataset.transforms.vision import Inter
+import mindspore.common.dtype as mstype
+
+
+def generate_mnist_dataset(data_path, batch_size=32, repeat_size=1,
+                           num_parallel_workers=1, sparse=True):
+    """
+    create dataset for training or testing
+    """
+    # define dataset
+    ds1 = ds.MnistDataset(data_path)
+
+    # define operation parameters
+    resize_height, resize_width = 32, 32
+    rescale = 1.0 / 255.0
+    shift = 0.0
+
+    # define map operations
+    resize_op = CV.Resize((resize_height, resize_width),
+                          interpolation=Inter.LINEAR)
+    rescale_op = CV.Rescale(rescale, shift)
+    hwc2chw_op = CV.HWC2CHW()
+    type_cast_op = C.TypeCast(mstype.int32)
+    one_hot_enco = C.OneHot(10)
+
+    # apply map operations on images
+    if not sparse:
+        ds1 = ds1.map(input_columns="label", operations=one_hot_enco,
+                      num_parallel_workers=num_parallel_workers)
+        type_cast_op = C.TypeCast(mstype.float32)
+    ds1 = ds1.map(input_columns="label", operations=type_cast_op,
+                  num_parallel_workers=num_parallel_workers)
+    ds1 = ds1.map(input_columns="image", operations=resize_op,
+                  num_parallel_workers=num_parallel_workers)
+    ds1 = ds1.map(input_columns="image", operations=rescale_op,
+                  num_parallel_workers=num_parallel_workers)
+    ds1 = ds1.map(input_columns="image", operations=hwc2chw_op,
+                  num_parallel_workers=num_parallel_workers)
+
+    # apply DatasetOps
+    buffer_size = 10000
+    ds1 = ds1.shuffle(buffer_size=buffer_size)
+    ds1 = ds1.batch(batch_size, drop_remainder=True)
+    ds1 = ds1.repeat(repeat_size)
+
+    return ds1

example/mnist_demo/README.md

+# mnist demo
+## Introduction
+
+The MNIST database of handwritten digits, available from this page, has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from MNIST. The digits have been size-normalized and centered in a fixed-size image.
+
+## run demo
+
+### 1. download dataset
+```sh
+$ cd example/mnist_demo
+$ mkdir MNIST_unzip
+$ cd MNIST_unzip
+$ mkdir train
+$ mkdir test
+$ cd train
+$ wget "http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz"
+$ wget "http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz"
+$ gzip train-images-idx3-ubyte.gz -d
+$ gzip train-labels-idx1-ubyte.gz -d
+$ cd ../test
+$ wget "http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz"
+$ wget "http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz"
+$ gzip t10k-images-idx3-ubyte.gz -d
+$ gzip t10k-images-idx3-ubyte.gz -d
+$ cd ../../
+```
+
+### 1. trian model
+```sh
+$ python mnist_train.py
+
+```
+
+### 2. run attack test
+```sh
+$ mkdir out.data
+$ python mnist_attack_jsma.py
+
+```
+
+### 3. run defense/detector test
+```sh
+$ python mnist_defense_nad.py
+$ python mnist_similarity_detector.py
+
+```

example/mnist_demo/lenet5_net.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import mindspore.nn as nn
+import mindspore.ops.operations as P
+from mindspore.common.initializer import TruncatedNormal
+
+
+def conv(in_channels, out_channels, kernel_size, stride=1, padding=0):
+    weight = weight_variable()
+    return nn.Conv2d(in_channels, out_channels,
+                     kernel_size=kernel_size, stride=stride, padding=padding,
+                     weight_init=weight, has_bias=False, pad_mode="valid")
+
+
+def fc_with_initialize(input_channels, out_channels):
+    weight = weight_variable()
+    bias = weight_variable()
+    return nn.Dense(input_channels, out_channels, weight, bias)
+
+
+def weight_variable():
+    return TruncatedNormal(0.2)
+
+
+class LeNet5(nn.Cell):
+    """
+    Lenet network
+    """
+    def __init__(self):
+        super(LeNet5, self).__init__()
+        self.conv1 = conv(1, 6, 5)
+        self.conv2 = conv(6, 16, 5)
+        self.fc1 = fc_with_initialize(16*5*5, 120)
+        self.fc2 = fc_with_initialize(120, 84)
+        self.fc3 = fc_with_initialize(84, 10)
+        self.relu = nn.ReLU()
+        self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2)
+        self.reshape = P.Reshape()
+
+    def construct(self, x):
+        x = self.conv1(x)
+        x = self.relu(x)
+        x = self.max_pool2d(x)
+        x = self.conv2(x)
+        x = self.relu(x)
+        x = self.max_pool2d(x)
+        x = self.reshape(x, (-1, 16*5*5))
+        x = self.fc1(x)
+        x = self.relu(x)
+        x = self.fc2(x)
+        x = self.relu(x)
+        x = self.fc3(x)
+        return x

example/mnist_demo/mnist_attack_cw.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import time
+import numpy as np
+import pytest
+from scipy.special import softmax
+
+from mindspore import Model
+from mindspore import Tensor
+from mindspore import context
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks.carlini_wagner import CarliniWagnerL2Attack
+from mindarmour.utils.logger import LogUtil
+from mindarmour.evaluations.attack_evaluation import AttackEvaluate
+
+from lenet5_net import LeNet5
+
+context.set_context(mode=context.GRAPH_MODE, device_target="Ascend")
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+LOGGER = LogUtil.get_instance()
+TAG = 'CW_Test'
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_carlini_wagner_attack():
+    """
+    CW-Attack test
+    """
+    # upload trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+
+    # get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds = generate_mnist_dataset(data_list, batch_size=batch_size)
+
+    # prediction accuracy before attack
+    model = Model(net)
+    batch_num = 3  # the number of batches of attacking samples
+    test_images = []
+    test_labels = []
+    predict_labels = []
+    i = 0
+    for data in ds.create_tuple_iterator():
+        i += 1
+        images = data[0].astype(np.float32)
+        labels = data[1]
+        test_images.append(images)
+        test_labels.append(labels)
+        pred_labels = np.argmax(model.predict(Tensor(images)).asnumpy(),
+                                axis=1)
+        predict_labels.append(pred_labels)
+        if i >= batch_num:
+            break
+    predict_labels = np.concatenate(predict_labels)
+    true_labels = np.concatenate(test_labels)
+    accuracy = np.mean(np.equal(predict_labels, true_labels))
+    LOGGER.info(TAG, "prediction accuracy before attacking is : %s", accuracy)
+
+    # attacking
+    num_classes = 10
+    attack = CarliniWagnerL2Attack(net, num_classes, targeted=False)
+    start_time = time.clock()
+    adv_data = attack.batch_generate(np.concatenate(test_images),
+                                     np.concatenate(test_labels), batch_size=32)
+    stop_time = time.clock()
+    pred_logits_adv = model.predict(Tensor(adv_data)).asnumpy()
+    # rescale predict confidences into (0, 1).
+    pred_logits_adv = softmax(pred_logits_adv, axis=1)
+    pred_labels_adv = np.argmax(pred_logits_adv, axis=1)
+    accuracy_adv = np.mean(np.equal(pred_labels_adv, true_labels))
+    LOGGER.info(TAG, "prediction accuracy after attacking is : %s",
+                accuracy_adv)
+    test_labels = np.eye(10)[np.concatenate(test_labels)]
+    attack_evaluate = AttackEvaluate(np.concatenate(test_images).transpose(0, 2, 3, 1),
+                                     test_labels, adv_data.transpose(0, 2, 3, 1),
+                                     pred_logits_adv)
+    LOGGER.info(TAG, 'mis-classification rate of adversaries is : %s',
+                attack_evaluate.mis_classification_rate())
+    LOGGER.info(TAG, 'The average confidence of adversarial class is : %s',
+                attack_evaluate.avg_conf_adv_class())
+    LOGGER.info(TAG, 'The average confidence of true class is : %s',
+                attack_evaluate.avg_conf_true_class())
+    LOGGER.info(TAG, 'The average distance (l0, l2, linf) between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_lp_distance())
+    LOGGER.info(TAG, 'The average structural similarity between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_ssim())
+    LOGGER.info(TAG, 'The average costing time is %s',
+                (stop_time - start_time)/(batch_num*batch_size))
+
+
+if __name__ == '__main__':
+    test_carlini_wagner_attack()

example/mnist_demo/mnist_attack_deepfool.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import time
+import numpy as np
+import pytest
+from scipy.special import softmax
+
+from mindspore import Model
+from mindspore import Tensor
+from mindspore import context
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks.deep_fool import DeepFool
+from mindarmour.utils.logger import LogUtil
+from mindarmour.evaluations.attack_evaluation import AttackEvaluate
+
+from lenet5_net import LeNet5
+
+context.set_context(mode=context.GRAPH_MODE, device_target="Ascend")
+
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+LOGGER = LogUtil.get_instance()
+TAG = 'DeepFool_Test'
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_deepfool_attack():
+    """
+    DeepFool-Attack test
+    """
+    # upload trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+
+    # get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds = generate_mnist_dataset(data_list, batch_size=batch_size)
+
+    # prediction accuracy before attack
+    model = Model(net)
+    batch_num = 3  # the number of batches of attacking samples
+    test_images = []
+    test_labels = []
+    predict_labels = []
+    i = 0
+    for data in ds.create_tuple_iterator():
+        i += 1
+        images = data[0].astype(np.float32)
+        labels = data[1]
+        test_images.append(images)
+        test_labels.append(labels)
+        pred_labels = np.argmax(model.predict(Tensor(images)).asnumpy(),
+                                axis=1)
+        predict_labels.append(pred_labels)
+        if i >= batch_num:
+            break
+    predict_labels = np.concatenate(predict_labels)
+    true_labels = np.concatenate(test_labels)
+    accuracy = np.mean(np.equal(predict_labels, true_labels))
+    LOGGER.info(TAG, "prediction accuracy before attacking is : %s", accuracy)
+
+    # attacking
+    classes = 10
+    attack = DeepFool(net, classes, norm_level=2,
+                      bounds=(0.0, 1.0))
+    start_time = time.clock()
+    adv_data = attack.batch_generate(np.concatenate(test_images),
+                                     np.concatenate(test_labels), batch_size=32)
+    stop_time = time.clock()
+    pred_logits_adv = model.predict(Tensor(adv_data)).asnumpy()
+    # rescale predict confidences into (0, 1).
+    pred_logits_adv = softmax(pred_logits_adv, axis=1)
+    pred_labels_adv = np.argmax(pred_logits_adv, axis=1)
+    accuracy_adv = np.mean(np.equal(pred_labels_adv, true_labels))
+    LOGGER.info(TAG, "prediction accuracy after attacking is : %s",
+                accuracy_adv)
+    test_labels = np.eye(10)[np.concatenate(test_labels)]
+    attack_evaluate = AttackEvaluate(np.concatenate(test_images).transpose(0, 2, 3, 1),
+                                     test_labels, adv_data.transpose(0, 2, 3, 1),
+                                     pred_logits_adv)
+    LOGGER.info(TAG, 'mis-classification rate of adversaries is : %s',
+                attack_evaluate.mis_classification_rate())
+    LOGGER.info(TAG, 'The average confidence of adversarial class is : %s',
+                attack_evaluate.avg_conf_adv_class())
+    LOGGER.info(TAG, 'The average confidence of true class is : %s',
+                attack_evaluate.avg_conf_true_class())
+    LOGGER.info(TAG, 'The average distance (l0, l2, linf) between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_lp_distance())
+    LOGGER.info(TAG, 'The average structural similarity between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_ssim())
+    LOGGER.info(TAG, 'The average costing time is %s',
+                (stop_time - start_time)/(batch_num*batch_size))
+
+
+if __name__ == '__main__':
+    test_deepfool_attack()

example/mnist_demo/mnist_attack_fgsm.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import time
+import numpy as np
+import pytest
+from scipy.special import softmax
+
+from mindspore import Model
+from mindspore import Tensor
+from mindspore import context
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks.gradient_method import FastGradientSignMethod
+
+from mindarmour.utils.logger import LogUtil
+from mindarmour.evaluations.attack_evaluation import AttackEvaluate
+
+from lenet5_net import LeNet5
+
+context.set_context(mode=context.GRAPH_MODE, device_target="Ascend")
+
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+LOGGER = LogUtil.get_instance()
+TAG = 'FGSM_Test'
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_fast_gradient_sign_method():
+    """
+    FGSM-Attack test
+    """
+    # upload trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+
+    # get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds = generate_mnist_dataset(data_list, batch_size, sparse=False)
+
+    # prediction accuracy before attack
+    model = Model(net)
+    batch_num = 3  # the number of batches of attacking samples
+    test_images = []
+    test_labels = []
+    predict_labels = []
+    i = 0
+    for data in ds.create_tuple_iterator():
+        i += 1
+        images = data[0].astype(np.float32)
+        labels = data[1]
+        test_images.append(images)
+        test_labels.append(labels)
+        pred_labels = np.argmax(model.predict(Tensor(images)).asnumpy(),
+                                axis=1)
+        predict_labels.append(pred_labels)
+        if i >= batch_num:
+            break
+    predict_labels = np.concatenate(predict_labels)
+    true_labels = np.argmax(np.concatenate(test_labels), axis=1)
+    accuracy = np.mean(np.equal(predict_labels, true_labels))
+    LOGGER.info(TAG, "prediction accuracy before attacking is : %s", accuracy)
+
+    # attacking
+    attack = FastGradientSignMethod(net, eps=0.3)
+    start_time = time.clock()
+    adv_data = attack.batch_generate(np.concatenate(test_images),
+                                     np.concatenate(test_labels), batch_size=32)
+    stop_time = time.clock()
+    np.save('./adv_data', adv_data)
+    pred_logits_adv = model.predict(Tensor(adv_data)).asnumpy()
+    # rescale predict confidences into (0, 1).
+    pred_logits_adv = softmax(pred_logits_adv, axis=1)
+    pred_labels_adv = np.argmax(pred_logits_adv, axis=1)
+    accuracy_adv = np.mean(np.equal(pred_labels_adv, true_labels))
+    LOGGER.info(TAG, "prediction accuracy after attacking is : %s", accuracy_adv)
+    attack_evaluate = AttackEvaluate(np.concatenate(test_images).transpose(0, 2, 3, 1),
+                                     np.concatenate(test_labels),
+                                     adv_data.transpose(0, 2, 3, 1),
+                                     pred_logits_adv)
+    LOGGER.info(TAG, 'mis-classification rate of adversaries is : %s',
+                attack_evaluate.mis_classification_rate())
+    LOGGER.info(TAG, 'The average confidence of adversarial class is : %s',
+                attack_evaluate.avg_conf_adv_class())
+    LOGGER.info(TAG, 'The average confidence of true class is : %s',
+                attack_evaluate.avg_conf_true_class())
+    LOGGER.info(TAG, 'The average distance (l0, l2, linf) between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_lp_distance())
+    LOGGER.info(TAG, 'The average structural similarity between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_ssim())
+    LOGGER.info(TAG, 'The average costing time is %s',
+                (stop_time - start_time)/(batch_num*batch_size))
+
+
+if __name__ == '__main__':
+    test_fast_gradient_sign_method()

example/mnist_demo/mnist_attack_genetic.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import time
+import numpy as np
+import pytest
+from scipy.special import softmax
+
+from mindspore import Tensor
+from mindspore import context
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks.black.genetic_attack import GeneticAttack
+from mindarmour.attacks.black.black_model import BlackModel
+from mindarmour.utils.logger import LogUtil
+from mindarmour.evaluations.attack_evaluation import AttackEvaluate
+
+from lenet5_net import LeNet5
+
+context.set_context(mode=context.GRAPH_MODE, device_target="Ascend")
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+LOGGER = LogUtil.get_instance()
+TAG = 'Genetic_Attack'
+
+
+class ModelToBeAttacked(BlackModel):
+    """model to be attack"""
+
+    def __init__(self, network):
+        super(ModelToBeAttacked, self).__init__()
+        self._network = network
+
+    def predict(self, inputs):
+        """predict"""
+        result = self._network(Tensor(inputs.astype(np.float32)))
+        return result.asnumpy()
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_genetic_attack_on_mnist():
+    """
+    Genetic-Attack test
+    """
+    # upload trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+
+    # get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds = generate_mnist_dataset(data_list, batch_size=batch_size)
+
+    # prediction accuracy before attack
+    model = ModelToBeAttacked(net)
+    batch_num = 3  # the number of batches of attacking samples
+    test_images = []
+    test_labels = []
+    predict_labels = []
+    i = 0
+    for data in ds.create_tuple_iterator():
+        i += 1
+        images = data[0].astype(np.float32)
+        labels = data[1]
+        test_images.append(images)
+        test_labels.append(labels)
+        pred_labels = np.argmax(model.predict(images), axis=1)
+        predict_labels.append(pred_labels)
+        if i >= batch_num:
+            break
+    predict_labels = np.concatenate(predict_labels)
+    true_labels = np.concatenate(test_labels)
+    accuracy = np.mean(np.equal(predict_labels, true_labels))
+    LOGGER.info(TAG, "prediction accuracy before attacking is : %g", accuracy)
+
+    # attacking
+    attack = GeneticAttack(model=model, pop_size=6, mutation_rate=0.05,
+                           per_bounds=0.1, step_size=0.25, temp=0.1,
+                           sparse=True)
+    targeted_labels = np.random.randint(0, 10, size=len(true_labels))
+    for i in range(len(true_labels)):
+        if targeted_labels[i] == true_labels[i]:
+            targeted_labels[i] = (targeted_labels[i] + 1) % 10
+    start_time = time.clock()
+    success_list, adv_data, query_list = attack.generate(
+        np.concatenate(test_images), targeted_labels)
+    stop_time = time.clock()
+    LOGGER.info(TAG, 'success_list: %s', success_list)
+    LOGGER.info(TAG, 'average of query times is : %s', np.mean(query_list))
+    pred_logits_adv = model.predict(adv_data)
+    # rescale predict confidences into (0, 1).
+    pred_logits_adv = softmax(pred_logits_adv, axis=1)
+    pred_lables_adv = np.argmax(pred_logits_adv, axis=1)
+    accuracy_adv = np.mean(np.equal(pred_lables_adv, true_labels))
+    LOGGER.info(TAG, "prediction accuracy after attacking is : %g",
+                accuracy_adv)
+    test_labels_onehot = np.eye(10)[true_labels]
+    attack_evaluate = AttackEvaluate(np.concatenate(test_images),
+                                     test_labels_onehot, adv_data,
+                                     pred_logits_adv, targeted=True,
+                                     target_label=targeted_labels)
+    LOGGER.info(TAG, 'mis-classification rate of adversaries is : %s',
+                attack_evaluate.mis_classification_rate())
+    LOGGER.info(TAG, 'The average confidence of adversarial class is : %s',
+                attack_evaluate.avg_conf_adv_class())
+    LOGGER.info(TAG, 'The average confidence of true class is : %s',
+                attack_evaluate.avg_conf_true_class())
+    LOGGER.info(TAG, 'The average distance (l0, l2, linf) between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_lp_distance())
+    LOGGER.info(TAG, 'The average structural similarity between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_ssim())
+    LOGGER.info(TAG, 'The average costing time is %s',
+                (stop_time - start_time)/(batch_num*batch_size))
+
+
+if __name__ == '__main__':
+    test_genetic_attack_on_mnist()

example/mnist_demo/mnist_attack_hsja.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import numpy as np
+import pytest
+
+from mindspore import Tensor
+from mindspore import context
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks.black.hop_skip_jump_attack import HopSkipJumpAttack
+from mindarmour.attacks.black.black_model import BlackModel
+
+from mindarmour.utils.logger import LogUtil
+from lenet5_net import LeNet5
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+context.set_context(mode=context.GRAPH_MODE)
+context.set_context(device_target="Ascend")
+
+LOGGER = LogUtil.get_instance()
+TAG = 'HopSkipJumpAttack'
+
+
+class ModelToBeAttacked(BlackModel):
+    """model to be attack"""
+
+    def __init__(self, network):
+        super(ModelToBeAttacked, self).__init__()
+        self._network = network
+
+    def predict(self, inputs):
+        """predict"""
+        if len(inputs.shape) == 3:
+            inputs = inputs[np.newaxis, :]
+        result = self._network(Tensor(inputs.astype(np.float32)))
+        return result.asnumpy()
+
+
+def random_target_labels(true_labels):
+    target_labels = []
+    for label in true_labels:
+        while True:
+            target_label = np.random.randint(0, 10)
+            if target_label != label:
+                target_labels.append(target_label)
+                break
+    return target_labels
+
+
+def create_target_images(dataset, data_labels, target_labels):
+    res = []
+    for label in target_labels:
+        for i in range(len(data_labels)):
+            if data_labels[i] == label:
+                res.append(dataset[i])
+                break
+    return np.array(res)
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_hsja_mnist_attack():
+    """
+    hsja-Attack test
+    """
+    # upload trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+    net.set_train(False)
+
+    # get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds = generate_mnist_dataset(data_list, batch_size=batch_size)
+
+    # prediction accuracy before attack
+    model = ModelToBeAttacked(net)
+    batch_num = 5  # the number of batches of attacking samples
+    test_images = []
+    test_labels = []
+    predict_labels = []
+    i = 0
+    for data in ds.create_tuple_iterator():
+        i += 1
+        images = data[0].astype(np.float32)
+        labels = data[1]
+        test_images.append(images)
+        test_labels.append(labels)
+        pred_labels = np.argmax(model.predict(images), axis=1)
+        predict_labels.append(pred_labels)
+        if i >= batch_num:
+            break
+    predict_labels = np.concatenate(predict_labels)
+    true_labels = np.concatenate(test_labels)
+    accuracy = np.mean(np.equal(predict_labels, true_labels))
+    LOGGER.info(TAG, "prediction accuracy before attacking is : %s",
+                accuracy)
+    test_images = np.concatenate(test_images)
+
+    # attacking
+    norm = 'l2'
+    search = 'grid_search'
+    target = False
+    attack = HopSkipJumpAttack(model, constraint=norm, stepsize_search=search)
+    if target:
+        target_labels = random_target_labels(true_labels)
+        target_images = create_target_images(test_images, predict_labels,
+                                             target_labels)
+        attack.set_target_images(target_images)
+        success_list, adv_data, query_list = attack.generate(test_images, target_labels)
+    else:
+        success_list, adv_data, query_list = attack.generate(test_images, None)
+
+    adv_datas = []
+    gts = []
+    for success, adv, gt in zip(success_list, adv_data, true_labels):
+        if success:
+            adv_datas.append(adv)
+            gts.append(gt)
+    if len(gts) > 0:
+        adv_datas = np.concatenate(np.asarray(adv_datas), axis=0)
+        gts = np.asarray(gts)
+        pred_logits_adv = model.predict(adv_datas)
+        pred_lables_adv = np.argmax(pred_logits_adv, axis=1)
+        accuracy_adv = np.mean(np.equal(pred_lables_adv, gts))
+        LOGGER.info(TAG, 'mis-classification rate of adversaries is : %s',
+                    accuracy_adv)
+
+
+if __name__ == '__main__':
+    test_hsja_mnist_attack()

example/mnist_demo/mnist_attack_jsma.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import time
+import numpy as np
+import pytest
+from scipy.special import softmax
+
+from mindspore import Model
+from mindspore import Tensor
+from mindspore import context
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks.jsma import JSMAAttack
+from mindarmour.utils.logger import LogUtil
+from mindarmour.evaluations.attack_evaluation import AttackEvaluate
+
+from lenet5_net import LeNet5
+
+context.set_context(mode=context.GRAPH_MODE, device_target="Ascend")
+
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+LOGGER = LogUtil.get_instance()
+TAG = 'JSMA_Test'
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_jsma_attack():
+    """
+    JSMA-Attack test
+    """
+    # upload trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+
+    # get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds = generate_mnist_dataset(data_list, batch_size=batch_size)
+
+    # prediction accuracy before attack
+    model = Model(net)
+    batch_num = 3  # the number of batches of attacking samples
+    test_images = []
+    test_labels = []
+    predict_labels = []
+    i = 0
+    for data in ds.create_tuple_iterator():
+        i += 1
+        images = data[0].astype(np.float32)
+        labels = data[1]
+        test_images.append(images)
+        test_labels.append(labels)
+        pred_labels = np.argmax(model.predict(Tensor(images)).asnumpy(),
+                                axis=1)
+        predict_labels.append(pred_labels)
+        if i >= batch_num:
+            break
+    predict_labels = np.concatenate(predict_labels)
+    true_labels = np.concatenate(test_labels)
+    targeted_labels = np.random.randint(0, 10, size=len(true_labels))
+    for i in range(len(true_labels)):
+        if targeted_labels[i] == true_labels[i]:
+            targeted_labels[i] = (targeted_labels[i] + 1) % 10
+    accuracy = np.mean(np.equal(predict_labels, true_labels))
+    LOGGER.info(TAG, "prediction accuracy before attacking is : %g", accuracy)
+
+    # attacking
+    classes = 10
+    attack = JSMAAttack(net, classes)
+    start_time = time.clock()
+    adv_data = attack.batch_generate(np.concatenate(test_images),
+                                     targeted_labels, batch_size=32)
+    stop_time = time.clock()
+    pred_logits_adv = model.predict(Tensor(adv_data)).asnumpy()
+    # rescale predict confidences into (0, 1).
+    pred_logits_adv = softmax(pred_logits_adv, axis=1)
+    pred_lables_adv = np.argmax(pred_logits_adv, axis=1)
+    accuracy_adv = np.mean(np.equal(pred_lables_adv, true_labels))
+    LOGGER.info(TAG, "prediction accuracy after attacking is : %g",
+                accuracy_adv)
+    test_labels = np.eye(10)[np.concatenate(test_labels)]
+    attack_evaluate = AttackEvaluate(
+        np.concatenate(test_images).transpose(0, 2, 3, 1),
+        test_labels, adv_data.transpose(0, 2, 3, 1),
+        pred_logits_adv, targeted=True, target_label=targeted_labels)
+    LOGGER.info(TAG, 'mis-classification rate of adversaries is : %s',
+                attack_evaluate.mis_classification_rate())
+    LOGGER.info(TAG, 'The average confidence of adversarial class is : %s',
+                attack_evaluate.avg_conf_adv_class())
+    LOGGER.info(TAG, 'The average confidence of true class is : %s',
+                attack_evaluate.avg_conf_true_class())
+    LOGGER.info(TAG, 'The average distance (l0, l2, linf) between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_lp_distance())
+    LOGGER.info(TAG, 'The average structural similarity between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_ssim())
+    LOGGER.info(TAG, 'The average costing time is %s',
+                (stop_time - start_time) / (batch_num*batch_size))
+
+
+if __name__ == '__main__':
+    test_jsma_attack()

example/mnist_demo/mnist_attack_lbfgs.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import time
+import numpy as np
+import pytest
+from scipy.special import softmax
+
+from mindspore import Model
+from mindspore import Tensor
+from mindspore import context
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks.lbfgs import LBFGS
+from mindarmour.utils.logger import LogUtil
+from mindarmour.evaluations.attack_evaluation import AttackEvaluate
+
+from lenet5_net import LeNet5
+
+context.set_context(mode=context.GRAPH_MODE, device_target="Ascend")
+
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+LOGGER = LogUtil.get_instance()
+TAG = 'LBFGS_Test'
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_lbfgs_attack():
+    """
+    LBFGS-Attack test
+    """
+    # upload trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+
+    # get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds = generate_mnist_dataset(data_list, batch_size=batch_size, sparse=False)
+
+    # prediction accuracy before attack
+    model = Model(net)
+    batch_num = 3  # the number of batches of attacking samples
+    test_images = []
+    test_labels = []
+    predict_labels = []
+    i = 0
+    for data in ds.create_tuple_iterator():
+        i += 1
+        images = data[0].astype(np.float32)
+        labels = data[1]
+        test_images.append(images)
+        test_labels.append(labels)
+        pred_labels = np.argmax(model.predict(Tensor(images)).asnumpy(),
+                                axis=1)
+        predict_labels.append(pred_labels)
+        if i >= batch_num:
+            break
+    predict_labels = np.concatenate(predict_labels)
+    true_labels = np.argmax(np.concatenate(test_labels), axis=1)
+    accuracy = np.mean(np.equal(predict_labels, true_labels))
+    LOGGER.info(TAG, "prediction accuracy before attacking is : %s", accuracy)
+
+    # attacking
+    is_targeted = True
+    if is_targeted:
+        targeted_labels = np.random.randint(0, 10, size=len(true_labels)).astype(np.int32)
+        for i in range(len(true_labels)):
+            if targeted_labels[i] == true_labels[i]:
+                targeted_labels[i] = (targeted_labels[i] + 1) % 10
+    else:
+        targeted_labels = true_labels.astype(np.int32)
+    targeted_labels = np.eye(10)[targeted_labels].astype(np.float32)
+    attack = LBFGS(net, is_targeted=is_targeted)
+    start_time = time.clock()
+    adv_data = attack.batch_generate(np.concatenate(test_images),
+                                     targeted_labels,
+                                     batch_size=batch_size)
+    stop_time = time.clock()
+    pred_logits_adv = model.predict(Tensor(adv_data)).asnumpy()
+    # rescale predict confidences into (0, 1).
+    pred_logits_adv = softmax(pred_logits_adv, axis=1)
+    pred_labels_adv = np.argmax(pred_logits_adv, axis=1)
+
+    accuracy_adv = np.mean(np.equal(pred_labels_adv, true_labels))
+    LOGGER.info(TAG, "prediction accuracy after attacking is : %s",
+                accuracy_adv)
+    attack_evaluate = AttackEvaluate(np.concatenate(test_images).transpose(0, 2, 3, 1),
+                                     np.concatenate(test_labels),
+                                     adv_data.transpose(0, 2, 3, 1),
+                                     pred_logits_adv,
+                                     targeted=is_targeted,
+                                     target_label=np.argmax(targeted_labels,
+                                                            axis=1))
+    LOGGER.info(TAG, 'mis-classification rate of adversaries is : %s',
+                attack_evaluate.mis_classification_rate())
+    LOGGER.info(TAG, 'The average confidence of adversarial class is : %s',
+                attack_evaluate.avg_conf_adv_class())
+    LOGGER.info(TAG, 'The average confidence of true class is : %s',
+                attack_evaluate.avg_conf_true_class())
+    LOGGER.info(TAG, 'The average distance (l0, l2, linf) between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_lp_distance())
+    LOGGER.info(TAG, 'The average structural similarity between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_ssim())
+    LOGGER.info(TAG, 'The average costing time is %s',
+                (stop_time - start_time)/(batch_num*batch_size))
+
+
+if __name__ == '__main__':
+    test_lbfgs_attack()

example/mnist_demo/mnist_attack_nes.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import numpy as np
+import pytest
+
+from mindspore import Tensor
+from mindspore import context
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks.black.natural_evolutionary_strategy import NES
+from mindarmour.attacks.black.black_model import BlackModel
+
+from mindarmour.utils.logger import LogUtil
+from lenet5_net import LeNet5
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+context.set_context(mode=context.GRAPH_MODE)
+context.set_context(device_target="Ascend")
+
+LOGGER = LogUtil.get_instance()
+TAG = 'HopSkipJumpAttack'
+
+
+class ModelToBeAttacked(BlackModel):
+    """model to be attack"""
+
+    def __init__(self, network):
+        super(ModelToBeAttacked, self).__init__()
+        self._network = network
+
+    def predict(self, inputs):
+        """predict"""
+        if len(inputs.shape) == 3:
+            inputs = inputs[np.newaxis, :]
+        result = self._network(Tensor(inputs.astype(np.float32)))
+        return result.asnumpy()
+
+
+def random_target_labels(true_labels, labels_list):
+    target_labels = []
+    for label in true_labels:
+        while True:
+            target_label = np.random.choice(labels_list)
+            if target_label != label:
+                target_labels.append(target_label)
+                break
+    return target_labels
+
+
+def _pseudorandom_target(index, total_indices, true_class):
+    """ pseudo random_target """
+    rng = np.random.RandomState(index)
+    target = true_class
+    while target == true_class:
+        target = rng.randint(0, total_indices)
+    return target
+
+
+def create_target_images(dataset, data_labels, target_labels):
+    res = []
+    for label in target_labels:
+        for i in range(len(data_labels)):
+            if data_labels[i] == label:
+                res.append(dataset[i])
+                break
+    return np.array(res)
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_nes_mnist_attack():
+    """
+    hsja-Attack test
+    """
+    # upload trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+    net.set_train(False)
+
+    # get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds = generate_mnist_dataset(data_list, batch_size=batch_size)
+
+    # prediction accuracy before attack
+    model = ModelToBeAttacked(net)
+    # the number of batches of attacking samples
+    batch_num = 5
+    test_images = []
+    test_labels = []
+    predict_labels = []
+    i = 0
+    for data in ds.create_tuple_iterator():
+        i += 1
+        images = data[0].astype(np.float32)
+        labels = data[1]
+        test_images.append(images)
+        test_labels.append(labels)
+        pred_labels = np.argmax(model.predict(images), axis=1)
+        predict_labels.append(pred_labels)
+        if i >= batch_num:
+            break
+    predict_labels = np.concatenate(predict_labels)
+    true_labels = np.concatenate(test_labels)
+
+    accuracy = np.mean(np.equal(predict_labels, true_labels))
+    LOGGER.info(TAG, "prediction accuracy before attacking is : %s",
+                accuracy)
+    test_images = np.concatenate(test_images)
+
+    # attacking
+    scene = 'Query_Limit'
+    if scene == 'Query_Limit':
+        top_k = -1
+    elif scene == 'Partial_Info':
+        top_k = 5
+    elif scene == 'Label_Only':
+        top_k = 5
+
+    success = 0
+    queries_num = 0
+
+    nes_instance = NES(model, scene, top_k=top_k)
+    test_length = 32
+    advs = []
+    for img_index in range(test_length):
+        # Initial image and class selection
+        initial_img = test_images[img_index]
+        orig_class = true_labels[img_index]
+        initial_img = [initial_img]
+        target_class = random_target_labels([orig_class], true_labels)
+        target_image = create_target_images(test_images, true_labels,
+                                            target_class)
+        nes_instance.set_target_images(target_image)
+        tag, adv, queries = nes_instance.generate(initial_img, target_class)
+        if tag[0]:
+            success += 1
+        queries_num += queries[0]
+        advs.append(adv)
+
+    advs = np.reshape(advs, (len(advs), 1, 32, 32))
+    adv_pred = np.argmax(model.predict(advs), axis=1)
+    adv_accuracy = np.mean(np.equal(adv_pred, true_labels[:test_length]))
+    LOGGER.info(TAG, "prediction accuracy after attacking is : %s",
+                adv_accuracy)
+
+
+if __name__ == '__main__':
+    test_nes_mnist_attack()

example/mnist_demo/mnist_attack_pgd.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import time
+import numpy as np
+import pytest
+from scipy.special import softmax
+
+from mindspore import Model
+from mindspore import Tensor
+from mindspore import context
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks.iterative_gradient_method import ProjectedGradientDescent
+
+from mindarmour.utils.logger import LogUtil
+from mindarmour.evaluations.attack_evaluation import AttackEvaluate
+
+from lenet5_net import LeNet5
+
+context.set_context(mode=context.GRAPH_MODE, device_target="Ascend")
+
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+LOGGER = LogUtil.get_instance()
+TAG = 'PGD_Test'
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_projected_gradient_descent_method():
+    """
+    PGD-Attack test
+    """
+    # upload trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+
+    # get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds = generate_mnist_dataset(data_list, batch_size, sparse=False)
+
+    # prediction accuracy before attack
+    model = Model(net)
+    batch_num = 32  # the number of batches of attacking samples
+    test_images = []
+    test_labels = []
+    predict_labels = []
+    i = 0
+    for data in ds.create_tuple_iterator():
+        i += 1
+        images = data[0].astype(np.float32)
+        labels = data[1]
+        test_images.append(images)
+        test_labels.append(labels)
+        pred_labels = np.argmax(model.predict(Tensor(images)).asnumpy(),
+                                axis=1)
+        predict_labels.append(pred_labels)
+        if i >= batch_num:
+            break
+    predict_labels = np.concatenate(predict_labels)
+    true_labels = np.argmax(np.concatenate(test_labels), axis=1)
+    accuracy = np.mean(np.equal(predict_labels, true_labels))
+    LOGGER.info(TAG, "prediction accuracy before attacking is : %s", accuracy)
+
+    # attacking
+    attack = ProjectedGradientDescent(net, eps=0.3)
+    start_time = time.clock()
+    adv_data = attack.batch_generate(np.concatenate(test_images),
+                                     np.concatenate(test_labels), batch_size=32)
+    stop_time = time.clock()
+    np.save('./adv_data', adv_data)
+    pred_logits_adv = model.predict(Tensor(adv_data)).asnumpy()
+    # rescale predict confidences into (0, 1).
+    pred_logits_adv = softmax(pred_logits_adv, axis=1)
+    pred_labels_adv = np.argmax(pred_logits_adv, axis=1)
+    accuracy_adv = np.mean(np.equal(pred_labels_adv, true_labels))
+    LOGGER.info(TAG, "prediction accuracy after attacking is : %s", accuracy_adv)
+    attack_evaluate = AttackEvaluate(np.concatenate(test_images).transpose(0, 2, 3, 1),
+                                     np.concatenate(test_labels),
+                                     adv_data.transpose(0, 2, 3, 1),
+                                     pred_logits_adv)
+    LOGGER.info(TAG, 'mis-classification rate of adversaries is : %s',
+                attack_evaluate.mis_classification_rate())
+    LOGGER.info(TAG, 'The average confidence of adversarial class is : %s',
+                attack_evaluate.avg_conf_adv_class())
+    LOGGER.info(TAG, 'The average confidence of true class is : %s',
+                attack_evaluate.avg_conf_true_class())
+    LOGGER.info(TAG, 'The average distance (l0, l2, linf) between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_lp_distance())
+    LOGGER.info(TAG, 'The average structural similarity between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_ssim())
+    LOGGER.info(TAG, 'The average costing time is %s',
+                (stop_time - start_time)/(batch_num*batch_size))
+
+
+if __name__ == '__main__':
+    test_projected_gradient_descent_method()

example/mnist_demo/mnist_attack_pointwise.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import numpy as np
+import pytest
+from scipy.special import softmax
+
+from mindspore import Tensor
+from mindspore import context
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks.black.pointwise_attack import PointWiseAttack
+from mindarmour.attacks.black.black_model import BlackModel
+from mindarmour.utils.logger import LogUtil
+from mindarmour.evaluations.attack_evaluation import AttackEvaluate
+
+from lenet5_net import LeNet5
+
+context.set_context(mode=context.GRAPH_MODE, device_target="Ascend")
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+LOGGER = LogUtil.get_instance()
+TAG = 'Pointwise_Attack'
+LOGGER.set_level('INFO')
+
+
+class ModelToBeAttacked(BlackModel):
+    """model to be attack"""
+
+    def __init__(self, network):
+        super(ModelToBeAttacked, self).__init__()
+        self._network = network
+
+    def predict(self, inputs):
+        """predict"""
+        if len(inputs.shape) == 3:
+            inputs = inputs[np.newaxis, :]
+        result = self._network(Tensor(inputs.astype(np.float32)))
+        return result.asnumpy()
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_pointwise_attack_on_mnist():
+    """
+    Salt-and-Pepper-Attack test
+    """
+    # upload trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+
+    # get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds = generate_mnist_dataset(data_list, batch_size=batch_size)
+
+    # prediction accuracy before attack
+    model = ModelToBeAttacked(net)
+    batch_num = 3  # the number of batches of attacking samples
+    test_images = []
+    test_labels = []
+    predict_labels = []
+    i = 0
+    for data in ds.create_tuple_iterator():
+        i += 1
+        images = data[0].astype(np.float32)
+        labels = data[1]
+        test_images.append(images)
+        test_labels.append(labels)
+        pred_labels = np.argmax(model.predict(images), axis=1)
+        predict_labels.append(pred_labels)
+        if i >= batch_num:
+            break
+    predict_labels = np.concatenate(predict_labels)
+    true_labels = np.concatenate(test_labels)
+    accuracy = np.mean(np.equal(predict_labels, true_labels))
+    LOGGER.info(TAG, "prediction accuracy before attacking is : %g", accuracy)
+
+    # attacking
+    is_target = False
+    attack = PointWiseAttack(model=model, is_targeted=is_target)
+    if is_target:
+        targeted_labels = np.random.randint(0, 10, size=len(true_labels))
+        for i in range(len(true_labels)):
+            if targeted_labels[i] == true_labels[i]:
+                targeted_labels[i] = (targeted_labels[i] + 1) % 10
+    else:
+        targeted_labels = true_labels
+    success_list, adv_data, query_list = attack.generate(
+        np.concatenate(test_images), targeted_labels)
+    success_list = np.arange(success_list.shape[0])[success_list]
+    LOGGER.info(TAG, 'success_list: %s', success_list)
+    LOGGER.info(TAG, 'average of query times is : %s', np.mean(query_list))
+    adv_preds = []
+    for ite_data in adv_data:
+        pred_logits_adv = model.predict(ite_data)
+        # rescale predict confidences into (0, 1).
+        pred_logits_adv = softmax(pred_logits_adv, axis=1)
+        adv_preds.extend(pred_logits_adv)
+    accuracy_adv = np.mean(np.equal(np.max(adv_preds, axis=1), true_labels))
+    LOGGER.info(TAG, "prediction accuracy after attacking is : %g",
+                accuracy_adv)
+    test_labels_onehot = np.eye(10)[true_labels]
+    attack_evaluate = AttackEvaluate(np.concatenate(test_images),
+                                     test_labels_onehot, adv_data,
+                                     adv_preds, targeted=is_target,
+                                     target_label=targeted_labels)
+    LOGGER.info(TAG, 'mis-classification rate of adversaries is : %s',
+                attack_evaluate.mis_classification_rate())
+    LOGGER.info(TAG, 'The average confidence of adversarial class is : %s',
+                attack_evaluate.avg_conf_adv_class())
+    LOGGER.info(TAG, 'The average confidence of true class is : %s',
+                attack_evaluate.avg_conf_true_class())
+    LOGGER.info(TAG, 'The average distance (l0, l2, linf) between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_lp_distance())
+
+
+if __name__ == '__main__':
+    test_pointwise_attack_on_mnist()

example/mnist_demo/mnist_attack_pso.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import time
+import numpy as np
+import pytest
+from scipy.special import softmax
+
+from mindspore import Tensor
+from mindspore import context
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks.black.pso_attack import PSOAttack
+from mindarmour.attacks.black.black_model import BlackModel
+from mindarmour.utils.logger import LogUtil
+from mindarmour.evaluations.attack_evaluation import AttackEvaluate
+
+from lenet5_net import LeNet5
+
+context.set_context(mode=context.GRAPH_MODE, device_target="Ascend")
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+LOGGER = LogUtil.get_instance()
+TAG = 'PSO_Attack'
+
+
+class ModelToBeAttacked(BlackModel):
+    """model to be attack"""
+
+    def __init__(self, network):
+        super(ModelToBeAttacked, self).__init__()
+        self._network = network
+
+    def predict(self, inputs):
+        """predict"""
+        result = self._network(Tensor(inputs.astype(np.float32)))
+        return result.asnumpy()
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_pso_attack_on_mnist():
+    """
+    PSO-Attack test
+    """
+    # upload trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+
+    # get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds = generate_mnist_dataset(data_list, batch_size=batch_size)
+
+    # prediction accuracy before attack
+    model = ModelToBeAttacked(net)
+    batch_num = 3  # the number of batches of attacking samples
+    test_images = []
+    test_labels = []
+    predict_labels = []
+    i = 0
+    for data in ds.create_tuple_iterator():
+        i += 1
+        images = data[0].astype(np.float32)
+        labels = data[1]
+        test_images.append(images)
+        test_labels.append(labels)
+        pred_labels = np.argmax(model.predict(images), axis=1)
+        predict_labels.append(pred_labels)
+        if i >= batch_num:
+            break
+    predict_labels = np.concatenate(predict_labels)
+    true_labels = np.concatenate(test_labels)
+    accuracy = np.mean(np.equal(predict_labels, true_labels))
+    LOGGER.info(TAG, "prediction accuracy before attacking is : %s", accuracy)
+
+    # attacking
+    attack = PSOAttack(model, bounds=(0.0, 1.0), pm=0.5, sparse=True)
+    start_time = time.clock()
+    success_list, adv_data, query_list = attack.generate(
+        np.concatenate(test_images), np.concatenate(test_labels))
+    stop_time = time.clock()
+    LOGGER.info(TAG, 'success_list: %s', success_list)
+    LOGGER.info(TAG, 'average of query times is : %s', np.mean(query_list))
+    pred_logits_adv = model.predict(adv_data)
+    # rescale predict confidences into (0, 1).
+    pred_logits_adv = softmax(pred_logits_adv, axis=1)
+    pred_labels_adv = np.argmax(pred_logits_adv, axis=1)
+    accuracy_adv = np.mean(np.equal(pred_labels_adv, true_labels))
+    LOGGER.info(TAG, "prediction accuracy after attacking is : %s",
+                accuracy_adv)
+    test_labels_onehot = np.eye(10)[np.concatenate(test_labels)]
+    attack_evaluate = AttackEvaluate(np.concatenate(test_images),
+                                     test_labels_onehot, adv_data,
+                                     pred_logits_adv)
+    LOGGER.info(TAG, 'mis-classification rate of adversaries is : %s',
+                attack_evaluate.mis_classification_rate())
+    LOGGER.info(TAG, 'The average confidence of adversarial class is : %s',
+                attack_evaluate.avg_conf_adv_class())
+    LOGGER.info(TAG, 'The average confidence of true class is : %s',
+                attack_evaluate.avg_conf_true_class())
+    LOGGER.info(TAG, 'The average distance (l0, l2, linf) between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_lp_distance())
+    LOGGER.info(TAG, 'The average structural similarity between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_ssim())
+    LOGGER.info(TAG, 'The average costing time is %s',
+                (stop_time - start_time)/(batch_num*batch_size))
+
+
+if __name__ == '__main__':
+    test_pso_attack_on_mnist()

example/mnist_demo/mnist_attack_salt_and_pepper.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import numpy as np
+import pytest
+from scipy.special import softmax
+
+from mindspore import Tensor
+from mindspore import context
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks.black.salt_and_pepper_attack import SaltAndPepperNoiseAttack
+from mindarmour.attacks.black.black_model import BlackModel
+from mindarmour.utils.logger import LogUtil
+from mindarmour.evaluations.attack_evaluation import AttackEvaluate
+
+from lenet5_net import LeNet5
+
+context.set_context(mode=context.GRAPH_MODE, device_target="Ascend")
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+LOGGER = LogUtil.get_instance()
+TAG = 'Salt_and_Pepper_Attack'
+LOGGER.set_level('DEBUG')
+
+
+class ModelToBeAttacked(BlackModel):
+    """model to be attack"""
+
+    def __init__(self, network):
+        super(ModelToBeAttacked, self).__init__()
+        self._network = network
+
+    def predict(self, inputs):
+        """predict"""
+        if len(inputs.shape) == 3:
+            inputs = inputs[np.newaxis, :]
+        result = self._network(Tensor(inputs.astype(np.float32)))
+        return result.asnumpy()
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_salt_and_pepper_attack_on_mnist():
+    """
+    Salt-and-Pepper-Attack test
+    """
+    # upload trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+
+    # get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds = generate_mnist_dataset(data_list, batch_size=batch_size)
+
+    # prediction accuracy before attack
+    model = ModelToBeAttacked(net)
+    batch_num = 3  # the number of batches of attacking samples
+    test_images = []
+    test_labels = []
+    predict_labels = []
+    i = 0
+    for data in ds.create_tuple_iterator():
+        i += 1
+        images = data[0].astype(np.float32)
+        labels = data[1]
+        test_images.append(images)
+        test_labels.append(labels)
+        pred_labels = np.argmax(model.predict(images), axis=1)
+        predict_labels.append(pred_labels)
+        if i >= batch_num:
+            break
+    LOGGER.debug(TAG, 'model input image shape is: {}'.format(np.array(test_images).shape))
+    predict_labels = np.concatenate(predict_labels)
+    true_labels = np.concatenate(test_labels)
+    accuracy = np.mean(np.equal(predict_labels, true_labels))
+    LOGGER.info(TAG, "prediction accuracy before attacking is : %g", accuracy)
+
+    # attacking
+    is_target = False
+    attack = SaltAndPepperNoiseAttack(model=model,
+                                      is_targeted=is_target,
+                                      sparse=True)
+    if is_target:
+        targeted_labels = np.random.randint(0, 10, size=len(true_labels))
+        for i in range(len(true_labels)):
+            if targeted_labels[i] == true_labels[i]:
+                targeted_labels[i] = (targeted_labels[i] + 1) % 10
+    else:
+        targeted_labels = true_labels
+    LOGGER.debug(TAG, 'input shape is: {}'.format(np.concatenate(test_images).shape))
+    success_list, adv_data, query_list = attack.generate(
+        np.concatenate(test_images), targeted_labels)
+    success_list = np.arange(success_list.shape[0])[success_list]
+    LOGGER.info(TAG, 'success_list: %s', success_list)
+    LOGGER.info(TAG, 'average of query times is : %s', np.mean(query_list))
+    adv_preds = []
+    for ite_data in adv_data:
+        pred_logits_adv = model.predict(ite_data)
+        # rescale predict confidences into (0, 1).
+        pred_logits_adv = softmax(pred_logits_adv, axis=1)
+        adv_preds.extend(pred_logits_adv)
+    accuracy_adv = np.mean(np.equal(np.max(adv_preds, axis=1), true_labels))
+    LOGGER.info(TAG, "prediction accuracy after attacking is : %g",
+                accuracy_adv)
+    test_labels_onehot = np.eye(10)[true_labels]
+    attack_evaluate = AttackEvaluate(np.concatenate(test_images),
+                                     test_labels_onehot, adv_data,
+                                     adv_preds, targeted=is_target,
+                                     target_label=targeted_labels)
+    LOGGER.info(TAG, 'mis-classification rate of adversaries is : %s',
+                attack_evaluate.mis_classification_rate())
+    LOGGER.info(TAG, 'The average confidence of adversarial class is : %s',
+                attack_evaluate.avg_conf_adv_class())
+    LOGGER.info(TAG, 'The average confidence of true class is : %s',
+                attack_evaluate.avg_conf_true_class())
+    LOGGER.info(TAG, 'The average distance (l0, l2, linf) between original '
+                     'samples and adversarial samples are: %s',
+                attack_evaluate.avg_lp_distance())
+
+
+if __name__ == '__main__':
+    test_salt_and_pepper_attack_on_mnist()

example/mnist_demo/mnist_defense_nad.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""defense example using nad"""
+import sys
+
+import logging
+
+import numpy as np
+import pytest
+
+from mindspore import Tensor
+from mindspore import context
+from mindspore import nn
+from mindspore.nn import SoftmaxCrossEntropyWithLogits
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks import FastGradientSignMethod
+from mindarmour.defenses import NaturalAdversarialDefense
+from mindarmour.utils.logger import LogUtil
+
+from lenet5_net import LeNet5
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+context.set_context(mode=context.GRAPH_MODE, device_target="Ascend")
+
+LOGGER = LogUtil.get_instance()
+TAG = 'Nad_Example'
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_nad_method():
+    """
+    NAD-Defense test.
+    """
+    # 1. load trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+
+    loss = SoftmaxCrossEntropyWithLogits(is_grad=False, sparse=False)
+    opt = nn.Momentum(net.trainable_params(), 0.01, 0.09)
+
+    nad = NaturalAdversarialDefense(net, loss_fn=loss, optimizer=opt,
+                                    bounds=(0.0, 1.0), eps=0.3)
+
+    # 2. get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds_test = generate_mnist_dataset(data_list, batch_size=batch_size,
+                                     sparse=False)
+    inputs = []
+    labels = []
+    for data in ds_test.create_tuple_iterator():
+        inputs.append(data[0].astype(np.float32))
+        labels.append(data[1])
+    inputs = np.concatenate(inputs)
+    labels = np.concatenate(labels)
+
+    # 3. get accuracy of test data on original model
+    net.set_train(False)
+    acc_list = []
+    batchs = inputs.shape[0] // batch_size
+    for i in range(batchs):
+        batch_inputs = inputs[i*batch_size : (i + 1)*batch_size]
+        batch_labels = np.argmax(labels[i*batch_size : (i + 1)*batch_size], axis=1)
+        logits = net(Tensor(batch_inputs)).asnumpy()
+        label_pred = np.argmax(logits, axis=1)
+        acc_list.append(np.mean(batch_labels == label_pred))
+
+    LOGGER.debug(TAG, 'accuracy of TEST data on original model is : %s',
+                 np.mean(acc_list))
+
+    # 4. get adv of test data
+    attack = FastGradientSignMethod(net, eps=0.3)
+    adv_data = attack.batch_generate(inputs, labels)
+    LOGGER.debug(TAG, 'adv_data.shape is : %s', adv_data.shape)
+
+    # 5. get accuracy of adv data on original model
+    net.set_train(False)
+    acc_list = []
+    batchs = adv_data.shape[0] // batch_size
+    for i in range(batchs):
+        batch_inputs = adv_data[i*batch_size : (i + 1)*batch_size]
+        batch_labels = np.argmax(labels[i*batch_size : (i + 1)*batch_size], axis=1)
+        logits = net(Tensor(batch_inputs)).asnumpy()
+        label_pred = np.argmax(logits, axis=1)
+        acc_list.append(np.mean(batch_labels == label_pred))
+
+    LOGGER.debug(TAG, 'accuracy of adv data on original model is : %s',
+                 np.mean(acc_list))
+
+    # 6. defense
+    net.set_train()
+    nad.batch_defense(inputs, labels, batch_size=32, epochs=10)
+
+    # 7. get accuracy of test data on defensed model
+    net.set_train(False)
+    acc_list = []
+    batchs = inputs.shape[0] // batch_size
+    for i in range(batchs):
+        batch_inputs = inputs[i*batch_size : (i + 1)*batch_size]
+        batch_labels = np.argmax(labels[i*batch_size : (i + 1)*batch_size], axis=1)
+        logits = net(Tensor(batch_inputs)).asnumpy()
+        label_pred = np.argmax(logits, axis=1)
+        acc_list.append(np.mean(batch_labels == label_pred))
+
+    LOGGER.debug(TAG, 'accuracy of TEST data on defensed model is : %s',
+                 np.mean(acc_list))
+
+    # 8. get accuracy of adv data on defensed model
+    acc_list = []
+    batchs = adv_data.shape[0] // batch_size
+    for i in range(batchs):
+        batch_inputs = adv_data[i*batch_size : (i + 1)*batch_size]
+        batch_labels = np.argmax(labels[i*batch_size : (i + 1)*batch_size], axis=1)
+        logits = net(Tensor(batch_inputs)).asnumpy()
+        label_pred = np.argmax(logits, axis=1)
+        acc_list.append(np.mean(batch_labels == label_pred))
+
+    LOGGER.debug(TAG, 'accuracy of adv data on defensed model is : %s',
+                 np.mean(acc_list))
+
+
+if __name__ == '__main__':
+    LOGGER.set_level(logging.DEBUG)
+    test_nad_method()

example/mnist_demo/mnist_evaluation.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""evaluate example"""
+import sys
+import os
+import time
+import numpy as np
+from scipy.special import softmax
+
+from lenet5_net import LeNet5
+from mindspore import Model
+from mindspore import Tensor
+from mindspore import context
+from mindspore import nn
+from mindspore.nn import Cell
+from mindspore.ops.operations import TensorAdd
+from mindspore.nn import SoftmaxCrossEntropyWithLogits
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.attacks import FastGradientSignMethod
+from mindarmour.attacks import GeneticAttack
+from mindarmour.attacks.black.black_model import BlackModel
+from mindarmour.defenses import NaturalAdversarialDefense
+from mindarmour.evaluations import BlackDefenseEvaluate
+from mindarmour.evaluations import DefenseEvaluate
+from mindarmour.utils.logger import LogUtil
+from mindarmour.detectors.black.similarity_detector import SimilarityDetector
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+context.set_context(mode=context.GRAPH_MODE, device_target="Ascend")
+
+LOGGER = LogUtil.get_instance()
+TAG = 'Defense_Evaluate_Example'
+
+
+def get_detector(train_images):
+    encoder = Model(EncoderNet(encode_dim=256))
+    detector = SimilarityDetector(max_k_neighbor=50, trans_model=encoder)
+    detector.fit(inputs=train_images)
+    return detector
+
+
+class EncoderNet(Cell):
+    """
+    Similarity encoder for input data
+    """
+
+    def __init__(self, encode_dim):
+        super(EncoderNet, self).__init__()
+        self._encode_dim = encode_dim
+        self.add = TensorAdd()
+
+    def construct(self, inputs):
+        """
+        construct the neural network
+        Args:
+            inputs (Tensor): input data to neural network.
+        Returns:
+            Tensor, output of neural network.
+        """
+        return self.add(inputs, inputs)
+
+    def get_encode_dim(self):
+        """
+        Get the dimension of encoded inputs
+
+        Returns:
+            int, dimension of encoded inputs.
+        """
+        return self._encode_dim
+
+
+class ModelToBeAttacked(BlackModel):
+    """
+    model to be attack
+    """
+
+    def __init__(self, network, defense=False, train_images=None):
+        super(ModelToBeAttacked, self).__init__()
+        self._network = network
+        self._queries = []
+        self._defense = defense
+        self._detector = None
+        self._detected_res = []
+        if self._defense:
+            self._detector = get_detector(train_images)
+
+    def predict(self, inputs):
+        """
+        predict function
+        """
+        query_num = inputs.shape[0]
+        results = []
+        if self._detector:
+            for i in range(query_num):
+                query = np.expand_dims(inputs[i].astype(np.float32), axis=0)
+                result = self._network(Tensor(query)).asnumpy()
+                det_num = len(self._detector.get_detected_queries())
+                self._detector.detect([query])
+                new_det_num = len(self._detector.get_detected_queries())
+                # If attack query detected, return random predict result
+                if new_det_num > det_num:
+                    results.append(result + np.random.rand(*result.shape))
+                    self._detected_res.append(True)
+                else:
+                    results.append(result)
+                    self._detected_res.append(False)
+            results = np.concatenate(results)
+        else:
+            results = self._network(Tensor(inputs.astype(np.float32))).asnumpy()
+        return results
+
+    def get_detected_result(self):
+        return self._detected_res
+
+
+def test_black_defense():
+    # load trained network
+    current_dir = os.path.dirname(os.path.abspath(__file__))
+    ckpt_name = os.path.abspath(os.path.join(
+        current_dir, './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'))
+    # ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    wb_net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(wb_net, load_dict)
+
+    # get test data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 32
+    ds_test = generate_mnist_dataset(data_list, batch_size=batch_size,
+                                     sparse=False)
+    inputs = []
+    labels = []
+    for data in ds_test.create_tuple_iterator():
+        inputs.append(data[0].astype(np.float32))
+        labels.append(data[1])
+    inputs = np.concatenate(inputs).astype(np.float32)
+    labels = np.concatenate(labels).astype(np.float32)
+    labels_sparse = np.argmax(labels, axis=1)
+
+    target_label = np.random.randint(0, 10, size=labels_sparse.shape[0])
+    for idx in range(labels_sparse.shape[0]):
+        while target_label[idx] == labels_sparse[idx]:
+            target_label[idx] = np.random.randint(0, 10)
+    target_label = np.eye(10)[target_label].astype(np.float32)
+
+    attacked_size = 50
+    benign_size = 500
+
+    attacked_sample = inputs[:attacked_size]
+    attacked_true_label = labels[:attacked_size]
+    benign_sample = inputs[attacked_size:attacked_size + benign_size]
+
+    wb_model = ModelToBeAttacked(wb_net)
+
+    # gen white-box adversarial examples of test data
+    wb_attack = FastGradientSignMethod(wb_net, eps=0.3)
+    wb_adv_sample = wb_attack.generate(attacked_sample,
+                                       attacked_true_label)
+
+    wb_raw_preds = softmax(wb_model.predict(wb_adv_sample), axis=1)
+    accuracy_test = np.mean(
+        np.equal(np.argmax(wb_model.predict(attacked_sample), axis=1),
+                 np.argmax(attacked_true_label, axis=1)))
+    LOGGER.info(TAG, "prediction accuracy before white-box attack is : %s",
+                accuracy_test)
+    accuracy_adv = np.mean(np.equal(np.argmax(wb_raw_preds, axis=1),
+                                    np.argmax(attacked_true_label, axis=1)))
+    LOGGER.info(TAG, "prediction accuracy after white-box attack is : %s",
+                accuracy_adv)
+
+    # improve the robustness of model with white-box adversarial examples
+    loss = SoftmaxCrossEntropyWithLogits(is_grad=False, sparse=False)
+    opt = nn.Momentum(wb_net.trainable_params(), 0.01, 0.09)
+
+    nad = NaturalAdversarialDefense(wb_net, loss_fn=loss, optimizer=opt,
+                                    bounds=(0.0, 1.0), eps=0.3)
+    wb_net.set_train(False)
+    nad.batch_defense(inputs[:5000], labels[:5000], batch_size=32, epochs=10)
+
+    wb_def_preds = wb_net(Tensor(wb_adv_sample)).asnumpy()
+    wb_def_preds = softmax(wb_def_preds, axis=1)
+    accuracy_def = np.mean(np.equal(np.argmax(wb_def_preds, axis=1),
+                                    np.argmax(attacked_true_label, axis=1)))
+    LOGGER.info(TAG, "prediction accuracy after defense is : %s", accuracy_def)
+
+    # calculate defense evaluation metrics for defense against white-box attack
+    wb_def_evaluate = DefenseEvaluate(wb_raw_preds, wb_def_preds,
+                                      np.argmax(attacked_true_label, axis=1))
+    LOGGER.info(TAG, 'defense evaluation for white-box adversarial attack')
+    LOGGER.info(TAG,
+                'classification accuracy variance (CAV) is : {:.2f}'.format(
+                    wb_def_evaluate.cav()))
+    LOGGER.info(TAG, 'classification rectify ratio (CRR) is : {:.2f}'.format(
+        wb_def_evaluate.crr()))
+    LOGGER.info(TAG, 'classification sacrifice ratio (CSR) is : {:.2f}'.format(
+        wb_def_evaluate.csr()))
+    LOGGER.info(TAG,
+                'classification confidence variance (CCV) is : {:.2f}'.format(
+                    wb_def_evaluate.ccv()))
+    LOGGER.info(TAG, 'classification output stability is : {:.2f}'.format(
+        wb_def_evaluate.cos()))
+
+    # calculate defense evaluation metrics for defense against black-box attack
+    LOGGER.info(TAG, 'defense evaluation for black-box adversarial attack')
+    bb_raw_preds = []
+    bb_def_preds = []
+    raw_query_counts = []
+    raw_query_time = []
+    def_query_counts = []
+    def_query_time = []
+    def_detection_counts = []
+
+    # gen black-box adversarial examples of test data
+    bb_net = LeNet5()
+    load_param_into_net(bb_net, load_dict)
+    bb_model = ModelToBeAttacked(bb_net, defense=False)
+    attack_rm = GeneticAttack(model=bb_model, pop_size=6, mutation_rate=0.05,
+                              per_bounds=0.1, step_size=0.25, temp=0.1,
+                              sparse=False)
+    attack_target_label = target_label[:attacked_size]
+    true_label = labels_sparse[:attacked_size + benign_size]
+    # evaluate robustness of original model
+    # gen black-box adversarial examples of test data
+    for idx in range(attacked_size):
+        raw_st = time.time()
+        raw_sl, raw_a, raw_qc = attack_rm.generate(
+            np.expand_dims(attacked_sample[idx], axis=0),
+            np.expand_dims(attack_target_label[idx], axis=0))
+        raw_t = time.time() - raw_st
+        bb_raw_preds.extend(softmax(bb_model.predict(raw_a), axis=1))
+        raw_query_counts.extend(raw_qc)
+        raw_query_time.append(raw_t)
+
+    for idx in range(benign_size):
+        raw_st = time.time()
+        bb_raw_pred = softmax(
+            bb_model.predict(np.expand_dims(benign_sample[idx], axis=0)),
+            axis=1)
+        raw_t = time.time() - raw_st
+        bb_raw_preds.extend(bb_raw_pred)
+        raw_query_counts.extend([0])
+        raw_query_time.append(raw_t)
+
+    accuracy_test = np.mean(
+        np.equal(np.argmax(bb_raw_preds[0:len(attack_target_label)], axis=1),
+                 np.argmax(attack_target_label, axis=1)))
+    LOGGER.info(TAG, "attack success before adv defense is : %s",
+                accuracy_test)
+
+    # improve the robustness of model with similarity-based detector
+    bb_def_model = ModelToBeAttacked(bb_net, defense=True,
+                                     train_images=inputs[0:6000])
+    # attack defensed model
+    attack_dm = GeneticAttack(model=bb_def_model, pop_size=6,
+                              mutation_rate=0.05,
+                              per_bounds=0.1, step_size=0.25, temp=0.1,
+                              sparse=False)
+    for idx in range(attacked_size):
+        def_st = time.time()
+        def_sl, def_a, def_qc = attack_dm.generate(
+            np.expand_dims(attacked_sample[idx], axis=0),
+            np.expand_dims(attack_target_label[idx], axis=0))
+        def_t = time.time() - def_st
+        det_res = bb_def_model.get_detected_result()
+        def_detection_counts.append(np.sum(det_res[-def_qc[0]:]))
+        bb_def_preds.extend(softmax(bb_def_model.predict(def_a), axis=1))
+        def_query_counts.extend(def_qc)
+        def_query_time.append(def_t)
+
+    for idx in range(benign_size):
+        def_st = time.time()
+        bb_def_pred = softmax(
+            bb_def_model.predict(np.expand_dims(benign_sample[idx], axis=0)),
+            axis=1)
+        def_t = time.time() - def_st
+        det_res = bb_def_model.get_detected_result()
+        def_detection_counts.append(np.sum(det_res[-1]))
+        bb_def_preds.extend(bb_def_pred)
+        def_query_counts.extend([0])
+        def_query_time.append(def_t)
+
+    accuracy_adv = np.mean(
+        np.equal(np.argmax(bb_def_preds[0:len(attack_target_label)], axis=1),
+                 np.argmax(attack_target_label, axis=1)))
+    LOGGER.info(TAG, "attack success rate after adv defense is : %s",
+                accuracy_adv)
+
+    bb_raw_preds = np.array(bb_raw_preds).astype(np.float32)
+    bb_def_preds = np.array(bb_def_preds).astype(np.float32)
+    # check evaluate data
+    max_queries = 6000
+
+    def_evaluate = BlackDefenseEvaluate(bb_raw_preds, bb_def_preds,
+                                        np.array(raw_query_counts),
+                                        np.array(def_query_counts),
+                                        np.array(raw_query_time),
+                                        np.array(def_query_time),
+                                        np.array(def_detection_counts),
+                                        true_label, max_queries)
+
+    LOGGER.info(TAG, 'query count variance of adversaries is : {:.2f}'.format(
+        def_evaluate.qcv()))
+    LOGGER.info(TAG, 'attack success rate variance of adversaries '
+                     'is : {:.2f}'.format(def_evaluate.asv()))
+    LOGGER.info(TAG, 'false positive rate (FPR) of the query-based detector '
+                     'is : {:.2f}'.format(def_evaluate.fpr()))
+    LOGGER.info(TAG, 'the benign query response time variance (QRV) '
+                     'is : {:.2f}'.format(def_evaluate.qrv()))
+
+
+if __name__ == '__main__':
+    test_black_defense()

example/mnist_demo/mnist_similarity_detector.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+import numpy as np
+import pytest
+from scipy.special import softmax
+
+from mindspore import Model
+from mindspore import context
+from mindspore import Tensor
+from mindspore.nn import Cell
+from mindspore.ops.operations import TensorAdd
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+
+from mindarmour.utils.logger import LogUtil
+from mindarmour.attacks.black.pso_attack import PSOAttack
+from mindarmour.attacks.black.black_model import BlackModel
+from mindarmour.detectors.black.similarity_detector import SimilarityDetector
+
+from lenet5_net import LeNet5
+
+context.set_context(mode=context.GRAPH_MODE, device_target="Ascend")
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+
+LOGGER = LogUtil.get_instance()
+TAG = 'Similarity Detector test'
+
+
+class ModelToBeAttacked(BlackModel):
+    """
+    model to be attack
+    """
+
+    def __init__(self, network):
+        super(ModelToBeAttacked, self).__init__()
+        self._network = network
+        self._queries = []
+
+    def predict(self, inputs):
+        """
+        predict function
+        """
+        query_num = inputs.shape[0]
+        for i in range(query_num):
+            self._queries.append(inputs[i].astype(np.float32))
+        result = self._network(Tensor(inputs.astype(np.float32)))
+        return result.asnumpy()
+
+    def get_queries(self):
+        return self._queries
+
+
+class EncoderNet(Cell):
+    """
+    Similarity encoder for input data
+    """
+
+    def __init__(self, encode_dim):
+        super(EncoderNet, self).__init__()
+        self._encode_dim = encode_dim
+        self.add = TensorAdd()
+
+    def construct(self, inputs):
+        """
+        construct the neural network
+        Args:
+            inputs (Tensor): input data to neural network.
+        Returns:
+            Tensor, output of neural network.
+        """
+        return self.add(inputs, inputs)
+
+    def get_encode_dim(self):
+        """
+        Get the dimension of encoded inputs
+
+        Returns:
+            int, dimension of encoded inputs.
+        """
+        return self._encode_dim
+
+
+@pytest.mark.level1
+@pytest.mark.platform_arm_ascend_training
+@pytest.mark.platform_x86_ascend_training
+@pytest.mark.env_card
+@pytest.mark.component_mindarmour
+def test_similarity_detector():
+    """
+    Similarity Detector test.
+    """
+    # load trained network
+    ckpt_name = './trained_ckpt_file/checkpoint_lenet-10_1875.ckpt'
+    net = LeNet5()
+    load_dict = load_checkpoint(ckpt_name)
+    load_param_into_net(net, load_dict)
+
+    # get mnist data
+    data_list = "./MNIST_unzip/test"
+    batch_size = 1000
+    ds = generate_mnist_dataset(data_list, batch_size=batch_size)
+    model = ModelToBeAttacked(net)
+
+    batch_num = 10  # the number of batches of input samples
+    all_images = []
+    true_labels = []
+    predict_labels = []
+    i = 0
+    for data in ds.create_tuple_iterator():
+        i += 1
+        images = data[0].astype(np.float32)
+        labels = data[1]
+        all_images.append(images)
+        true_labels.append(labels)
+        pred_labels = np.argmax(model.predict(images), axis=1)
+        predict_labels.append(pred_labels)
+        if i >= batch_num:
+            break
+    all_images = np.concatenate(all_images)
+    true_labels = np.concatenate(true_labels)
+    predict_labels = np.concatenate(predict_labels)
+    accuracy = np.mean(np.equal(predict_labels, true_labels))
+    LOGGER.info(TAG, "prediction accuracy before attacking is : %s", accuracy)
+
+    train_images = all_images[0:6000, :, :, :]
+    attacked_images = all_images[0:10, :, :, :]
+    attacked_labels = true_labels[0:10]
+
+    # generate malicious query sequence of black attack
+    attack = PSOAttack(model, bounds=(0.0, 1.0), pm=0.5, sparse=True,
+                       t_max=1000)
+    success_list, adv_data, query_list = attack.generate(attacked_images,
+                                                         attacked_labels)
+    LOGGER.info(TAG, 'pso attack success_list: %s', success_list)
+    LOGGER.info(TAG, 'average of query counts is : %s', np.mean(query_list))
+    pred_logits_adv = model.predict(adv_data)
+    # rescale predict confidences into (0, 1).
+    pred_logits_adv = softmax(pred_logits_adv, axis=1)
+    pred_lables_adv = np.argmax(pred_logits_adv, axis=1)
+    accuracy_adv = np.mean(np.equal(pred_lables_adv, attacked_labels))
+    LOGGER.info(TAG, "prediction accuracy after attacking is : %g",
+                accuracy_adv)
+
+    benign_queries = all_images[6000:10000, :, :, :]
+    suspicious_queries = model.get_queries()
+
+    # explicit threshold not provided, calculate threshold for K
+    encoder = Model(EncoderNet(encode_dim=256))
+    detector = SimilarityDetector(max_k_neighbor=50, trans_model=encoder)
+    detector.fit(inputs=train_images)
+
+    # test benign queries
+    detector.detect(benign_queries)
+    fpr = len(detector.get_detected_queries()) / benign_queries.shape[0]
+    LOGGER.info(TAG, 'Number of false positive of attack detector is : %s',
+                len(detector.get_detected_queries()))
+    LOGGER.info(TAG, 'False positive rate of attack detector is : %s', fpr)
+
+    # test attack queries
+    detector.clear_buffer()
+    detector.detect(suspicious_queries)
+    LOGGER.info(TAG, 'Number of detected attack queries is : %s',
+                len(detector.get_detected_queries()))
+    LOGGER.info(TAG, 'The detected attack query indexes are : %s',
+                detector.get_detected_queries())
+
+
+if __name__ == '__main__':
+    test_similarity_detector()

example/mnist_demo/mnist_train.py

+# Copyright 2020 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+import os
+import sys
+
+import mindspore.nn as nn
+from mindspore import context, Tensor
+from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor
+from mindspore.train.serialization import load_checkpoint, load_param_into_net
+from mindspore.train import Model
+import mindspore.ops.operations as P
+from mindspore.nn.metrics import Accuracy
+from mindspore.ops import functional as F
+from mindspore.common import dtype as mstype
+
+from mindarmour.utils.logger import LogUtil
+
+from lenet5_net import LeNet5
+
+sys.path.append("..")
+from data_processing import generate_mnist_dataset
+LOGGER = LogUtil.get_instance()
+TAG = 'Lenet5_train'
+
+
+class CrossEntropyLoss(nn.Cell):
+    """
+    Define loss for network
+    """
+    def __init__(self):
+        super(CrossEntropyLoss, self).__init__()
+        self.cross_entropy = P.SoftmaxCrossEntropyWithLogits()
+        self.mean = P.ReduceMean()
+        self.one_hot = P.OneHot()
+        self.on_value = Tensor(1.0, mstype.float32)
+        self.off_value = Tensor(0.0, mstype.float32)
+
+    def construct(self, logits, label):
+        label = self.one_hot(label, F.shape(logits)[1], self.on_value, self.off_value)
+        loss = self.cross_entropy(logits, label)[0]
+        loss = self.mean(loss, (-1,))
+        return loss
+
+
+def mnist_train(epoch_size, batch_size, lr, momentum):
+    context.set_context(mode=context.GRAPH_MODE, device_target="Ascend",
+                        enable_mem_reuse=False)
+
+    lr = lr
+    momentum = momentum
+    epoch_size = epoch_size
+    mnist_path = "./MNIST_unzip/"
+    ds = generate_mnist_dataset(os.path.join(mnist_path, "train"),
+                                batch_size=batch_size, repeat_size=1)
+
+    network = LeNet5()
+    network.set_train()
+    net_loss = CrossEntropyLoss()
+    net_opt = nn.Momentum(network.trainable_params(), lr, momentum)
+    config_ck = CheckpointConfig(save_checkpoint_steps=1875, keep_checkpoint_max=10)
+    ckpoint_cb = ModelCheckpoint(prefix="checkpoint_lenet", directory='./trained_ckpt_file/', config=config_ck)
+    model = Model(network, net_loss, net_opt, metrics={"Accuracy": Accuracy()})
+
+    LOGGER.info(TAG, "============== Starting Training ==============")
+    model.train(epoch_size, ds, callbacks=[ckpoint_cb, LossMonitor()], dataset_sink_mode=False) # train
+
+    LOGGER.info(TAG, "============== Starting Testing ==============")
+    param_dict = load_checkpoint("trained_ckpt_file/checkpoint_lenet-10_1875.ckpt")
+    load_param_into_net(network, param_dict)
+    ds_eval = generate_mnist_dataset(os.path.join(mnist_path, "test"), batch_size=batch_size)
+    acc = model.eval(ds_eval)
+    LOGGER.info(TAG, "============== Accuracy: %s ==============", acc)
+
+
+if __name__ == '__main__':
+    mnist_train(10, 32, 0.001, 0.9)

mindarmour/__init__.py

+"""
+MindArmour, a tool box of MindSpore to enhance model security and
+trustworthiness against adversarial examples.
+"""
+from .attacks import Attack
+from .attacks.black.black_model import BlackModel
+from .defenses.defense import Defense
+from .detectors.detector import Detector
+
+__all__ = ['Attack',
+           'BlackModel',
+           'Detector',
+           'Defense']

mindarmour/attacks/__init__.py

+"""
+This module includes classical black-box and white-box attack algorithms
+in making adversarial examples.
+"""
+from .gradient_method import *
+from .iterative_gradient_method import *
+from .deep_fool import DeepFool
+from .jsma import JSMAAttack
+from .carlini_wagner import CarliniWagnerL2Attack
+from .lbfgs import LBFGS
+from . import black
+from .black.hop_skip_jump_attack import HopSkipJumpAttack
+from .black.genetic_attack import GeneticAttack
+from .black.natural_evolutionary_strategy import NES
+from .black.pointwise_attack import PointWiseAttack
+from .black.pso_attack import PSOAttack
+from .black.salt_and_pepper_attack import SaltAndPepperNoiseAttack
+
+__all__ = ['FastGradientMethod',
+           'RandomFastGradientMethod',
+           'FastGradientSignMethod',
+           'RandomFastGradientSignMethod',
+           'LeastLikelyClassMethod',
+           'RandomLeastLikelyClassMethod',
+           'IterativeGradientMethod',
+           'BasicIterativeMethod',
+           'MomentumIterativeMethod',
+           'ProjectedGradientDescent',
+           'DeepFool',
+           'CarliniWagnerL2Attack',
+           'JSMAAttack',
+           'LBFGS',
+           'GeneticAttack',
+           'HopSkipJumpAttack',
+           'NES',
+           'PointWiseAttack',
+           'PSOAttack',
+           'SaltAndPepperNoiseAttack'
+           ]

mindarmour/attacks/attack.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+Base Class of Attack.
+"""
+from abc import abstractmethod
+
+import numpy as np
+
+from mindarmour.utils._check_param import check_pair_numpy_param, \
+    check_int_positive
+from mindarmour.utils.logger import LogUtil
+
+LOGGER = LogUtil.get_instance()
+TAG = 'Attack'
+
+
+class Attack:
+    """
+    The abstract base class for all attack classes creating adversarial examples.
+    """
+    def __init__(self):
+        pass
+
+    def batch_generate(self, inputs, labels, batch_size=64):
+        """
+        Generate adversarial examples in batch, based on input samples and
+        their labels.
+
+        Args:
+            inputs (numpy.ndarray): Samples based on which adversarial
+                examples are generated.
+            labels (numpy.ndarray): Labels of samples, whose values determined
+                by specific attacks.
+            batch_size (int): The number of samples in one batch.
+
+        Returns:
+            numpy.ndarray, generated adversarial examples
+
+        Examples:
+            >>> inputs = Tensor([[0.2, 0.4, 0.5, 0.2], [0.7, 0.2, 0.4, 0.3]])
+            >>> labels = [3, 0]
+            >>> advs = attack.batch_generate(inputs, labels, batch_size=2)
+        """
+        arr_x, arr_y = check_pair_numpy_param('inputs', inputs, 'labels', labels)
+        len_x = arr_x.shape[0]
+        batch_size = check_int_positive('batch_size', batch_size)
+        batchs = int(len_x / batch_size)
+        rest = len_x - batchs*batch_size
+        res = []
+        for i in range(batchs):
+            x_batch = arr_x[i*batch_size: (i + 1)*batch_size]
+            y_batch = arr_y[i*batch_size: (i + 1)*batch_size]
+            adv_x = self.generate(x_batch, y_batch)
+            # Black-attack methods will return 3 values, just get the second.
+            res.append(adv_x[1] if isinstance(adv_x, tuple) else adv_x)
+
+        if rest != 0:
+            x_batch = arr_x[batchs*batch_size:]
+            y_batch = arr_y[batchs*batch_size:]
+            adv_x = self.generate(x_batch, y_batch)
+            # Black-attack methods will return 3 values, just get the second.
+            res.append(adv_x[1] if isinstance(adv_x, tuple) else adv_x)
+
+
+        adv_x = np.concatenate(res, axis=0)
+        return adv_x
+
+    @abstractmethod
+    def generate(self, inputs, labels):
+        """
+        Generate adversarial examples based on normal samples and their labels.
+
+        Args:
+            inputs (numpy.ndarray): Samples based on which adversarial
+                examples are generated.
+            labels (numpy.ndarray): Labels of samples, whose values determined
+                by specific attacks.
+
+        Raises:
+            NotImplementedError: It is an abstract method.
+        """
+        msg = 'The function generate() is an abstract function in class ' \
+              '`Attack` and should be implemented in child class.'
+        LOGGER.error(TAG, msg)
+        raise NotImplementedError(msg)

mindarmour/attacks/black/black_model.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+Black model.
+"""
+from abc import abstractmethod
+
+import numpy as np
+
+from mindarmour.utils.logger import LogUtil
+
+LOGGER = LogUtil.get_instance()
+TAG = 'BlackModel'
+
+
+class BlackModel:
+    """
+    The abstract class which treats the target model as a black box. The model
+    should be defined by users.
+    """
+    def __init__(self):
+        pass
+
+    @abstractmethod
+    def predict(self, inputs):
+        """
+        Predict using the user specified model. The shape of predict results
+        should be (m, n), where n represents the number of classes this model
+        classifies.
+
+        Args:
+            inputs (numpy.ndarray): The input samples to be predicted.
+
+        Raises:
+            NotImplementedError: It is an abstract method.
+        """
+        msg = 'The function predict() is an abstract function in class ' \
+              '`BlackModel` and should be implemented in child class by user.'
+        LOGGER.error(TAG, msg)
+        raise NotImplementedError(msg)
+
+    def is_adversarial(self, data, label, is_targeted):
+        """
+        Check if input sample is adversarial example or not.
+
+        Args:
+            data (numpy.ndarray): The input sample to be check, typically some
+                maliciously perturbed examples.
+            label (numpy.ndarray): For targeted attacks, label is intended
+                label of perturbed example. For untargeted attacks, label is
+                original label of corresponding unperturbed sample.
+            is_targeted (bool): For targeted/untargeted attacks, select True/False.
+
+        Returns:
+            bool.
+                - If True, the input sample is adversarial.
+
+                - If False, the input sample is not adversarial.
+        """
+        logits = self.predict(np.expand_dims(data, axis=0))[0]
+        predicts = np.argmax(logits)
+        if is_targeted:
+            return predicts == label
+        return predicts != label

mindarmour/attacks/black/genetic_attack.py

+# Copyright 2019 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+Genetic-Attack.
+"""
+import numpy as np
+from scipy.special import softmax
+
+from mindarmour.attacks.attack import Attack
+from mindarmour.utils.logger import LogUtil
+from mindarmour.attacks.black.black_model import BlackModel
+from mindarmour.utils._check_param import check_numpy_param, check_model, \
+    check_pair_numpy_param, check_param_type, check_value_positive, \
+    check_int_positive, check_param_multi_types
+
+
+LOGGER = LogUtil.get_instance()
+TAG = 'GeneticAttack'
+
+
+def _mutation(cur_pop, step_noise=0.01, prob=0.005):
+    """
+    Generate mutation samples in genetic_attack.
+
+    Args:
+        cur_pop (numpy.ndarray): Samples before mutation.
+        step_noise (float): Noise range. Default: 0.01.
+        prob (float): Mutation probability. Default: 0.005.
+
+    Returns:
+        numpy.ndarray, samples after mutation operation in genetic_attack.
+
+    Examples:
+        >>> mul_pop = self._mutation_op([0.2, 0.3, 0.4], step_noise=0.03,
+        >>> prob=0.01)
+    """
+    cur_pop = check_numpy_param('cur_pop', cur_pop)
+    perturb_noise = np.clip(np.random.random(cur_pop.shape) - 0.5,
+                            -step_noise, step_noise)
+    mutated_pop = perturb_noise*(
+        np.random.random(cur_pop.shape) < prob) + cur_pop
+    return mutated_pop
+
+
+class GeneticAttack(Attack):
+    """
+    The Genetic Attack represents the black-box attack based on the genetic algorithm,
+    which belongs to differential evolution algorithms.
+
+    This attack was proposed by Moustafa Alzantot et al. (2018).
+
+    References: `Moustafa Alzantot, Yash Sharma, Supriyo Chakraborty,
+    "GeneticAttack: Practical Black-box Attacks with
+    Gradient-FreeOptimization" <https://arxiv.org/abs/1805.11090>`_
+
+    Args:
+        model (BlackModel): Target model.
+        pop_size (int): The number of particles, which should be greater than
+            zero. Default: 6.
+        mutation_rate (float): The probability of mutations. Default: 0.005.
+        per_bounds (float): Maximum L_inf distance.
+        max_steps (int): The maximum round of iteration for each adversarial
+            example. Default: 1000.
+        step_size (float): Attack step size. Default: 0.2.
+        temp (float): Sampling temperature for selection. Default: 0.3.
+        bounds (tuple): Upper and lower bounds of data. In form of (clip_min,
+            clip_max). Default: (0, 1.0)
+        adaptive (bool): If True, turns on dynamic scaling of mutation
+            parameters. If false, turns on static mutation parameters.
+            Default: False.
+        sparse (bool): If True, input labels are sparse-encoded. If False,
+            input labels are one-hot-encoded. Default: True.
+
+    Examples:
+        >>> attack = GeneticAttack(model)
+    """
+    def __init__(self, model, pop_size=6,
+                 mutation_rate=0.005, per_bounds=0.15, max_steps=1000,
+                 step_size=0.20, temp=0.3, bounds=(0, 1.0), adaptive=False,
+                 sparse=True):
+        super(GeneticAttack, self).__init__()
+        self._model = check_model('model', model, BlackModel)
+        self._per_bounds = check_value_positive('per_bounds', per_bounds)
+        self._pop_size = check_int_positive('pop_size', pop_size)
+        self._step_size = check_value_positive('step_size', step_size)
+        self._temp = check_value_positive('temp', temp)
+        self._max_steps = check_int_positive('max_steps', max_steps)
+        self._mutation_rate = check_value_positive('mutation_rate',
+                                                   mutation_rate)
+        self._adaptive = check_param_type('adaptive', adaptive, bool)