Merge pull request #108 from jhjiangcs/smc612

add lenet network demo, improve mnist demo.

python/paddle_fl/mpc/examples/lenet_with_mnist/README.md

+## Instructions for PaddleFL-MPC MNIST Demo
+
+([简体中文](./README_CN.md)|English)
+
+This document introduces how to run MNIST demo based on Paddle-MPC, which has two ways of running, i.e., single machine and multi machines.
+
+### 1. Running on Single Machine
+
+#### (1). Prepare Data
+
+Copy scripts `../logistic_with_mnist/process_data.py` and `../logistic_with_mnist/decrypt_save.py` into this demo's directory `lenet_with_mnist`. Generate encrypted training and testing data utilizing `generate_encrypted_data()` and `generate_encrypted_test_data()` in `process_data.py` script. Users can run the script with command `python process_data.py` to generate encrypted feature and label in given directory, e.g., `./mpc_data/`. Users can specify `class_num` (2 or 10) to determine the encrypted data is for `fc_sigmoid`(two classes) or `lenet`(10 classes) network.  Different suffix names are used for these files to indicate the ownership of different computation parties. For instance, a file named `mnist2_feature.part0` means it is a feature file of party 0.
+
+#### (2). Launch Demo with A Shell Script
+
+You should set the env params as follow:
+
+```
+export PYTHON=/yor/python
+export PATH_TO_REDIS_BIN=/path/to/redis_bin
+export LOCALHOST=/your/localhost
+export REDIS_PORT=/your/redis/port
+```
+
+Launch demo with the `run_standalone.sh` script. The concrete command is:
+
+```bash
+bash run_standalone.sh train_fc_sigmoid.py
+```
+
+The information of current epoch and step will be displayed on screen while training, as well as the total cost time when traning finished.
+
+Besides, predictions would be made in this demo once training is finished. The predictions with cypher text format would be save in `./mpc_infer_data/` directory (users can modify it in the python script `train_fc_sigmoid.py`), and the format of file name is similar to what is described in Step 1.
+
+#### (3). Decrypt Data
+
+Decrypt the saved prediction data and save the decrypted prediction results into a specified file using `decrypt_data_to_file()` in `process_data.py` script. For example, users can write the following code into a python script named `decrypt_save.py`, and then run the script with command `python decrypt_save.py decrypt_file`. The decrypted prediction results would be saved into `decrypt_file`.
+
+```python
+import sys
+
+decrypt_file=sys.argv[1]
+import process_data
+process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), decrypt_file)
+```
+
+
+### 2. Running on Multi Machines
+
+#### (1). Prepare Data
+
+Data owner encrypts data. Concrete operations are consistent with “Prepare Data” in “Running on Single Machine”.
+
+#### (2). Distribute Encrypted Data
+
+According to the suffix of file name, distribute encrypted data files to `./mpc_data/ ` directories of all 3 computation parties. For example, send `mnist2_feature.part0` and `mnist2_label.part0` to `./mpc_data/` of party 0 with `scp` command.
+
+#### (3). Modify mnist_demo.py
+
+Each computation party modifies `localhost` in the following code as the IP address of it's machine.
+
+```python
+pfl_mpc.init("aby3", int(role), "localhost", server, int(port))
+```
+
+#### (4). Launch Demo on Each Party
+
+**Note** that Redis service is necessary for demo running. Remember to clear the cache of Redis server before launching demo on each computation party, in order to avoid any negative influences caused by the cached records in Redis. The following command can be used for clear Redis, where REDIS_BIN is the executable binary of redis-cli, SERVER and PORT represent the IP and port of Redis server respectively.
+
+```
+$REDIS_BIN -h $SERVER -p $PORT flushall
+```
+
+Launch demo on each computation party with the following command,
+
+```
+$PYTHON_EXECUTABLE train_fc_sigmoid.py $PARTY_ID $SERVER $PORT
+```
+
+where PYTHON_EXECUTABLE is the python which installs PaddleFL, PARTY_ID is the ID of computation party, which is 0, 1, or 2, SERVER and PORT represent the IP and port of Redis server respectively.
+
+Similarly, predictions with cypher text format would be saved in `./mpc_infer_data/` directory, for example, a file named `mnist_output_prediction.part0` for party 0.
+
+#### (5). Decrypt Prediction Data
+
+Each computation party sends  `mnist_output_prediction.part` file in `./mpc_infer_data/` directory to the `./mpc_infer_data/` directory of data owner. Data owner decrypts the prediction data and saves the decrypted prediction results into a specified file using `decrypt_data_to_file()` in `process_data.py` script. For example, users can write the following code into a python script named `decrypt_save.py`, and then run the script with command `python decrypt_save.py decrypt_file`. The decrypted prediction results would be saved into file `decrypt_file`.
+
+```python
+import sys
+
+decrypt_file=sys.argv[1]
+import process_data
+process_data.decrypt_data_to_file("./mpc_infer_data/mnist_output_prediction", (BATCH_SIZE,), decrypt_file)
+```
+

python/paddle_fl/mpc/examples/lenet_with_mnist/README_CN.md

+## PaddleFL-MPC MNIST LeNet Demo运行说明
+
+(简体中文|[English](./README.md))
+
+本示例介绍基于PaddleFL-MPC进行MNIST数据集LeNet模型训练和预测的使用说明，分为单机运行和多机运行两种方式。
+
+### 一. 单机运行
+
+#### 1. 准备数据
+
+将数据处理脚本`../logistic_with_mnist/process_data.py`和预测结果解密脚本`../logistic_with_mnist/decrypt_save.py`复制到本demo目录`lenet_with_mnist`。使用`process_data.py`脚本中的`generate_encrypted_data()`和`generate_encrypted_test_data()`产生加密训练数据和测试数据，用户可以直接运行脚本`python process_data.py`在指定的目录下（比如`./mpc_data/`）产生加密特征和标签。用户可以通过参数`class_num`指定label的类别数目，从而产生适用于`fc_sigmoid`（二分类）和`lenet`（十分类）网络的加密数据。在指定目录下生成对应于3个计算party的feature和label的加密数据文件，以后缀名区分属于不同party的数据。比如，`mnist10_feature.part0`表示属于party0的feature数据。
+
+#### 2. 使用shell脚本启动demo
+
+运行demo之前，需设置以下环境变量：
+
+```
+export PYTHON=/yor/python
+export PATH_TO_REDIS_BIN=/path/to/redis_bin
+export LOCALHOST=/your/localhost
+export REDIS_PORT=/your/redis/port
+```
+
+然后使用`run_standalone.sh`脚本，启动并运行demo，命令如下：
+
+```bash 
+bash run_standalone.sh train_lenet.py
+```
+
+运行之后将在屏幕上打印训练过程中所处的epoch和step，并在完成训练后打印出训练花费的时间。
+
+此外，在完成训练之后，demo会继续进行预测，并将预测密文结果保存到./mpc_infer_data/目录下的文件中，文件命名格式类似于步骤1中所述。
+
+#### 3. 解密数据
+
+使用`process_data.py`脚本中的`decrypt_data_to_file()`，将保存的密文预测结果进行解密，并且将解密得到的明文预测结果保存到指定文件中。例如，将下面的内容写到一个`decrypt_save.py`脚本中，然后`python decrypt_save.py decrypt_file`，将把明文预测结果保存在`decrypt_file`文件中。
+
+```python
+import sys
+
+decrypt_file=sys.argv[1]
+import process_data
+process_data.decrypt_data_to_file("./mpc_infer_data/mnist_output_prediction", (BATCH_SIZE, 10), decrypt_file)
+```
+
+
+### 二. 多机运行
+
+#### 1. 准备数据
+
+数据方对数据进行加密处理。具体操作和单机运行中的准备数据步骤一致。
+
+#### 2. 分发数据
+
+按照后缀名，将步骤1中准备好的数据分别发送到对应的计算party的./mpc_data/目录下。比如，使用scp命令，将
+
+`mnist10_feature.part0`和`mnist10_label.part0`发送到party0的./mpc_data/目录下。
+
+#### 3. 修改各计算party的train_lenet脚本
+
+各计算party根据自己的机器环境，将脚本如下内容中的`localhost`修改为自己的IP地址：
+
+```python
+pfl_mpc.init("aby3", int(role), "localhost", server, int(port))
+```
+
+#### 4. 各计算party启动demo
+
+**注意**：运行需要用到redis服务。为了确保redis中已保存的数据不会影响demo的运行，请在各计算party启动demo之前，使用如下命令清空redis。其中，REDIS_BIN表示redis-cli可执行程序，SERVER和PORT分别表示redis server的IP地址和端口号。
+
+```
+$REDIS_BIN -h $SERVER -p $PORT flushall
+```
+
+在各计算party分别执行以下命令，启动demo：
+
+```
+$PYTHON_EXECUTABLE train_lenet.py $PARTY_ID $SERVER $PORT
+```
+
+其中，PYTHON_EXECUTABLE表示自己安装了PaddleFL的python，PARTY_ID表示计算party的编号，值为0、1或2，SERVER和PORT分别表示redis server的IP地址和端口号。
+
+同样地，密文prediction数据将会保存到`./mpc_infer_data/`目录下的文件中。比如，在party0中将保存为文件`mnist_output_prediction.part0`.
+
+#### 5. 解密预测数据
+
+各计算party将`./mpc_infer_data/`目录下的`mnist_output_prediction.part`文件发送到数据方的`./mpc_infer_data/`目录下。数据方使用`process_data.py`脚本中的`decrypt_data_to_file()`，将密文预测结果进行解密，并且将解密得到的明文预测结果保存到指定文件中。例如，将下面的内容写到一个`decrypt_save.py`脚本中，然后`python decrypt_save.py decrypt_file`，将把明文预测结果保存在`decrypt_file`文件中。
+
+```python
+import sys
+
+decrypt_file=sys.argv[1]
+import process_data
+process_data.decrypt_data_to_file("./mpc_infer_data/mnist_output_prediction", (BATCH_SIZE, 10), decrypt_file)
+```
+

python/paddle_fl/mpc/examples/lenet_with_mnist/train_lenet.py

+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+MNIST CNN Demo (LeNet5)
+"""
+
+import sys
+import os
+import numpy as np
+import time
+import logging
+import math
+
+import paddle
+import paddle.fluid as fluid
+import paddle.fluid.profiler as profiler
+import paddle_fl.mpc as pfl_mpc
+import paddle_fl.mpc.data_utils.aby3 as aby3
+
+logging.basicConfig(format='%(asctime)s - %(levelname)s - %(message)s')
+logger = logging.getLogger("fluid")
+logger.setLevel(logging.INFO)
+
+role, server, port = sys.argv[1], sys.argv[2], sys.argv[3]
+# modify host(localhost).
+pfl_mpc.init("aby3", int(role), "localhost", server, int(port))
+role = int(role)
+
+# data preprocessing
+BATCH_SIZE = 128
+epoch_num = 5
+
+x = pfl_mpc.data(name='x', shape=[BATCH_SIZE, 1, 28, 28], dtype='int64')
+y = pfl_mpc.data(name='y', shape=[BATCH_SIZE, 10], dtype='int64')
+
+
+class Model(object):
+    def __int__(self):
+        pass
+
+    def lenet3(self):
+        conv = pfl_mpc.layers.conv2d(input=x, num_filters=16, filter_size=5, act='relu')
+        pool = pfl_mpc.layers.pool2d(input=conv, pool_size=2, pool_stride=2)
+        fc_1 = pfl_mpc.layers.fc(input=pool, size=100, act='relu')
+        fc_out = pfl_mpc.layers.fc(input=fc_1, size=10)
+        cost, softmax = pfl_mpc.layers.softmax_with_cross_entropy(logits=fc_out,
+                                                                  label=y,
+                                                                  soft_label=True,
+                                                                  return_softmax=True)
+        return cost, softmax
+
+    def lenet5(self):
+        conv_1 = pfl_mpc.layers.conv2d(input=x, num_filters=16, filter_size=5, act='relu')
+        pool_1 = pfl_mpc.layers.pool2d(input=conv_1, pool_size=2, pool_stride=2)
+        conv_2 = pfl_mpc.layers.conv2d(input=pool_1, num_filters=16, filter_size=5, act='relu')
+        pool_2 = pfl_mpc.layers.pool2d(input=conv_2, pool_size=2, pool_stride=2)
+        fc_1 = pfl_mpc.layers.fc(input=pool_2, size=100, act='relu')
+        fc_out = pfl_mpc.layers.fc(input=fc_1, size=10)
+        cost, softmax = pfl_mpc.layers.softmax_with_cross_entropy(logits=fc_out,
+                                                                  label=y,
+                                                                  soft_label=True,
+                                                                  return_softmax=True)
+        return cost, softmax
+
+model = Model()
+cost, softmax = model.lenet5()
+
+infer_program = fluid.default_main_program().clone(for_test=False)
+
+avg_loss = pfl_mpc.layers.mean(cost)
+optimizer = pfl_mpc.optimizer.SGD(learning_rate=0.1)
+optimizer.minimize(avg_loss)
+
+# prepare train and test reader
+mpc_data_dir = "./mpc_data/"
+if not os.path.exists(mpc_data_dir):
+    raise ValueError("mpc_data_dir is not found. Please prepare encrypted data.")
+
+# train_reader
+feature_reader = aby3.load_aby3_shares(mpc_data_dir + "mnist10_feature", id=role, shape=(1, 28, 28))
+label_reader = aby3.load_aby3_shares(mpc_data_dir + "mnist10_label", id=role, shape=(10,))
+batch_feature = aby3.batch(feature_reader, BATCH_SIZE, drop_last=True)
+batch_label = aby3.batch(label_reader, BATCH_SIZE, drop_last=True)
+
+# test_reader
+test_feature_reader = aby3.load_aby3_shares(mpc_data_dir + "mnist10_test_feature", id=role, shape=(1, 28, 28))
+test_label_reader = aby3.load_aby3_shares(mpc_data_dir + "mnist10_test_label", id=role, shape=(10,))
+test_batch_feature = aby3.batch(test_feature_reader, BATCH_SIZE, drop_last=True)
+test_batch_label = aby3.batch(test_label_reader, BATCH_SIZE, drop_last=True)
+
+place = fluid.CPUPlace()
+
+# async data loader
+loader = fluid.io.DataLoader.from_generator(feed_list=[x, y], capacity=BATCH_SIZE)
+batch_sample = paddle.reader.compose(batch_feature, batch_label)
+loader.set_batch_generator(batch_sample, places=place)
+
+test_loader = fluid.io.DataLoader.from_generator(feed_list=[x, y], capacity=BATCH_SIZE)
+test_batch_sample = paddle.reader.compose(test_batch_feature, test_batch_label)
+test_loader.set_batch_generator(test_batch_sample, places=place)
+
+# infer
+def infer():
+    """
+    MPC infer
+    """
+    mpc_infer_data_dir = "./mpc_infer_data/"
+    if not os.path.exists(mpc_infer_data_dir):
+        os.mkdir(mpc_infer_data_dir)
+    prediction_file = mpc_infer_data_dir + "mnist_debug_prediction"
+    prediction_file_part = prediction_file + ".part{}".format(role)
+
+    if os.path.exists(prediction_file_part):
+        os.remove(prediction_file_part)
+
+    step = 0
+    start_time = time.time()
+    for sample in test_loader():
+        step += 1
+        prediction = exe.run(program=infer_program, feed=sample, fetch_list=[softmax])
+        with open(prediction_file_part, 'ab') as f:
+            f.write(np.array(prediction).tostring())
+        if step % 10 == 0:
+            end_time = time.time()
+            logger.info('MPC infer of step={}, cost time in seconds:{}'.format(step, (end_time - start_time)))
+
+    end_time = time.time()
+    logger.info('MPC infer time in seconds:{}'.format((end_time - start_time)))
+
+# train
+exe = fluid.Executor(place)
+exe.run(fluid.default_startup_program())
+
+mpc_model_basedir = "./mpc_model/"
+
+step = 0
+start_time = time.time()
+logger.info('MPC training start...')
+for epoch_id in range(epoch_num):
+    for sample in loader():
+        step += 1
+        results = exe.run(feed=sample, fetch_list=[softmax])
+        if step % 100 == 0:
+            end_time = time.time()
+            logger.info('MPC training of epoch_id={} step={},  cost time in seconds:{}'
+                        .format(epoch_id, step, (end_time - start_time)))
+    
+    # For each epoch: infer or save infer program
+    #infer()
+    mpc_model_dir = mpc_model_basedir + "epoch{}/party{}".format(epoch_id, role)
+    fluid.io.save_inference_model(dirname=mpc_model_dir,
+        feeded_var_names=["x", "y"],
+        target_vars=[softmax],
+        executor=exe,
+        main_program=infer_program,
+        model_filename="__model__")
+
+end_time = time.time()
+logger.info('MPC training of epoch_num={} batch_size={}, cost time in seconds:{}'
+      .format(epoch_num, BATCH_SIZE, (end_time - start_time)))
+
+# infer
+infer()
+

python/paddle_fl/mpc/examples/mnist_demo/README.md → python/paddle_fl/mpc/examples/logistic_with_mnist/README.md

@@ -8,15 +8,7 @@ This document introduces how to run MNIST demo based on Paddle-MPC, which has tw
 
 #### (1). Prepare Data
 
-Generate encrypted training and testing data utilizing `generate_encrypted_data()` and `generate_encrypted_test_data()` in `process_data.py` script. For example, users can write the following code into a python script named `prepare.py`, and then run the script with command `python prepare.py`.
-
-```python
-import process_data
-process_data.generate_encrypted_data()
-process_data.generate_encrypted_test_data()
-```
-
-Encrypted data files of feature and label would be generated and saved in `/tmp` directory. Different suffix names are used for these files to indicate the ownership of different computation parties. For instance, a file named `mnist2_feature.part0` means it is a feature file of party 0.
+Generate encrypted training and testing data utilizing `generate_encrypted_data()` and `generate_encrypted_test_data()` in `process_data.py` script. Users can run the script with command `python process_data.py` to generate encrypted feature and label in given directory, e.g., `./mpc_data/`. Users can specify `class_num` (2 or 10) to determine the encrypted data is for `logisticfc_sigmoid`(two classes) or `lenet` and `logistic_fc_softmax`(10 classes) network.  Different suffix names are used for these files to indicate the ownership of different computation parties. For instance, a file named `mnist2_feature.part0` means it is a feature file of party 0.
 
 #### (2). Launch Demo with A Shell Script
 
@@ -32,12 +24,12 @@ export REDIS_PORT=/your/redis/port
 Launch demo with the `run_standalone.sh` script. The concrete command is:
 
 ```bash
-bash run_standalone.sh mnist_demo.py
+bash run_standalone.sh train_fc_sigmoid.py
 ```
 
 The information of current epoch and step will be displayed on screen while training, as well as the total cost time when traning finished.
 
-Besides, predictions would be made in this demo once training is finished. The predictions with cypher text format would be save in `/tmp` directory, and the format of file name is similar to what is described in Step 1.
+Besides, predictions would be made in this demo once training is finished. The predictions with cypher text format would be save in `./mpc_infer_data/` directory (users can modify it in the python script `train_fc_sigmoid.py`), and the format of file name is similar to what is described in Step 1.
 
 #### (3). Decrypt Data
 
@@ -51,17 +43,6 @@ import process_data
 process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), decrypt_file)
 ```
 
-**Note** that remember to delete the prediction files in `/tmp` directory generated in last running, in case of any influence on the decrypted results of current running. For simplifying users operations, we provide the following commands in `run_standalone.sh`, which can delete the files mentioned above when running this script.
-
-```bash
-# remove temp data generated in last time
-PRED_FILE="/tmp/mnist_output_prediction.*"
-if [ "$PRED_FILE" ]; then
-        rm -rf $PRED_FILE
-fi
-```
-
-
 
 ### 2. Running on Multi Machines
 
@@ -71,9 +52,9 @@ Data owner encrypts data. Concrete operations are consistent with “Prepare Dat
 
 #### (2). Distribute Encrypted Data
 
-According to the suffix of file name, distribute encrypted data files to `/tmp ` directories of all 3 computation parties. For example, send `mnist2_feature.part0` and `mnist2_label.part0` to `/tmp` of party 0 with `scp` command.
+According to the suffix of file name, distribute encrypted data files to `./mpc_data/ ` directories of all 3 computation parties. For example, send `mnist2_feature.part0` and `mnist2_label.part0` to `./mpc_data/` of party 0 with `scp` command.
 
-#### (3). Modify mnist_demo.py
+#### (3). Modify train_fc_sigmoid.py
 
 Each computation party modifies `localhost` in the following code as the IP address of it's machine.
 
@@ -92,24 +73,22 @@ $REDIS_BIN -h $SERVER -p $PORT flushall
 Launch demo on each computation party with the following command,
 
 ```
-$PYTHON_EXECUTABLE mnist_demo.py $PARTY_ID $SERVER $PORT
+$PYTHON_EXECUTABLE train_fc_sigmoid.py $PARTY_ID $SERVER $PORT
 ```
 
 where PYTHON_EXECUTABLE is the python which installs PaddleFL, PARTY_ID is the ID of computation party, which is 0, 1, or 2, SERVER and PORT represent the IP and port of Redis server respectively.
 
-Similarly, predictions with cypher text format would be saved in `/tmp` directory, for example, a file named `mnist_output_prediction.part0` for party 0.
-
-**Note** that remember to delete the prediction files in `/tmp` directory generated in last running, in case of any influence on the decrypted results of current running.
+Similarly, predictions with cypher text format would be saved in `./mpc_infer_data/` directory, for example, a file named `mnist_output_prediction.part0` for party 0.
 
 #### (5). Decrypt Prediction Data
 
-Each computation party sends  `mnist_output_prediction.part` file in `/tmp` directory to the `/tmp` directory of data owner. Data owner decrypts the prediction data and saves the decrypted prediction results into a specified file using `decrypt_data_to_file()` in `process_data.py` script. For example, users can write the following code into a python script named `decrypt_save.py`, and then run the script with command `python decrypt_save.py decrypt_file`. The decrypted prediction results would be saved into file `decrypt_file`.
+Each computation party sends  `mnist_output_prediction.part` file in `./mpc_infer_data/` directory to the `./mpc_infer_data/` directory of data owner. Data owner decrypts the prediction data and saves the decrypted prediction results into a specified file using `decrypt_data_to_file()` in `process_data.py` script. For example, users can write the following code into a python script named `decrypt_save.py`, and then run the script with command `python decrypt_save.py decrypt_file`. The decrypted prediction results would be saved into file `decrypt_file`.
 
 ```python
 import sys
 
 decrypt_file=sys.argv[1]
 import process_data
-process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), decrypt_file)
+process_data.decrypt_data_to_file("./mpc_infer_data/mnist_output_prediction", (BATCH_SIZE,), decrypt_file)
 ```
 

python/paddle_fl/mpc/examples/mnist_demo/README_CN.md → python/paddle_fl/mpc/examples/logistic_with_mnist/README_CN.md

@@ -8,15 +8,7 @@
 
 #### 1. 准备数据
 
-使用`process_data.py`脚本中的`generate_encrypted_data()`和`generate_encrypted_test_data()`产生加密训练数据和测试数据，比如将如下内容写到一个`prepare.py`脚本中，然后`python prepare.py`
-
-```python
-import process_data
-process_data.generate_encrypted_data()
-process_data.generate_encrypted_test_data()
-```
-
-将在/tmp目录下生成对应于3个计算party的feature和label的加密数据文件，以后缀名区分属于不同party的数据。比如，`mnist2_feature.part0`表示属于party0的feature数据。
+使用`process_data.py`脚本中的`generate_encrypted_data()`和`generate_encrypted_test_data()`产生加密训练数据和测试数据，用户可以直接运行脚本`python process_data.py`在指定的目录下（比如`./mpc_data/`）产生加密特征和标签。用户可以通过参数`class_num`指定label的类别数目，从而产生适用于`logistic_fc_sigmoid`（二分类）或`lenet``logistic_fc_softmax`（十分类）网络的加密数据。在指定目录下生成对应于3个计算party的feature和label的加密数据文件，以后缀名区分属于不同party的数据。比如，`mnist2_feature.part0`表示属于party0的feature数据。
 
 #### 2. 使用shell脚本启动demo
 
@@ -32,12 +24,12 @@ export REDIS_PORT=/your/redis/port
 然后使用`run_standalone.sh`脚本，启动并运行demo，命令如下：
 
 ```bash 
-bash run_standalone.sh mnist_demo.py
+bash run_standalone.sh train_fc_sigmoid.py
 ```
 
 运行之后将在屏幕上打印训练过程中所处的epoch和step，并在完成训练后打印出训练花费的时间。
 
-此外，在完成训练之后，demo会继续进行预测，并将预测密文结果保存到/tmp目录下的文件中，文件命名格式类似于步骤1中所述。
+此外，在完成训练之后，demo会继续进行预测，并将预测密文结果保存到./mpc_infer_data/目录下的文件中，文件命名格式类似于步骤1中所述。
 
 #### 3. 解密数据
 
@@ -48,20 +40,9 @@ import sys
 
 decrypt_file=sys.argv[1]
 import process_data
-process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), decrypt_file)
+process_data.decrypt_data_to_file("./mpc_infer_data/mnist_output_prediction", (BATCH_SIZE,), decrypt_file)
 ```
 
-**注意**：再次启动运行demo之前，请先将上次在`/tmp`保存的预测密文结果文件删除，以免影响本次密文数据的恢复结果。为了简化用户操作，我们在`run_standalone.sh`脚本中加入了如下的内容，可以在执行脚本时删除上次的数据。
-
-```bash
-# remove temp data generated in last time
-PRED_FILE="/tmp/mnist_output_prediction.*"
-if [ "$PRED_FILE" ]; then
-        rm -rf $PRED_FILE
-fi
-```
-
-
 
 ### 二. 多机运行
 
@@ -71,11 +52,11 @@ fi
 
 #### 2. 分发数据
 
-按照后缀名，将步骤1中准备好的数据分别发送到对应的计算party的/tmp目录下。比如，使用scp命令，将
+按照后缀名，将步骤1中准备好的数据分别发送到对应的计算party的./mpc_data/目录下。比如，使用scp命令，将
 
-`mnist2_feature.part0`和`mnist2_label.part0`发送到party0的/tmp目录下。
+`mnist2_feature.part0`和`mnist2_label.part0`发送到party0的./mpc_data/目录下。
 
-#### 3. 计算party修改mnist_demo.py脚本
+#### 3. 修改各计算party的train_fc_sigmoid.py脚本
 
 各计算party根据自己的机器环境，将脚本如下内容中的`localhost`修改为自己的IP地址：
 
@@ -94,24 +75,22 @@ $REDIS_BIN -h $SERVER -p $PORT flushall
 在各计算party分别执行以下命令，启动demo：
 
 ```
-$PYTHON_EXECUTABLE mnist_demo.py $PARTY_ID $SERVER $PORT
+$PYTHON_EXECUTABLE train_fc_sigmoid.py $PARTY_ID $SERVER $PORT
 ```
 
 其中，PYTHON_EXECUTABLE表示自己安装了PaddleFL的python，PARTY_ID表示计算party的编号，值为0、1或2，SERVER和PORT分别表示redis server的IP地址和端口号。
 
-同样地，密文prediction数据将会保存到`/tmp`目录下的文件中。比如，在party0中将保存为文件`mnist_output_prediction.part0`.
-
-**注意**：再次启动运行demo之前，请先将上次在`/tmp`保存的prediction文件删除，以免影响本次密文数据的恢复结果。
+同样地，密文prediction数据将会保存到`./mpc_infer_data/`目录下的文件中。比如，在party0中将保存为文件`mnist_output_prediction.part0`.
 
 #### 5. 解密预测数据
 
-各计算party将`/tmp`目录下的`mnist_output_prediction.part`文件发送到数据方的/tmp目录下。数据方使用`process_data.py`脚本中的`decrypt_data_to_file()`，将密文预测结果进行解密，并且将解密得到的明文预测结果保存到指定文件中。例如，将下面的内容写到一个`decrypt_save.py`脚本中，然后`python decrypt_save.py decrypt_file`，将把明文预测结果保存在`decrypt_file`文件中。
+各计算party将`./mpc_infer_data/`目录下的`mnist_output_prediction.part`文件发送到数据方的`./mpc_infer_data/`目录下。数据方使用`process_data.py`脚本中的`decrypt_data_to_file()`，将密文预测结果进行解密，并且将解密得到的明文预测结果保存到指定文件中。例如，将下面的内容写到一个`decrypt_save.py`脚本中，然后`python decrypt_save.py decrypt_file`，将把明文预测结果保存在`decrypt_file`文件中。
 
 ```python
 import sys
 
 decrypt_file=sys.argv[1]
 import process_data
-process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), decrypt_file)
+process_data.decrypt_data_to_file("./mpc_infer_data/mnist_output_prediction", (BATCH_SIZE,), decrypt_file)
 ```
 

python/paddle_fl/mpc/examples/mnist_demo/decrypt_save.py → python/paddle_fl/mpc/examples/logistic_with_mnist/decrypt_save.py

@@ -15,9 +15,23 @@
 Decrypt Prediction Data.
 """
 import sys
-
-import process_data
+import os
+from process_data import decrypt_data_to_file
 
 decrypt_file=sys.argv[1]
 BATCH_SIZE=128
-process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), decrypt_file)
+class_num = 2
+
+mpc_infer_data_dir = "./mpc_infer_data/"
+prediction_file = mpc_infer_data_dir + "mnist_debug_prediction"
+
+if os.path.exists(decrypt_file):
+    os.remove(decrypt_file)
+
+if class_num == 2:
+    decrypt_data_to_file(prediction_file, (BATCH_SIZE,), decrypt_file)
+elif class_num == 10:
+    decrypt_data_to_file(prediction_file, (BATCH_SIZE, 10), decrypt_file)
+else:
+    raise ValueError("class_num should be 2 or 10, but received {}.".format(class_num))
+

python/paddle_fl/mpc/examples/mnist_demo/process_data.py → python/paddle_fl/mpc/examples/logistic_with_mnist/process_data.py

@@ -12,21 +12,31 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 """
-Process data for MNIST.
+Process data for MNIST: 10 classes.
 """
+import os
+import time
+import logging
 import numpy as np
-import paddle
 import six
-import os
+import paddle
 from paddle_fl.mpc.data_utils import aby3
 
+
+logging.basicConfig(format='%(asctime)s - %(levelname)s - %(message)s')
+logger = logging.getLogger("fluid")
+logger.setLevel(logging.INFO)
+
+
 sample_reader = paddle.dataset.mnist.train()
 test_reader = paddle.dataset.mnist.test()
 
-def generate_encrypted_data():
+
+def generate_encrypted_train_data(mpc_data_dir, class_num):
     """
     generate encrypted samples
     """
+
     def encrypted_mnist_features():
         """
         feature reader
@@ -39,15 +49,23 @@ def generate_encrypted_data():
         label reader
         """
         for instance in sample_reader():
-            yield aby3.make_shares(np.array(1) if instance[1] == 0 else np.array(0))
-    
-    aby3.save_aby3_shares(encrypted_mnist_features, "/tmp/mnist2_feature")
-    aby3.save_aby3_shares(encrypted_mnist_labels, "/tmp/mnist2_label")
+            if class_num == 2:
+                label = np.array(1) if instance[1] == 0 else np.array(0)
+            elif class_num == 10:
+                label = np.eye(N=1, M=10, k=instance[1], dtype=float).reshape(10)
+            else:
+                raise ValueError("class_num should be 2 or 10, but received {}.".format(class_num))
+            yield aby3.make_shares(label)
+
+    aby3.save_aby3_shares(encrypted_mnist_features, mpc_data_dir + "mnist{}_feature".format(class_num))
+    aby3.save_aby3_shares(encrypted_mnist_labels, mpc_data_dir + "mnist{}_label".format(class_num))
 
-def generate_encrypted_test_data():
+
+def generate_encrypted_test_data(mpc_data_dir, class_num, label_mnist_filepath):
     """
     generate encrypted samples
     """
+
     def encrypted_mnist_features():
         """
         feature reader
@@ -60,10 +78,21 @@ def generate_encrypted_test_data():
         label reader
         """
         for instance in test_reader():
-            yield aby3.make_shares(np.array(1) if instance[1] == 0 else np.array(0))
-    
-    aby3.save_aby3_shares(encrypted_mnist_features, "/tmp/mnist2_test_feature")
-    aby3.save_aby3_shares(encrypted_mnist_labels, "/tmp/mnist2_test_label")
+            if class_num == 2:
+                label = np.array(1) if instance[1] == 0 else np.array(0)
+                with open(label_mnist_filepath, 'a+') as f:
+                    f.write(str(1 if instance[1] == 0 else 0) + '\n')
+            elif class_num == 10:
+                label = np.eye(N=1, M=10, k=instance[1], dtype=float).reshape(10)
+                with open(label_mnist_filepath, 'a+') as f:
+                    f.write(str(instance[1]) + '\n')
+            else:
+                raise ValueError("class_num should be 2 or 10, but received {}.".format(class_num))
+            yield aby3.make_shares(label)
+
+    aby3.save_aby3_shares(encrypted_mnist_features, mpc_data_dir + "mnist{}_test_feature".format(class_num))
+    aby3.save_aby3_shares(encrypted_mnist_labels, mpc_data_dir + "mnist{}_test_label".format(class_num))
+
 
 def load_decrypt_data(filepath, shape):
     """
@@ -76,14 +105,30 @@ def load_decrypt_data(filepath, shape):
 
     for instance in aby3_share_reader():
         p = aby3.reconstruct(np.array(instance))
-        print(p)
+        logger.info(p)
+
+
+def load_decrypt_bs_data(filepath, shape):
+    """
+    load the encrypted data and reconstruct
+    """
+    part_readers = []
+    for id in six.moves.range(3):
+        part_readers.append(aby3.load_aby3_shares(filepath, id=id, shape=shape))
+    aby3_share_reader = paddle.reader.compose(part_readers[0], part_readers[1], part_readers[2])
+
+    for instance in aby3_share_reader():
+        p = np.bitwise_xor(np.array(instance[0]), np.array(instance[1]))
+        p = np.bitwise_xor(p, np.array(instance[2]))
+        logger.info(p)
 
-def decrypt_data_to_file(filepath, shape, decrypted_file):
+
+def decrypt_data_to_file(filepath, shape, decrypted_filepath):
     """
-    load the encrypted data and reconstruct to a file
+    load the encrypted data (arithmetic share) and reconstruct to a file
     """
-    if os.path.exists(decrypted_file):
-        os.remove(decrypted_file)
+    if os.path.exists(decrypted_filepath):
+        os.remove(decrypted_filepath)
     part_readers = []
     for id in six.moves.range(3):
         part_readers.append(aby3.load_aby3_shares(filepath, id=id, shape=shape))
@@ -91,6 +136,38 @@ def decrypt_data_to_file(filepath, shape, decrypted_file):
 
     for instance in aby3_share_reader():
         p = aby3.reconstruct(np.array(instance))
-        with open(decrypted_file, 'a+') as f:
+        with open(decrypted_filepath, 'a+') as f:
+            for i in p:
+                f.write(str(np.argmax(i)) + '\n')
+
+
+def decrypt_bs_data_to_file(filepath, shape, decrypted_filepath):
+    """
+    load the encrypted data (boolean share) and reconstruct to a file
+    """
+    if os.path.exists(decrypted_filepath):
+        os.remove(decrypted_filepath)
+    part_readers = []
+    for id in six.moves.range(3):
+        part_readers.append(aby3.load_aby3_shares(filepath, id=id, shape=shape))
+    aby3_share_reader = paddle.reader.compose(part_readers[0], part_readers[1], part_readers[2])
+
+    for instance in aby3_share_reader():
+        p = np.bitwise_xor(np.array(instance[0]), np.array(instance[1]))
+        p = np.bitwise_xor(p, np.array(instance[2]))
+        with open(decrypted_filepath, 'a+') as f:
             for i in p:
                 f.write(str(i) + '\n')
+
+
+if __name__ == '__main__':
+    mpc_data_dir = './mpc_data/'
+    label_mnist_filepath = mpc_data_dir + "label_mnist"
+    if not os.path.exists(mpc_data_dir):
+        os.mkdir(mpc_data_dir)
+    if os.path.exists(label_mnist_filepath):
+        os.remove(label_mnist_filepath)
+
+    class_num = 2
+    generate_encrypted_train_data(mpc_data_dir, class_num)
+    generate_encrypted_test_data(mpc_data_dir, class_num, label_mnist_filepath)

python/paddle_fl/mpc/examples/mnist_demo/mnist_demo.py → python/paddle_fl/mpc/examples/logistic_with_mnist/train_fc_sigmoid.py

@@ -49,15 +49,19 @@ avg_loss = pfl_mpc.layers.mean(cost)
 optimizer = pfl_mpc.optimizer.SGD(learning_rate=0.001)
 optimizer.minimize(avg_loss)
 
+mpc_data_dir = "./mpc_data/"
+if not os.path.exists(mpc_data_dir):
+    raise ValueError("mpc_data_dir is not found. Please prepare encrypted data.")
+
 # train_reader
-feature_reader = aby3.load_aby3_shares("/tmp/mnist2_feature", id=role, shape=(784,))
-label_reader = aby3.load_aby3_shares("/tmp/mnist2_label", id=role, shape=(1,))
+feature_reader = aby3.load_aby3_shares(mpc_data_dir + "mnist2_feature", id=role, shape=(784,))
+label_reader = aby3.load_aby3_shares(mpc_data_dir + "mnist2_label", id=role, shape=(1,))
 batch_feature = aby3.batch(feature_reader, BATCH_SIZE, drop_last=True)
 batch_label = aby3.batch(label_reader, BATCH_SIZE, drop_last=True)
 
 # test_reader
-test_feature_reader = aby3.load_aby3_shares("/tmp/mnist2_test_feature", id=role, shape=(784,))
-test_label_reader = aby3.load_aby3_shares("/tmp/mnist2_test_label", id=role, shape=(1,))
+test_feature_reader = aby3.load_aby3_shares(mpc_data_dir + "mnist2_test_feature", id=role, shape=(784,))
+test_label_reader = aby3.load_aby3_shares(mpc_data_dir + "mnist2_test_label", id=role, shape=(1,))
 test_batch_feature = aby3.batch(test_feature_reader, BATCH_SIZE, drop_last=True)
 test_batch_label = aby3.batch(test_label_reader, BATCH_SIZE, drop_last=True)
 
@@ -79,24 +83,28 @@ test_loader.set_batch_generator(test_batch_sample, places=place)
 exe = fluid.Executor(place)
 exe.run(fluid.default_startup_program())
 
+start_time = time.time()
+step = 0
 for epoch_id in range(epoch_num):
-    start_time = time.time()
-    step = 0
     # feed data via loader
     for sample in loader():
-        batch_start = time.time()
+        step += 1
         exe.run(feed=sample, fetch_list=[cost.name])
         batch_end = time.time()
         if step % 50 == 0:
-            print('Epoch={}, Step={}, batch_cost={:.4f} s'.format(epoch_id, step, (batch_end - batch_start)))
-        step += 1
+            print('Epoch={}, Step={}'.format(epoch_id, step))
 
-    end_time = time.time()
-    print('Mpc Training of Epoch={} Batch_size={}, epoch_cost={:.4f} s'
+end_time = time.time()
+print('Mpc Training of Epoch={} Batch_size={}, epoch_cost={:.4f} s'
       .format(epoch_num, BATCH_SIZE, (end_time - start_time)))
 
 # prediction
-prediction_file = "/tmp/mnist_output_prediction.part{}".format(role)
+mpc_infer_data_dir = "./mpc_infer_data/"
+if not os.path.exists(mpc_infer_data_dir):
+    os.mkdir(mpc_infer_data_dir)
+prediction_file = mpc_infer_data_dir + "mnist_debug_prediction.part{}".format(role)
+if os.path.exists(prediction_file):
+    os.remove(prediction_file)
 for sample in test_loader():
     prediction = exe.run(program=infer_program, feed=sample, fetch_list=[cost])
     with open(prediction_file, 'ab') as f:

python/paddle_fl/mpc/examples/logistic_with_mnist/train_fc_softmax.py

+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+MNIST CNN Demo (LeNet5)
+"""
+
+import sys
+import os
+import numpy as np
+import time
+import logging
+import math
+
+import paddle
+import paddle.fluid as fluid
+import paddle.fluid.profiler as profiler
+import paddle_fl.mpc as pfl_mpc
+import paddle_fl.mpc.data_utils.aby3 as aby3
+
+logging.basicConfig(format='%(asctime)s - %(levelname)s - %(message)s')
+logger = logging.getLogger("fluid")
+logger.setLevel(logging.INFO)
+
+role, server, port = sys.argv[1], sys.argv[2], sys.argv[3]
+# modify host(localhost).
+pfl_mpc.init("aby3", int(role), "localhost", server, int(port))
+role = int(role)
+
+# data preprocessing
+BATCH_SIZE = 128
+epoch_num = 5
+
+x = pfl_mpc.data(name='x', shape=[BATCH_SIZE, 1, 28, 28], dtype='int64')
+y = pfl_mpc.data(name='y', shape=[BATCH_SIZE, 10], dtype='int64')
+
+fc_out = pfl_mpc.layers.fc(input=x, size=10)
+cost, softmax = pfl_mpc.layers.softmax_with_cross_entropy(logits=fc_out,
+                                                          label=y,
+                                                          soft_label=True,
+                                                          return_softmax=True)
+
+infer_program = fluid.default_main_program().clone(for_test=False)
+
+avg_loss = pfl_mpc.layers.mean(cost)
+optimizer = pfl_mpc.optimizer.SGD(learning_rate=0.1)
+optimizer.minimize(avg_loss)
+
+# prepare train and test reader
+mpc_data_dir = "./mpc_data/"
+if not os.path.exists(mpc_data_dir):
+    raise ValueError("mpc_data_dir is not found. Please prepare encrypted data.")
+
+# train_reader
+feature_reader = aby3.load_aby3_shares(mpc_data_dir + "mnist10_feature", id=role, shape=(1, 28, 28))
+label_reader = aby3.load_aby3_shares(mpc_data_dir + "mnist10_label", id=role, shape=(10,))
+batch_feature = aby3.batch(feature_reader, BATCH_SIZE, drop_last=True)
+batch_label = aby3.batch(label_reader, BATCH_SIZE, drop_last=True)
+
+# test_reader
+test_feature_reader = aby3.load_aby3_shares(mpc_data_dir + "mnist10_test_feature", id=role, shape=(1, 28, 28))
+test_label_reader = aby3.load_aby3_shares(mpc_data_dir + "mnist10_test_label", id=role, shape=(10,))
+test_batch_feature = aby3.batch(test_feature_reader, BATCH_SIZE, drop_last=True)
+test_batch_label = aby3.batch(test_label_reader, BATCH_SIZE, drop_last=True)
+
+place = fluid.CPUPlace()
+
+# async data loader
+loader = fluid.io.DataLoader.from_generator(feed_list=[x, y], capacity=BATCH_SIZE)
+batch_sample = paddle.reader.compose(batch_feature, batch_label)
+loader.set_batch_generator(batch_sample, places=place)
+
+test_loader = fluid.io.DataLoader.from_generator(feed_list=[x, y], capacity=BATCH_SIZE)
+test_batch_sample = paddle.reader.compose(test_batch_feature, test_batch_label)
+test_loader.set_batch_generator(test_batch_sample, places=place)
+
+# infer
+def infer():
+    """
+    MPC infer
+    """
+    mpc_infer_data_dir = "./mpc_infer_data/"
+    if not os.path.exists(mpc_infer_data_dir):
+        os.mkdir(mpc_infer_data_dir)
+    prediction_file = mpc_infer_data_dir + "mnist_debug_prediction"
+    prediction_file_part = prediction_file + ".part{}".format(role)
+
+    if os.path.exists(prediction_file_part):
+        os.remove(prediction_file_part)
+
+    step = 0
+    start_time = time.time()
+    for sample in test_loader():
+        step += 1
+        prediction = exe.run(program=infer_program, feed=sample, fetch_list=[softmax])
+        with open(prediction_file_part, 'ab') as f:
+            f.write(np.array(prediction).tostring())
+        if step % 10 == 0:
+            end_time = time.time()
+            logger.info('MPC infer of step={}, cost time in seconds:{}'.format(step, (end_time - start_time)))
+
+    end_time = time.time()
+    logger.info('MPC infer time in seconds:{}'.format((end_time - start_time)))
+
+# train
+exe = fluid.Executor(place)
+exe.run(fluid.default_startup_program())
+
+mpc_model_basedir = "./mpc_model/"
+
+step = 0
+start_time = time.time()
+logger.info('MPC training start...')
+for epoch_id in range(epoch_num):
+    for sample in loader():
+        step += 1
+        results = exe.run(feed=sample, fetch_list=[softmax])
+        if step % 100 == 0:
+            end_time = time.time()
+            logger.info('MPC training of epoch_id={} step={},  cost time in seconds:{}'
+                        .format(epoch_id, step, (end_time - start_time)))
+    
+    # For each epoch: infer or save infer program
+    #infer()
+    mpc_model_dir = mpc_model_basedir + "epoch{}/party{}".format(epoch_id, role)
+    fluid.io.save_inference_model(dirname=mpc_model_dir,
+        feeded_var_names=["x", "y"],
+        target_vars=[softmax],
+        executor=exe,
+        main_program=infer_program,
+        model_filename="__model__")
+
+end_time = time.time()
+logger.info('MPC training of epoch_num={} batch_size={}, cost time in seconds:{}'
+      .format(epoch_num, BATCH_SIZE, (end_time - start_time)))
+
+# infer
+infer()
+

python/paddle_fl/mpc/examples/mnist_demo/prepare.py

-# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-"""
-Prepare data for MNIST.
-"""
-import process_data
-
-
-process_data.generate_encrypted_data()
-process_data.generate_encrypted_test_data()

python/paddle_fl/mpc/examples/mnist_demo/run_standalone.sh → python/paddle_fl/mpc/examples/run_standalone.sh

@@ -62,24 +62,6 @@ fi
 # clear the redis cache
 $REDIS_BIN -h $SERVER -p $PORT flushall
 
-# remove temp data generated in last time
-PRED_FILE="/tmp/mnist_output_prediction.*"
-ls ${PRED_FILE}
-if [ $? -eq 0 ]; then
-        rm -rf $PRED_FILE
-fi
-
-TRAINING_FILE="/tmp/mnist2_feature.part*"
-ls ${TRAINING_FILE}
-if [ $? -ne 0 ]; then
-    echo "There is no data in /tmp, please prepare data with "python prepare.py" firstly"
-    exit 1
-else
-    echo "There are data for mnist:"
-    echo "`ls ${TRAINING_FILE}`"
-fi
-
-
 # kick off script with roles of 1 and 2, and redirect output to /dev/null
 for role in {1..2}; do
     $PYTHON $SCRIPT $role $SERVER $PORT 2>&1 >/dev/null &