diff --git a/python/paddle_fl/mpc/examples/mnist_demo/README.md b/python/paddle_fl/mpc/examples/mnist_demo/README.md index ecfa90edffb85cc2f3c922355d9559bf2d87a9cd..803ff782fbaef9a82221876f719ec0eb73ff60c5 100644 --- a/python/paddle_fl/mpc/examples/mnist_demo/README.md +++ b/python/paddle_fl/mpc/examples/mnist_demo/README.md @@ -20,6 +20,15 @@ Encrypted data files of feature and label would be generated and saved in `/tmp` #### (2). Launch Demo with A Shell Script +You should set the env params as follow: + +``` +export PYTHON=/yor/python +export PATH_TO_REDIS_BIN=/path/to/redis_bin +export LOCALHOST=/your/localhost +export REDIS_PORT=/your/redis/port +``` + Launch demo with the `run_standalone.sh` script. The concrete command is: ```bash @@ -32,11 +41,14 @@ Besides, predictions would be made in this demo once training is finished. The p #### (3). Decrypt Data -Decrypt the saved prediction data and save the decrypted prediction results into a specified file using `decrypt_data_to_file()` in `process_data.py` script. For example, users can write the following code into a python script named `decrypt_save.py`, and then run the script with command `python decrypt_save.py`. The decrypted prediction results would be saved into `mpc_label`. +Decrypt the saved prediction data and save the decrypted prediction results into a specified file using `decrypt_data_to_file()` in `process_data.py` script. For example, users can write the following code into a python script named `decrypt_save.py`, and then run the script with command `python decrypt_save.py decrypt_file`. The decrypted prediction results would be saved into `decrypt_file`. ```python +import sys + +decrypt_file=sys.argv[1] import process_data -process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), "mpc_label") +process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), decrypt_file) ``` **Note** that remember to delete the prediction files in `/tmp` directory generated in last running, in case of any influence on the decrypted results of current running. For simplifying users operations, we provide the following commands in `run_standalone.sh`, which can delete the files mentioned above when running this script. @@ -91,10 +103,13 @@ Similarly, predictions with cypher text format would be saved in `/tmp` director #### (5). Decrypt Prediction Data -Each computation party sends `mnist_output_prediction.part` file in `/tmp` directory to the `/tmp` directory of data owner. Data owner decrypts the prediction data and saves the decrypted prediction results into a specified file using `decrypt_data_to_file()` in `process_data.py` script. For example, users can write the following code into a python script named `decrypt_save.py`, and then run the script with command `python decrypt_save.py`. The decrypted prediction results would be saved into file `mpc_label`. +Each computation party sends `mnist_output_prediction.part` file in `/tmp` directory to the `/tmp` directory of data owner. Data owner decrypts the prediction data and saves the decrypted prediction results into a specified file using `decrypt_data_to_file()` in `process_data.py` script. For example, users can write the following code into a python script named `decrypt_save.py`, and then run the script with command `python decrypt_save.py decrypt_file`. The decrypted prediction results would be saved into file `decrypt_file`. ```python +import sys + +decrypt_file=sys.argv[1] import process_data -process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), "mpc_label") +process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), decrypt_file) ``` diff --git a/python/paddle_fl/mpc/examples/mnist_demo/README_CN.md b/python/paddle_fl/mpc/examples/mnist_demo/README_CN.md index a1a6339884823fea2b698ac4202944649552ad4e..f5b0a6c8db524bf9cacb9d562107114a425e051d 100644 --- a/python/paddle_fl/mpc/examples/mnist_demo/README_CN.md +++ b/python/paddle_fl/mpc/examples/mnist_demo/README_CN.md @@ -20,7 +20,16 @@ process_data.generate_encrypted_test_data() #### 2. 使用shell脚本启动demo -使用`run_standalone.sh`脚本,启动并运行demo,命令如下: +运行demo之前,需设置以下环境变量: + +``` +export PYTHON=/yor/python +export PATH_TO_REDIS_BIN=/path/to/redis_bin +export LOCALHOST=/your/localhost +export REDIS_PORT=/your/redis/port +``` + +然后使用`run_standalone.sh`脚本,启动并运行demo,命令如下: ```bash  bash run_standalone.sh mnist_demo.py @@ -32,11 +41,14 @@ bash run_standalone.sh mnist_demo.py #### 3. 解密数据 -使用`process_data.py`脚本中的`decrypt_data_to_file()`,将保存的密文预测结果进行解密,并且将解密得到的明文预测结果保存到指定文件中。例如,将下面的内容写到一个`decrypt_save.py`脚本中,然后`python decrypt_save.py`,将把明文预测结果保存在`mpc_label`文件中。 +使用`process_data.py`脚本中的`decrypt_data_to_file()`,将保存的密文预测结果进行解密,并且将解密得到的明文预测结果保存到指定文件中。例如,将下面的内容写到一个`decrypt_save.py`脚本中,然后`python decrypt_save.py decrypt_file`,将把明文预测结果保存在`decrypt_file`文件中。 ```python +import sys + +decrypt_file=sys.argv[1] import process_data -process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), "mpc_label") +process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), decrypt_file) ``` **注意**:再次启动运行demo之前,请先将上次在`/tmp`保存的预测密文结果文件删除,以免影响本次密文数据的恢复结果。为了简化用户操作,我们在`run_standalone.sh`脚本中加入了如下的内容,可以在执行脚本时删除上次的数据。 @@ -93,10 +105,13 @@ $PYTHON_EXECUTABLE mnist_demo.py $PARTY_ID $SERVER $PORT #### 5. 解密预测数据 -各计算party将`/tmp`目录下的`mnist_output_prediction.part`文件发送到数据方的/tmp目录下。数据方使用`process_data.py`脚本中的`decrypt_data_to_file()`,将密文预测结果进行解密,并且将解密得到的明文预测结果保存到指定文件中。例如,将下面的内容写到一个`decrypt_save.py`脚本中,然后`python decrypt_save.py`,将把明文预测结果保存在`mpc_label`文件中。 +各计算party将`/tmp`目录下的`mnist_output_prediction.part`文件发送到数据方的/tmp目录下。数据方使用`process_data.py`脚本中的`decrypt_data_to_file()`,将密文预测结果进行解密,并且将解密得到的明文预测结果保存到指定文件中。例如,将下面的内容写到一个`decrypt_save.py`脚本中,然后`python decrypt_save.py decrypt_file`,将把明文预测结果保存在`decrypt_file`文件中。 ```python +import sys + +decrypt_file=sys.argv[1] import process_data -process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), "mpc_label") +process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), decrypt_file) ``` diff --git a/python/paddle_fl/mpc/examples/mnist_demo/decrypt_save.py b/python/paddle_fl/mpc/examples/mnist_demo/decrypt_save.py new file mode 100644 index 0000000000000000000000000000000000000000..c4ae109b30df9800281d0e4062ce8348e709d731 --- /dev/null +++ b/python/paddle_fl/mpc/examples/mnist_demo/decrypt_save.py @@ -0,0 +1,23 @@ +# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +""" +Decrypt Prediction Data. +""" +import sys + +import process_data + +decrypt_file=sys.argv[1] +BATCH_SIZE=128 +process_data.decrypt_data_to_file("/tmp/mnist_output_prediction", (BATCH_SIZE,), decrypt_file) diff --git a/python/paddle_fl/mpc/examples/mnist_demo/mnist_demo.py b/python/paddle_fl/mpc/examples/mnist_demo/mnist_demo.py index 5a8b0a9045ebc57ee9da840a5b52481c3fb7bde9..d441dcb439b859fabaa2f938ba730322256fa890 100644 --- a/python/paddle_fl/mpc/examples/mnist_demo/mnist_demo.py +++ b/python/paddle_fl/mpc/examples/mnist_demo/mnist_demo.py @@ -16,6 +16,7 @@ MNIST Demo """ import sys +import os import numpy as np import time @@ -78,18 +79,20 @@ test_loader.set_batch_generator(test_batch_sample, places=place) exe = fluid.Executor(place) exe.run(fluid.default_startup_program()) -start_time = time.time() -step = 0 for epoch_id in range(epoch_num): + start_time = time.time() + step = 0 # feed data via loader for sample in loader(): + batch_start = time.time() exe.run(feed=sample, fetch_list=[cost.name]) + batch_end = time.time() if step % 50 == 0: - print('Epoch={}, Step={}'.format(epoch_id, step)) + print('Epoch={}, Step={}, batch_cost={:.4f} s'.format(epoch_id, step, (batch_end - batch_start))) step += 1 -end_time = time.time() -print('Mpc Training of Epoch={} Batch_size={}, cost time in seconds:{}' + end_time = time.time() + print('Mpc Training of Epoch={} Batch_size={}, epoch_cost={:.4f} s' .format(epoch_num, BATCH_SIZE, (end_time - start_time))) # prediction diff --git a/python/paddle_fl/mpc/examples/mnist_demo/prepare.py b/python/paddle_fl/mpc/examples/mnist_demo/prepare.py new file mode 100644 index 0000000000000000000000000000000000000000..b569ba41e98ac0554aa58517f42cfbe79a465bc2 --- /dev/null +++ b/python/paddle_fl/mpc/examples/mnist_demo/prepare.py @@ -0,0 +1,21 @@ +# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +""" +Prepare data for MNIST. +""" +import process_data + + +process_data.generate_encrypted_data() +process_data.generate_encrypted_test_data() diff --git a/python/paddle_fl/mpc/examples/mnist_demo/run_standalone.sh b/python/paddle_fl/mpc/examples/mnist_demo/run_standalone.sh index 33ce47df381942bb4ab841f84de917f57ea87d7e..df688403172f6dfd6e7111d55c0baf5b070a592e 100755 --- a/python/paddle_fl/mpc/examples/mnist_demo/run_standalone.sh +++ b/python/paddle_fl/mpc/examples/mnist_demo/run_standalone.sh @@ -31,12 +31,13 @@ # bash run_standalone.sh TEST_SCRIPT_NAME # -# modify the following vars according to your environment -PYTHON="python" -REDIS_HOME="path_to_redis_bin" -SERVER="localhost" -PORT=9937 +# please set the following environment vars according in your environment +PYTHON=${PYTHON} +REDIS_HOME=${PATH_TO_REDIS_BIN} +SERVER=${LOCALHOST} +PORT=${REDIS_PORT} +echo "redis home in ${REDIS_HOME}, server is ${SERVER}, port is ${PORT}" function usage() { echo 'run_standalone.sh SCRIPT_NAME [ARG...]' exit 0 @@ -63,10 +64,21 @@ $REDIS_BIN -h $SERVER -p $PORT flushall # remove temp data generated in last time PRED_FILE="/tmp/mnist_output_prediction.*" -if [ "$PRED_FILE" ]; then +ls ${PRED_FILE} +if [ $? -eq 0 ]; then rm -rf $PRED_FILE fi +TRAINING_FILE="/tmp/mnist2_feature.part*" +ls ${TRAINING_FILE} +if [ $? -ne 0 ]; then + echo "There is no data in /tmp, please prepare data with "python prepare.py" firstly" + exit 1 +else + echo "There are data for mnist:" + echo "`ls ${TRAINING_FILE}`" +fi + # kick off script with roles of 1 and 2, and redirect output to /dev/null for role in {1..2}; do