Skip to content

E
ERNIE

邱崇源 / ERNIE
与 Fork 源项目一致

Fork自 PaddlePaddle / ERNIE

通知 1
Star 0
Fork 0
E
ERNIE

前往新版Gitcode，体验更适合开发者的 AI 搜索 >>

未验证 提交 4f1537e2 编写于 作者: M Meiyim 提交者： GitHub
浏览文件
操作

Merge branch 'develop' into develop

上级 b6697bf0 050327e9
展开全部 隐藏空白更改
内联 并排
Showing with 6553 addition and 631 deletion
README.md
浏览文件 @ 4f1537e2
......@@ -19,7 +19,6 @@ English | [简体中文](./README.zh.md)
* [Results](#results)
* [Results on English Datasets](#results-on-english-datasets)
* [Results on Chinese Datasets](#results-on-chinese-datasets)
* [Release Notes](#release-notes)
* [Communication](#communication)
* [Usage](#usage)
......@@ -615,14 +614,6 @@ LCQMC is a Chinese question semantic matching corpus published in COLING2018. [u
BQ Corpus (Bank Question corpus) is a Chinese corpus for sentence semantic equivalence identification. This dataset was published in EMNLP 2018. [url: https://www.aclweb.org/anthology/D18-1536]
```
## Release Notes
- Aug 21, 2019: featuers update: fp16 finetuning, multiprocess finetining.
- July 30, 2019: release ERNIE 2.0
- Apr 10, 2019: update ERNIE_stable-1.0.1.tar.gz, update config and vocab
- Mar 18, 2019: update ERNIE_stable.tgz
- Mar 15, 2019: release ERNIE 1.0
## Communication
......@@ -657,11 +648,12 @@ BQ Corpus (Bank Question corpus) is a Chinese corpus for sentence semantic equiv
* [FAQ3: Is the argument batch_size for one GPU card or for all GPU cards?](#faq3-is-the--argument-batch_size-for-one-gpu-card-or-for-all-gpu-cards)
* [FAQ4: Can not find library: libcudnn.so. Please try to add the lib path to LD_LIBRARY_PATH.](#faq4-can-not-find-library-libcudnnso-please-try-to-add-the-lib-path-to-ld_library_path)
* [FAQ5: Can not find library: libnccl.so. Please try to add the lib path to LD_LIBRARY_PATH.](#faq5-can-not-find-library-libncclso-please-try-to-add-the-lib-path-to-ld_library_path)
* [FQA6: Runtime error: `ModuleNotFoundError No module named propeller`](#faq6)
### Install PaddlePaddle
This code base has been tested with Paddle Fluid 1.5.1 under Python2.
This code base has been tested with Paddle Fluid 1.6 with Python 2/3.5+, since Paddle 1.6 has changed some of APIs, using version before 1.6 might have bug on NER tasks.
**\*Important\*** When finished installing Paddle Fluid, remember to update LD_LIBRARY_PATH about CUDA, cuDNN, NCCL2, for more information on paddlepaddle setup, you can click [here](http://en.paddlepaddle.org/documentation/docs/en/1.5/beginners_guide/index_en.html) and [here](http://en.paddlepaddle.org/documentation/docs/en/1.5/beginners_guide/install/install_Ubuntu_en.html). Also, you can read FAQ at the end of this document when you encounter errors.
......@@ -1010,6 +1002,12 @@ Export the path of cuda to LD_LIBRARY_PATH, e.g.: `export LD_LIBRARY_PATH=/home/
Download [NCCL2](https://developer.nvidia.com/nccl/nccl-download), and export the library path to LD_LIBRARY_PATH, e.g.:`export LD_LIBRARY_PATH=/home/work/nccl/lib`
#### FAQ6: Cannot malloc XXX MB GPU memory.
### FAQ6: Runtime error: `ModuleNotFoundError No module named propeller`<a name="faq6"></a>
you can import propeller to your PYTHONPATH by `export PYTHONPATH:./:$PYTHONPATH`
#### FAQ7: Cannot malloc XXX MB GPU memory.
Try to reduce the batch_size, reduce the max_seq_len and set FLAGS_eager_delete_tensor_gb=0.0
README.zh.md
浏览文件 @ 4f1537e2
......@@ -19,7 +19,7 @@
* [效果验证](#效果验证)
* [中文效果验证](#中文效果验证)
* [英文效果验证](#英文效果验证)
* [开源记录](#开源记录)
* [ERNIE tiny](#ernie-tiny)
* [技术交流](#技术交流)
* [使用](#使用)
......@@ -589,7 +589,6 @@ ERNIE 2.0 的英文效果验证在 GLUE 上进行。GLUE 评测的官方地址
#### GLUE - 验证集结果
| <strong>数据集</strong> | <strong>CoLA</strong> | <strong>SST-2</strong> | <strong>MRPC</strong> | <strong>STS-B</strong> | <strong>QQP</strong> | <strong>MNLI-m</strong> | <strong>QNLI</strong> | <strong>RTE</strong> |
......@@ -617,11 +616,34 @@ ERNIE 2.0 的英文效果验证在 GLUE 上进行。GLUE 评测的官方地址
由于 XLNet 暂未公布 GLUE 测试集上的单模型结果，所以我们只与 BERT 进行单模型比较。上表为ERNIE 2.0 单模型在 GLUE 测试集的表现结果。
## 开源记录
- 2019-07-30 发布 ERNIE 2.0
- 2019-04-10 更新: update ERNIE_stable-1.0.1.tar.gz, 将模型参数、配置 ernie_config.json、vocab.txt 打包发布
- 2019-03-18 更新: update ERNIE_stable.tgz
- 2019-03-15 发布 ERNIE 1.0
### ERNIE tiny
为了提升ERNIE模型在实际工业应用中的落地能力，我们推出ERNIE-tiny模型。
![ernie_tiny](.metas/ernie_tiny.png)
ERNIE-tiny作为小型化ERNIE，采用了以下4点技术，保证了在实际真实数据中将近4.3倍的预测提速。
1. 浅：12层的ERNIE Base模型直接压缩为3层，线性提速4倍，但效果也会有较大幅度的下降；
1. 胖：模型变浅带来的损失可通过hidden size的增大来弥补。由于fluid inference框架对于通用矩阵运算（gemm）的最后一维（hidden size）参数的不同取值会有深度的优化，因为将hidden size从768提升至1024并不会带来速度线性的增加；
1. 短：ERNIE Tiny是首个开源的中文subword粒度的预训练模型。这里的短是指通过subword粒度替换字（char）粒度，能够明显地缩短输入文本的长度，而输入文本长度是和预测速度有线性相关。统计表明，在XNLI dev集上采用subword字典切分出来的序列长度比字表平均缩短40%；
1. 萃：为了进一步提升模型的效果，ERNIE Tiny扮演学生角色，利用模型蒸馏的方式在Transformer层和Prediction层去学习教师模型ERNIE模型对应层的分布或输出，这种方式能够缩近ERNIE Tiny和ERNIE的效果差异。
#### Benchmark
ERNIE Tiny轻量级模型在公开数据集的效果如下所示，任务均值相对于ERNIE Base只下降了2.37%，但相对于“SOTA Before BERT”提升了8%。在延迟测试中，ERNIE Tiny能够带来4.3倍的速度提升
（测试环境为：GPU P4，Paddle Inference C++ API，XNLI Dev集，最大maxlen=128，测试结果10次均值）
|model|XNLI(acc)|LCQCM(acc)|CHNSENTICORP(acc)|NLPCC-DBQA(mrr/f1)|Average|Latency
|--|--|--|--|--|--|--|
|SOTA-before-ERNIE|68.3|83.4|92.2|72.01/-|78.98|-|
|ERNIE2.0-base|79.7|87.9|95.5|95.7/85.3|89.70|633ms(1x)|
|ERNIE-tiny-subword|75.1|86.1|95.2|92.9/78.6|87.33|146ms(4.3x)|
## 技术交流
......@@ -646,6 +668,7 @@ ERNIE 2.0 的英文效果验证在 GLUE 上进行。GLUE 评测的官方地址
* [序列标注任务](#序列标注任务)
* [实体识别](#实体识别)
* [阅读理解任务](#阅读理解任务-1)
* [ERNIE tiny](#tune-ernie-tiny)
* [利用Propeller进行二次开发](#利用propeller进行二次开发)
* [预训练 (ERNIE 1.0)](#预训练-ernie-10)
* [数据预处理](#数据预处理)
......@@ -666,7 +689,7 @@ ERNIE 2.0 的英文效果验证在 GLUE 上进行。GLUE 评测的官方地址
## PaddlePaddle安装
本项目依赖于 Paddle Fluid 1.5，* 暂时不支持Paddle 1.6的使用 *，请参考[安装指南](http://www.paddlepaddle.org/#quick-start)进行安装。
本项目依赖于 Paddle 1.6，* 由于Paddle 1.6版本相比之前版本有较大API改动，使用Paddle 1.6以前版本运行本代码库会导致序列标注等任务报错 *，请参考[安装指南](http://www.paddlepaddle.org/#quick-start)进行安装。
**【重要】安装后，需要及时的将 CUDA、cuDNN、NCCL2 等动态库路径加入到环境变量 LD_LIBRARY_PATH 之中，否则训练过程中会报相关的库错误。具体的paddlepaddle配置细节请查阅[这里](http://en.paddlepaddle.org/documentation/docs/zh/1.5/beginners_guide/quick_start_cn.html)**
......@@ -695,6 +718,7 @@ pip install -r requirements.txt
| [ERNIE 1.0 中文 Base 模型(max_len=512)](https://ernie.bj.bcebos.com/ERNIE_1.0_max-len-512.tar.gz) | 包含预训练模型参数、词典 vocab.txt、模型配置 ernie_config.json|
| [ERNIE 2.0 英文 Base 模型](https://ernie.bj.bcebos.com/ERNIE_Base_en_stable-2.0.0.tar.gz) | 包含预训练模型参数、词典 vocab.txt、模型配置 ernie_config.json|
| [ERNIE 2.0 英文 Large 模型](https://ernie.bj.bcebos.com/ERNIE_Large_en_stable-2.0.0.tar.gz) | 包含预训练模型参数、词典 vocab.txt、模型配置 ernie_config.json|
| [ERNIE tiny 中文模型](https://ernie.bj.bcebos.com/ernie_tiny.tar.gz)|包含预训练模型参数、词典 vocab.txt、模型配置 ernie_config.json 以及切词词表|
......@@ -894,6 +918,16 @@ text_a label
[test evaluation] em: 88.061838, f1: 93.520152, avg: 90.790995, question_num: 3493
```
### ERNIE tiny <a name="tune-ernie-tiny"></a>
ERNIE tiny 模型采用了subword粒度输入，需要在数据前处理中加入切词(segmentation)并使用[sentence piece](https://github.com/google/sentencepiece)进行tokenization.
segmentation 以及 tokenization 需要使用的模型包含在了 ERNIE tiny 的[预训练模型文件](#预训练模型下载)中，分别是 `./subword/dict.wordseg.pickle` 和 `./subword/spm_cased_simp_sampled.model`.
目前`./example/`下的代码针对 ERNIE tiny 的前处理进行了适配只需在脚本中通过 `--sentence_piece_model` 引入tokenization 模型，再通过 `--word_dict` 引入 segmentation 模型之后即可进行 ERNIE tiny 的 Fine-tune。
对于命名实体识别类型的任务，为了跟输入标注对齐，ERNIE tiny 仍然采用中文单字粒度进行作为输入。因此使用 `./example/finetune_ner.py` 时只需要打开 `--use_sentence_piece_vocab` 即可。
具体的使用方法可以参考[下节](#利用propeller进行二次开发).
## 利用Propeller进行二次开发
[Propeller](./propeller/README.md) 是基于PaddlePaddle构建的一键式训练API，对于具备一定机器学习应用经验的开发者可以使用Propeller获得定制化开发体验。
......@@ -947,7 +981,7 @@ cat input_file | python3 ./example/finetune_classifier.py --do_predict ... > out
### 数据预处理
基于百科类、资讯类、论坛对话类数据构造具有上下文关系的句子对数据，利用百度内部词法分析工具对句对数据进行字、词、实体等不同粒度的切分，然后基于 [`tokenization.py`](tokenization.py) 中的 CharTokenizer 对切分后的数据进行 token 化处理，得到明文的 token 序列及切分边界，然后将明文数据根据词典 [`config/vocab.txt`](config/vocab.txt) 映射为 id 数据，在训练过程中，根据切分边界对连续的 token 进行随机 mask 操作；
基于百科类、资讯类、论坛对话类数据构造具有上下文关系的句子对数据，利用百度内部词法分析工具对句对数据进行字、词、实体等不同粒度的切分，然后基于 [`tokenization.py`](./ernie/tokenization.py) 中的 CharTokenizer 对切分后的数据进行 token 化处理，得到明文的 token 序列及切分边界，然后将明文数据根据词典 [`config/vocab.txt`](config/vocab.txt) 映射为 id 数据，在训练过程中，根据切分边界对连续的 token 进行随机 mask 操作；
我们给出了 id 化后的部分训练数据：[`data/demo_train_set.gz`](./data/demo_train_set.gz)、和测试数据：[`data/demo_valid_set.gz`](./data/demo_valid_set.gz)，每行数据为1个训练样本，示例如下:
......@@ -1022,7 +1056,7 @@ ERNIE提供了通过数据蒸馏从而达到模型压缩、加速的开发套件
如果您采用 `propeller` 完成finetune，则 `BestInferenceExporter` 会在finetune过程中根据预测指标，挑最好的模型生成 inference_model .
### 在线预测
随后您可以使用[PaddleInference C++ API](https://www.paddlepaddle.org.cn/documentation/docs/zh/advanced_usage/deploy/inference/native_infer.html)将模型的前向预测代码联编到您的生产环境中。或者您可以使用我们为您构建好的python预测引擎来完成一个简单的服务。执行如下指令，便可以开启一个propeller server：
随后您可以使用[ERNIE fast inference C++ API](./inference/README.md)将模型的前向预测代码联编到您的生产环境中。或者您可以使用我们为您构建好的python预测引擎来完成一个简单的服务。执行如下指令，便可以开启一个propeller server：
```script
python -m propeller.tools.start_server -m /path/to/saved/model -p 8888
......@@ -1099,7 +1133,7 @@ python -u infer_classifyer.py \
需要先下载 [NCCL](https://developer.nvidia.com/nccl/nccl-download)，然后在 LD_LIBRARY_PATH 中添加 NCCL 库的路径，如`export LD_LIBRARY_PATH=/home/work/nccl/lib`
### FQA6: 运行报错`ModuleNotFoundError: No module named 'propeller'`<a name="faq6"></a>
### FAQ6: 运行报错`ModuleNotFoundError: No module named 'propeller'`<a name="faq6"></a>
您可以通过`export PYTHONPATH=./:$PYTHONPATH`的方式引入Propeller.
......
ernie/batching.py
浏览文件 @ 4f1537e2
......@@ -208,7 +208,7 @@ def pad_batch_data(insts,
if return_seq_lens:
seq_lens = np.array([len(inst) for inst in insts])
return_list += [seq_lens.astype("int64").reshape([-1, 1])]
return_list += [seq_lens.astype("int64").reshape([-1])]
return return_list if len(return_list) > 1 else return_list[0]
......
ernie/ernie_encoder.py
浏览文件 @ 4f1537e2
......@@ -22,13 +22,16 @@ import argparse
import numpy as np
import multiprocessing
import logging
import paddle.fluid as fluid
import reader.task_reader as task_reader
from model.ernie import ErnieConfig, ErnieModel
from utils.args import ArgumentGroup, print_arguments
from model.ernie_v1 import ErnieConfig, ErnieModel
from utils.args import ArgumentGroup, print_arguments, prepare_logger
from utils.init import init_pretraining_params
log = logging.getLogger()
# yapf: disable
parser = argparse.ArgumentParser(__doc__)
model_g = ArgumentGroup(parser, "model", "model configuration and paths.")
......@@ -52,24 +55,21 @@ run_type_g.add_arg("use_cuda", bool, True, "If set, use G
def create_model(args, pyreader_name, ernie_config):
pyreader = fluid.layers.py_reader(
capacity=50,
shapes=[[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
[-1, args.max_seq_len, 1], [-1, 1]],
dtypes=['int64', 'int64', 'int64', 'int64', 'float', 'int64'],
lod_levels=[0, 0, 0, 0, 0, 0],
name=pyreader_name,
use_double_buffer=True)
(src_ids, sent_ids, pos_ids, task_ids, input_mask,
seq_lens) = fluid.layers.read_file(pyreader)
src_ids = fluid.layers.data(name='1', shape=[-1, args.max_seq_len, 1], dtype='int64')
sent_ids = fluid.layers.data(name='2', shape=[-1, args.max_seq_len, 1], dtype='int64')
pos_ids = fluid.layers.data(name='3', shape=[-1, args.max_seq_len, 1], dtype='int64')
task_ids = fluid.layers.data(name='4', shape=[-1, args.max_seq_len, 1], dtype='int64')
input_mask = fluid.layers.data(name='5', shape=[-1, args.max_seq_len, 1], dtype='float32')
seq_lens = fluid.layers.data(name='8', shape=[-1], dtype='int64')
pyreader = fluid.io.DataLoader.from_generator(feed_list=[src_ids, sent_ids, pos_ids, task_ids, input_mask, seq_lens],
capacity=70,
iterable=False)
ernie = ErnieModel(
src_ids=src_ids,
position_ids=pos_ids,
sentence_ids=sent_ids,
task_ids=task_ids,
input_mask=input_mask,
config=ernie_config)
......@@ -143,7 +143,7 @@ def main(args):
exec_strategy = fluid.ExecutionStrategy()
exec_strategy.num_threads = dev_count
pyreader.decorate_tensor_provider(data_generator)
pyreader.set_batch_generator(data_generator)
pyreader.start()
total_cls_emb = []
......@@ -167,6 +167,11 @@ def main(args):
total_cls_emb = np.concatenate(total_cls_emb)
total_top_layer_emb = np.concatenate(total_top_layer_emb)
if not os.path.exists(args.output_dir):
os.mkdir(args.output_dir)
else:
raise RuntimeError('output dir exists: %s' % args.output_dir)
with open(os.path.join(args.output_dir, "cls_emb.npy"),
"wb") as cls_emb_file:
np.save(cls_emb_file, total_cls_emb)
......@@ -176,6 +181,7 @@ def main(args):
if __name__ == '__main__':
prepare_logger(log)
args = parser.parse_args()
print_arguments(args)
......
ernie/finetune/classifier.py
浏览文件 @ 4f1537e2
......@@ -39,34 +39,22 @@ def create_model(args,
is_classify=False,
is_regression=False,
ernie_version="1.0"):
src_ids = fluid.layers.data(name='eval_placeholder_0', shape=[-1, args.max_seq_len, 1], dtype='int64')
sent_ids = fluid.layers.data(name='eval_placeholder_1', shape=[-1, args.max_seq_len, 1], dtype='int64')
pos_ids = fluid.layers.data(name='eval_placeholder_2', shape=[-1, args.max_seq_len, 1], dtype='int64')
input_mask = fluid.layers.data(name='eval_placeholder_3', shape=[-1, args.max_seq_len, 1], dtype='float32')
task_ids = fluid.layers.data(name='eval_placeholder_4', shape=[-1, args.max_seq_len, 1], dtype='int64')
qids = fluid.layers.data(name='eval_placeholder_5', shape=[-1, 1], dtype='int64')
if is_classify:
pyreader = fluid.layers.py_reader(
capacity=50,
shapes=[[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
[-1, args.max_seq_len, 1], [-1, 1], [-1, 1]],
dtypes=[
'int64', 'int64', 'int64', 'int64', 'float32', 'int64', 'int64'
],
lod_levels=[0, 0, 0, 0, 0, 0, 0],
name=task_name + "_" + pyreader_name,
use_double_buffer=True)
labels = fluid.layers.data(name='6', shape=[-1, 1], dtype='int64')
elif is_regression:
pyreader = fluid.layers.py_reader(
capacity=50,
shapes=[[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
[-1, args.max_seq_len, 1], [-1, 1], [-1, 1]],
dtypes=[
'int64', 'int64', 'int64', 'int64', 'float32', 'float32',
'int64'
],
lod_levels=[0, 0, 0, 0, 0, 0, 0],
name=task_name + "_" + pyreader_name,
use_double_buffer=True)
(src_ids, sent_ids, pos_ids, task_ids, input_mask, labels,
qids) = fluid.layers.read_file(pyreader)
labels = fluid.layers.data(name='6', shape=[-1, 1], dtype='float32')
pyreader = fluid.io.DataLoader.from_generator(feed_list=[src_ids, sent_ids, pos_ids, task_ids, input_mask, labels, qids],
capacity=70,
iterable=False)
ernie = ErnieModel(
src_ids=src_ids,
......@@ -99,7 +87,7 @@ def create_model(args,
else:
probs = logits
feed_targets_name = [
src_ids.name, sent_ids.name, pos_ids.name, input_mask.name
src_ids.name, sent_ids.name, pos_ids.name, input_mask.name
]
if ernie_version == "2.0":
feed_targets_name += [task_ids.name]
......
ernie/finetune/mrc.py
浏览文件 @ 4f1537e2
......@@ -40,20 +40,18 @@ import tokenization
log = logging.getLogger(__name__)
def create_model(args, pyreader_name, ernie_config, is_training):
pyreader = fluid.layers.py_reader(
capacity=50,
shapes=[[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
[-1, args.max_seq_len, 1], [-1, 1], [-1, 1], [-1, 1]],
dtypes=[
'int64', 'int64', 'int64', 'int64', 'float32', 'int64', 'int64',
'int64'
],
lod_levels=[0, 0, 0, 0, 0, 0, 0, 0],
name=pyreader_name,
use_double_buffer=True)
(src_ids, sent_ids, pos_ids, task_ids, input_mask, start_positions,
end_positions, unique_id) = fluid.layers.read_file(pyreader)
src_ids = fluid.layers.data(name='1', shape=[-1, args.max_seq_len, 1], dtype='int64')
pos_ids = fluid.layers.data(name='2', shape=[-1, args.max_seq_len, 1], dtype='int64')
sent_ids= fluid.layers.data(name='3', shape=[-1, args.max_seq_len, 1], dtype='int64')
task_ids= fluid.layers.data(name='4', shape=[-1, args.max_seq_len, 1], dtype='int64')
input_mask = fluid.layers.data(name='5', shape=[-1, args.max_seq_len, 1], dtype='float32')
start_positions = fluid.layers.data(name='6', shape=[-1, 1], dtype='int64')
end_positions = fluid.layers.data(name='7', shape=[-1, 1], dtype='int64')
unique_id = fluid.layers.data(name='8', shape=[-1, 1], dtype='int64')
pyreader = fluid.io.DataLoader.from_generator(feed_list=[
src_ids, sent_ids, pos_ids, task_ids, input_mask, start_positions,
end_positions, unique_id], capacity=50, iterable=False)
ernie = ErnieModel(
src_ids=src_ids,
......
ernie/finetune/sequence_label.py
浏览文件 @ 4f1537e2
......@@ -36,20 +36,17 @@ from model.ernie import ErnieModel
log = logging.getLogger(__name__)
def create_model(args, pyreader_name, ernie_config, is_prediction=False):
pyreader = fluid.layers.py_reader(
capacity=50,
shapes=[[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1], [-1, 1]],
dtypes=[
'int64', 'int64', 'int64', 'int64', 'float32', 'int64', 'int64'
],
lod_levels=[0, 0, 0, 0, 0, 0, 0],
name=pyreader_name,
use_double_buffer=True)
(src_ids, sent_ids, pos_ids, task_ids, input_mask, labels,
seq_lens) = fluid.layers.read_file(pyreader)
src_ids = fluid.layers.data(name='1', shape=[-1, args.max_seq_len, 1], dtype='int64')
sent_ids = fluid.layers.data(name='2', shape=[-1, args.max_seq_len, 1], dtype='int64')
pos_ids = fluid.layers.data(name='3', shape=[-1, args.max_seq_len, 1], dtype='int64')
task_ids = fluid.layers.data(name='4', shape=[-1, args.max_seq_len, 1], dtype='int64')
input_mask = fluid.layers.data(name='5', shape=[-1, args.max_seq_len, 1], dtype='float32')
labels = fluid.layers.data(name='7', shape=[-1, args.max_seq_len, 1], dtype='int64')
seq_lens = fluid.layers.data(name='8', shape=[-1], dtype='int64')
pyreader = fluid.io.DataLoader.from_generator(feed_list=[src_ids, sent_ids, pos_ids, task_ids, input_mask, labels, seq_lens],
capacity=70,
iterable=False)
ernie = ErnieModel(
src_ids=src_ids,
......
ernie/infer_classifyer.py
浏览文件 @ 4f1537e2
......@@ -130,6 +130,7 @@ def main(args):
if not args.use_cuda:
log.info("disable gpu")
config.disable_gpu()
config.switch_ir_optim(True)
else:
log.info("using gpu")
config.enable_use_gpu(1024)
......@@ -162,8 +163,7 @@ def main(args):
# parse outputs
output = outputs[0]
output_data = output.data.float_data()
batch_result = np.array(output_data).reshape(output.shape)
batch_result = output.as_ndarray()
for single_example_probs in batch_result:
print('\t'.join(map(str, single_example_probs.tolist())))
index += 1
......@@ -173,17 +173,7 @@ def main(args):
def array2tensor(ndarray):
""" convert numpy array to PaddleTensor"""
assert isinstance(ndarray, np.ndarray), "input type must be np.ndarray"
tensor = PaddleTensor()
tensor.name = "data"
tensor.shape = ndarray.shape
if "float" in str(ndarray.dtype):
tensor.dtype = PaddleDType.FLOAT32
elif "int" in str(ndarray.dtype):
tensor.dtype = PaddleDType.INT64
else:
raise ValueError("{} type ndarray is unsupported".format(tensor.dtype))
tensor.data = PaddleBuf(ndarray.flatten().tolist())
tensor = PaddleTensor(data=ndarray)
return tensor
if __name__ == '__main__':
......
ernie/model/ernie.py
浏览文件 @ 4f1537e2
......@@ -24,6 +24,7 @@ import six
import logging
import paddle.fluid as fluid
from io import open
from paddle.fluid.layers import core
from model.transformer_encoder import encoder, pre_process_layer
......@@ -85,8 +86,8 @@ class ErnieModel(object):
self._pos_emb_name = "pos_embedding"
self._sent_emb_name = "sent_embedding"
self._task_emb_name = "task_embedding"
self._dtype = "float16" if use_fp16 else "float32"
self._emb_dtype = "float32"
self._dtype = core.VarDesc.VarType.FP16 if use_fp16 else core.VarDesc.VarType.FP32
self._emb_dtype = core.VarDesc.VarType.FP32
# Initialize all weigths by truncated normal initializer, and all biases
# will be initialized by constant zero by default.
......@@ -138,7 +139,7 @@ class ErnieModel(object):
emb_out = pre_process_layer(
emb_out, 'nd', self._prepostprocess_dropout, name='pre_encoder')
if self._dtype == "float16":
if self._dtype == core.VarDesc.VarType.FP16:
emb_out = fluid.layers.cast(x=emb_out, dtype=self._dtype)
input_mask = fluid.layers.cast(x=input_mask, dtype=self._dtype)
self_attn_mask = fluid.layers.matmul(
......@@ -167,7 +168,7 @@ class ErnieModel(object):
postprocess_cmd="dan",
param_initializer=self._param_initializer,
name='encoder')
if self._dtype == "float16":
if self._dtype == core.VarDesc.VarType.FP16:
self._enc_out = fluid.layers.cast(
x=self._enc_out, dtype=self._emb_dtype)
......
ernie/model/ernie_v1.py
浏览文件 @ 4f1537e2
......@@ -24,6 +24,7 @@ import logging
import six
import paddle.fluid as fluid
from io import open
from paddle.fluid.layers import core
from model.transformer_encoder import encoder, pre_process_layer
......@@ -76,7 +77,7 @@ class ErnieModel(object):
self._word_emb_name = "word_embedding"
self._pos_emb_name = "pos_embedding"
self._sent_emb_name = "sent_embedding"
self._dtype = "float16" if use_fp16 else "float32"
self._dtype = core.VarDesc.VarType.FP16 if use_fp16 else core.VarDesc.VarType.FP32
# Initialize all weigths by truncated normal initializer, and all biases
# will be initialized by constant zero by default.
......@@ -114,7 +115,7 @@ class ErnieModel(object):
emb_out = pre_process_layer(
emb_out, 'nd', self._prepostprocess_dropout, name='pre_encoder')
if self._dtype == "float16":
if self._dtype == core.VarDesc.VarType.FP16:
input_mask = fluid.layers.cast(x=input_mask, dtype=self._dtype)
self_attn_mask = fluid.layers.matmul(
x=input_mask, y=input_mask, transpose_y=True)
......
ernie/run_classifier.py
浏览文件 @ 4f1537e2
......@@ -228,7 +228,7 @@ def main(args):
num_trainers=nccl2_num_trainers,
trainer_id=nccl2_trainer_id)
train_pyreader.decorate_tensor_provider(train_data_generator)
train_pyreader.set_batch_generator(train_data_generator)
else:
train_exe = None
......@@ -349,7 +349,7 @@ def main(args):
# final eval on dianostic, hack for glue-ax
if args.diagnostic:
test_pyreader.decorate_tensor_provider(
test_pyreader.set_batch_generator(
reader.data_generator(
args.diagnostic,
batch_size=args.batch_size,
......@@ -380,7 +380,7 @@ def evaluate_wrapper(args, reader, exe, test_prog, test_pyreader, graph_vars,
# evaluate dev set
batch_size = args.batch_size if args.predict_batch_size is None else args.predict_batch_size
for ds in args.dev_set.split(','):
test_pyreader.decorate_tensor_provider(
test_pyreader.set_batch_generator(
reader.data_generator(
ds,
batch_size=batch_size,
......@@ -409,7 +409,7 @@ def predict_wrapper(args, reader, exe, test_prog, test_pyreader, graph_vars,
batch_size = args.batch_size if args.predict_batch_size is None else args.predict_batch_size
for test_f, save_f in zip(test_sets, save_dirs):
test_pyreader.decorate_tensor_provider(
test_pyreader.set_batch_generator(
reader.data_generator(
test_f,
batch_size=batch_size,
......
ernie/run_mrc.py
浏览文件 @ 4f1537e2
......@@ -228,7 +228,7 @@ def main(args):
num_trainers=nccl2_num_trainers,
trainer_id=nccl2_trainer_id)
train_pyreader.decorate_tensor_provider(train_data_generator)
train_pyreader.set_batch_generator(train_data_generator)
else:
train_exe = None
......@@ -272,7 +272,7 @@ def main(args):
if steps % args.validation_steps == 0:
if args.do_val:
test_pyreader.decorate_tensor_provider(
test_pyreader.set_batch_generator(
reader.data_generator(
args.dev_set,
batch_size=args.batch_size,
......@@ -291,7 +291,7 @@ def main(args):
args=args)
if args.do_test:
test_pyreader.decorate_tensor_provider(
test_pyreader.set_batch_generator(
reader.data_generator(
args.test_set,
batch_size=args.batch_size,
......@@ -318,7 +318,7 @@ def main(args):
# final eval on dev set
if args.do_val:
log.info("Final validation result:")
test_pyreader.decorate_tensor_provider(
test_pyreader.set_batch_generator(
reader.data_generator(
args.dev_set,
batch_size=args.batch_size,
......@@ -339,7 +339,7 @@ def main(args):
# final eval on test set
if args.do_test:
log.info("Final test result:")
test_pyreader.decorate_tensor_provider(
test_pyreader.set_batch_generator(
reader.data_generator(
args.test_set,
batch_size=args.batch_size,
......@@ -361,7 +361,6 @@ def main(args):
if __name__ == '__main__':
prepare_logger(log)
print_arguments(args)
while True:
scope = fluid.core.Scope()
with fluid.scope_guard(scope):
main(args)
scope = fluid.core.Scope()
with fluid.scope_guard(scope):
main(args)
ernie/run_sequence_labeling.py
浏览文件 @ 4f1537e2
......@@ -217,7 +217,7 @@ def main(args):
num_trainers=nccl2_num_trainers,
trainer_id=nccl2_trainer_id)
train_pyreader.decorate_tensor_provider(train_data_generator)
train_pyreader.set_batch_generator(train_data_generator)
else:
train_exe = None
......@@ -302,7 +302,7 @@ def evaluate_wrapper(reader, exe, test_prog, test_pyreader, graph_vars,
# evaluate dev set
batch_size = args.batch_size if args.predict_batch_size is None else args.predict_batch_size
for ds in args.dev_set.split(','): #single card eval
test_pyreader.decorate_tensor_provider(
test_pyreader.set_batch_generator(
reader.data_generator(
ds,
batch_size=batch_size,
......@@ -324,7 +324,7 @@ def predict_wrapper(reader, exe, test_prog, test_pyreader, graph_vars,
batch_size = args.batch_size if args.predict_batch_size is None else args.predict_batch_size
for test_f, save_f in zip(test_sets, save_dirs):
test_pyreader.decorate_tensor_provider(reader.data_generator(
test_pyreader.set_batch_generator(reader.data_generator(
test_f,
batch_size=batch_size,
epoch=1,
......
ernie/service/encoder_server.py
浏览文件 @ 4f1537e2
......@@ -52,6 +52,10 @@ if __name__ == "__main__":
cuda_env = os.getenv("CUDA_VISIBLE_DEVICES")
if cuda_env is None:
raise RuntimeError('CUDA_VISIBLE_DEVICES not set')
if not os.path.exists(args.model_dir):
raise ValueError('model_dir not found: %s' % args.model_dir)
if not os.path.exists(args.model_dir):
raise ValueError('model_dir not found: %s' % args.model_dir)
n_devices = len(cuda_env.split(","))
if args.encode_layer.lower() == 'pooler':
model_dir = os.path.join(args.model_dir, 'pooler')
......
ernie/train.py
浏览文件 @ 4f1537e2
......@@ -41,20 +41,17 @@ args = parser.parse_args()
def create_model(pyreader_name, ernie_config):
pyreader = fluid.layers.py_reader(
capacity=70,
shapes=[[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1], [-1, 1],
[-1, 1], [-1, 1]],
dtypes=[
'int64', 'int64', 'int64', 'float32', 'int64', 'int64', 'int64'
],
lod_levels=[0, 0, 0, 0, 0, 0, 0],
name=pyreader_name,
use_double_buffer=True)
(src_ids, pos_ids, sent_ids, input_mask, mask_label, mask_pos,
labels) = fluid.layers.read_file(pyreader)
src_ids = fluid.layers.data(name='1', shape=[-1, args.max_seq_len, 1], dtype='int64')
pos_ids = fluid.layers.data(name='2', shape=[-1, args.max_seq_len, 1], dtype='int64')
sent_ids= fluid.layers.data(name='3', shape=[-1, args.max_seq_len, 1], dtype='int64')
input_mask = fluid.layers.data(name='4', shape=[-1, args.max_seq_len, 1], dtype='float32')
mask_label = fluid.layers.data(name='5', shape=[-1, 1], dtype='int64')
mask_pos = fluid.layers.data(name='6', shape=[-1, 1], dtype='int64')
labels = fluid.layers.data(name='r', shape=[-1, 1], dtype='int64')
pyreader = fluid.io.DataLoader.from_generator(feed_list=[
src_ids, pos_ids, sent_ids, input_mask, mask_label, mask_pos, labels
], capacity=70, iterable=False)
ernie = ErnieModel(
src_ids=src_ids,
......@@ -97,7 +94,7 @@ def predict_wrapper(args,
def predict(exe=exe, pyreader=pyreader):
pyreader.decorate_tensor_provider(data_reader.data_generator())
pyreader.set_batch_generator(data_reader.data_generator())
pyreader.start()
cost = 0
......@@ -285,7 +282,7 @@ def train(args):
next_sent_acc.name, mask_lm_loss.name, total_loss.name
])
train_pyreader.decorate_tensor_provider(data_reader.data_generator())
train_pyreader.set_batch_generator(data_reader.data_generator())
train_pyreader.start()
steps = 0
cost = []
......
ernie/utils/args.py
浏览文件 @ 4f1537e2
......@@ -20,6 +20,8 @@ from __future__ import absolute_import
import six
import os
import sys
import argparse
import logging
......@@ -33,7 +35,7 @@ def prepare_logger(logger, debug=False, save_to_file=None):
console_hdl = logging.StreamHandler()
console_hdl.setFormatter(formatter)
logger.addHandler(console_hdl)
if save_to_file is not None and not os.path.exits(save_to_file):
if save_to_file is not None and not os.path.exists(save_to_file):
file_hdl = logging.FileHandler(save_to_file)
file_hdl.setFormatter(formatter)
logger.addHandler(file_hdl)
......
ernie/utils/data.py
浏览文件 @ 4f1537e2
......@@ -4,6 +4,7 @@ import re
from propeller import log
import itertools
from propeller.paddle.data import Dataset
import pickle
import six
......@@ -101,7 +102,7 @@ class SpaceTokenizer(object):
class CharTokenizer(object):
def __init__(self, vocab, lower=True):
def __init__(self, vocab, lower=True, sentencepiece_style_vocab=False):
"""
char tokenizer (wordpiece english)
normed txt(space seperated or not) => list of word-piece
......@@ -110,6 +111,7 @@ class CharTokenizer(object):
#self.pat = re.compile(r'([,.!?\u3002\uff1b\uff0c\uff1a\u201c\u201d\uff08\uff09\u3001\uff1f\u300a\u300b]|[\u4e00-\u9fa5]|[a-zA-Z0-9]+)')
self.pat = re.compile(r'([a-zA-Z0-9]+|\S)')
self.lower = lower
self.sentencepiece_style_vocab = sentencepiece_style_vocab
def __call__(self, sen):
if len(sen) == 0:
......@@ -119,11 +121,51 @@ class CharTokenizer(object):
sen = sen.lower()
res = []
for match in self.pat.finditer(sen):
words, _ = wordpiece(match.group(0), vocab=self.vocab, unk_token='[UNK]')
words, _ = wordpiece(match.group(0), vocab=self.vocab, unk_token='[UNK]', sentencepiece_style_vocab=self.sentencepiece_style_vocab)
res.extend(words)
return res
class WSSPTokenizer(object):
def __init__(self, sp_model_dir, word_dict, ws=True, lower=True):
self.ws = ws
self.lower = lower
self.dict = pickle.load(open(word_dict, 'rb'), encoding='utf8')
import sentencepiece as spm
self.sp_model = spm.SentencePieceProcessor()
self.window_size = 5
self.sp_model.Load(sp_model_dir)
def cut(self, chars):
words = []
idx = 0
while idx < len(chars):
matched = False
for i in range(self.window_size, 0, -1):
cand = chars[idx: idx+i]
if cand in self.dict:
words.append(cand)
matched = True
break
if not matched:
i = 1
words.append(chars[idx])
idx += i
return words
def __call__(self, sen):
sen = sen.decode('utf8')
if self.ws:
sen = [s for s in self.cut(sen) if s != ' ']
else:
sen = sen.split(' ')
if self.lower:
sen = [s.lower() for s in sen]
sen = ' '.join(sen)
ret = self.sp_model.EncodeAsPieces(sen)
return ret
def build_2_pair(seg_a, seg_b, max_seqlen, cls_id, sep_id):
token_type_a = np.ones_like(seg_a, dtype=np.int64) * 0
token_type_b = np.ones_like(seg_b, dtype=np.int64) * 1
......
example/finetune_classifier.py
浏览文件 @ 4f1537e2
......@@ -55,7 +55,7 @@ class ClassificationErnieModel(propeller.train.Model):
pos_ids = L.cast(pos_ids, 'int64')
pos_ids.stop_gradient = True
input_mask.stop_gradient = True
task_ids = L.zeros_like(src_ids) + self.hparam.task_id #this shit wont use at the moment
task_ids = L.zeros_like(src_ids) + self.hparam.task_id
task_ids.stop_gradient = True
ernie = ErnieModel(
......@@ -128,6 +128,8 @@ if __name__ == '__main__':
parser.add_argument('--vocab_file', type=str, required=True)
parser.add_argument('--do_predict', action='store_true')
parser.add_argument('--warm_start_from', type=str)
parser.add_argument('--sentence_piece_model', type=str, default=None)
parser.add_argument('--word_dict', type=str, default=None)
args = parser.parse_args()
run_config = propeller.parse_runconfig(args)
hparams = propeller.parse_hparam(args)
......@@ -138,7 +140,12 @@ if __name__ == '__main__':
cls_id = vocab['[CLS]']
unk_id = vocab['[UNK]']
tokenizer = utils.data.CharTokenizer(vocab.keys())
if args.sentence_piece_model is not None:
if args.word_dict is None:
raise ValueError('--word_dict no specified in subword Model')
tokenizer = utils.data.WSSPTokenizer(args.sentence_piece_model, args.word_dict, ws=True, lower=True)
else:
tokenizer = utils.data.CharTokenizer(vocab.keys())
def tokenizer_func(inputs):
'''avoid pickle error'''
......@@ -179,7 +186,7 @@ if __name__ == '__main__':
dev_ds.data_shapes = shapes
dev_ds.data_types = types
varname_to_warmstart = re.compile('encoder.*|pooled.*|.*embedding|pre_encoder_.*')
varname_to_warmstart = re.compile(r'^encoder.*[wb]_0$|^.*embedding$|^.*bias$|^.*scale$|^pooled_fc.[wb]_0$')
warm_start_dir = args.warm_start_from
ws = propeller.WarmStartSetting(
predicate_fn=lambda v: varname_to_warmstart.match(v.name) and os.path.exists(os.path.join(warm_start_dir, v.name)),
......
example/finetune_ner.py
浏览文件 @ 4f1537e2
......@@ -32,7 +32,6 @@ import paddle.fluid.layers as L
from model.ernie import ErnieModel
from optimization import optimization
import tokenization
import utils.data
from propeller import log
......@@ -121,7 +120,7 @@ class SequenceLabelErnieModel(propeller.train.Model):
def make_sequence_label_dataset(name, input_files, label_list, tokenizer, batch_size, max_seqlen, is_train):
label_map = {v: i for i, v in enumerate(label_list)}
no_entity_id = label_map['O']
delimiter = ''
delimiter = b''
def read_bio_data(filename):
ds = propeller.data.Dataset.from_file(filename)
......@@ -132,10 +131,10 @@ def make_sequence_label_dataset(name, input_files, label_list, tokenizer, batch_
while 1:
line = next(iterator)
cols = line.rstrip(b'\n').split(b'\t')
tokens = cols[0].split(delimiter)
labels = cols[1].split(delimiter)
if len(cols) != 2:
continue
tokens = tokenization.convert_to_unicode(cols[0]).split(delimiter)
labels = tokenization.convert_to_unicode(cols[1]).split(delimiter)
if len(tokens) != len(labels) or len(tokens) == 0:
continue
yield [tokens, labels]
......@@ -151,7 +150,8 @@ def make_sequence_label_dataset(name, input_files, label_list, tokenizer, batch_
ret_tokens = []
ret_labels = []
for token, label in zip(tokens, labels):
sub_token = tokenizer.tokenize(token)
sub_token = tokenizer(token)
label = label.decode('utf8')
if len(sub_token) == 0:
continue
ret_tokens.extend(sub_token)
......@@ -179,7 +179,7 @@ def make_sequence_label_dataset(name, input_files, label_list, tokenizer, batch_
labels = labels[: max_seqlen - 2]
tokens = ['[CLS]'] + tokens + ['[SEP]']
token_ids = tokenizer.convert_tokens_to_ids(tokens)
token_ids = [vocab[t] for t in tokens]
label_ids = [no_entity_id] + [label_map[x] for x in labels] + [no_entity_id]
token_type_ids = [0] * len(token_ids)
input_seqlen = len(token_ids)
......@@ -211,7 +211,7 @@ def make_sequence_label_dataset(name, input_files, label_list, tokenizer, batch_
def make_sequence_label_dataset_from_stdin(name, tokenizer, batch_size, max_seqlen):
delimiter = ''
delimiter = b''
def stdin_gen():
if six.PY3:
......@@ -232,9 +232,9 @@ def make_sequence_label_dataset_from_stdin(name, tokenizer, batch_size, max_seql
while 1:
line, = next(iterator)
cols = line.rstrip(b'\n').split(b'\t')
tokens = cols[0].split(delimiter)
if len(cols) != 1:
continue
tokens = tokenization.convert_to_unicode(cols[0]).split(delimiter)
if len(tokens) == 0:
continue
yield tokens,
......@@ -247,7 +247,7 @@ def make_sequence_label_dataset_from_stdin(name, tokenizer, batch_size, max_seql
tokens, = next(iterator)
ret_tokens = []
for token in tokens:
sub_token = tokenizer.tokenize(token)
sub_token = tokenizer(token)
if len(sub_token) == 0:
continue
ret_tokens.extend(sub_token)
......@@ -266,7 +266,7 @@ def make_sequence_label_dataset_from_stdin(name, tokenizer, batch_size, max_seql
tokens = tokens[: max_seqlen - 2]
tokens = ['[CLS]'] + tokens + ['[SEP]']
token_ids = tokenizer.convert_tokens_to_ids(tokens)
token_ids = [vocab[t] for t in tokens]
token_type_ids = [0] * len(token_ids)
input_seqlen = len(token_ids)
......@@ -296,13 +296,15 @@ if __name__ == '__main__':
parser.add_argument('--data_dir', type=str, required=True)
parser.add_argument('--vocab_file', type=str, required=True)
parser.add_argument('--do_predict', action='store_true')
parser.add_argument('--use_sentence_piece_vocab', action='store_true')
parser.add_argument('--warm_start_from', type=str)
args = parser.parse_args()
run_config = propeller.parse_runconfig(args)
hparams = propeller.parse_hparam(args)
tokenizer = tokenization.FullTokenizer(args.vocab_file)
vocab = tokenizer.vocab
vocab = {j.strip().split('\t')[0]: i for i, j in enumerate(open(args.vocab_file, 'r', encoding='utf8'))}
tokenizer = utils.data.CharTokenizer(vocab, sentencepiece_style_vocab=args.use_sentence_piece_vocab)
sep_id = vocab['[SEP]']
cls_id = vocab['[CLS]']
unk_id = vocab['[UNK]']
......@@ -358,7 +360,7 @@ if __name__ == '__main__':
from_dir=warm_start_dir
)
best_exporter = propeller.train.exporter.BestExporter(os.path.join(run_config.model_dir, 'best'), cmp_fn=lambda old, new: new['dev']['f1'] > old['dev']['f1'])
best_exporter = propeller.train.exporter.BestInferenceModelExporter(os.path.join(run_config.model_dir, 'best'), cmp_fn=lambda old, new: new['dev']['f1'] > old['dev']['f1'])
propeller.train.train_and_eval(
model_class_or_model_fn=SequenceLabelErnieModel,
params=hparams,
......@@ -387,7 +389,6 @@ if __name__ == '__main__':
predict_ds.data_types = types
rev_label_map = {i: v for i, v in enumerate(label_list)}
best_exporter = propeller.train.exporter.BestExporter(os.path.join(run_config.model_dir, 'best'), cmp_fn=lambda old, new: new['dev']['f1'] > old['dev']['f1'])
learner = propeller.Learner(SequenceLabelErnieModel, run_config, hparams)
for pred, _ in learner.predict(predict_ds, ckpt=-1):
pred_str = ' '.join([rev_label_map[idx] for idx in np.argmax(pred, 1).tolist()])
......
example/finetune_ranker.py
浏览文件 @ 4f1537e2
......@@ -146,6 +146,7 @@ if __name__ == '__main__':
parser.add_argument('--data_dir', type=str, required=True)
parser.add_argument('--warm_start_from', type=str)
parser.add_argument('--sentence_piece_model', type=str, default=None)
parser.add_argument('--word_dict', type=str, default=None)
args = parser.parse_args()
run_config = propeller.parse_runconfig(args)
hparams = propeller.parse_hparam(args)
......@@ -157,7 +158,9 @@ if __name__ == '__main__':
unk_id = vocab['[UNK]']
if args.sentence_piece_model is not None:
tokenizer = utils.data.JBSPTokenizer(args.sentence_piece_model, jb=True, lower=True)
if args.word_dict is None:
raise ValueError('--word_dict no specified in subword Model')
tokenizer = utils.data.WSSPTokenizer(args.sentence_piece_model, args.word_dict, ws=True, lower=True)
else:
tokenizer = utils.data.CharTokenizer(vocab.keys())
......@@ -218,7 +221,7 @@ if __name__ == '__main__':
from_dir=warm_start_dir
)
best_exporter = propeller.train.exporter.BestExporter(os.path.join(run_config.model_dir, 'best'), cmp_fn=lambda old, new: new['dev']['f1'] > old['dev']['f1'])
best_exporter = propeller.train.exporter.BestInferenceModelExporter(os.path.join(run_config.model_dir, 'best'), cmp_fn=lambda old, new: new['dev']['f1'] > old['dev']['f1'])
propeller.train_and_eval(
model_class_or_model_fn=RankingErnieModel,
params=hparams,
......@@ -258,6 +261,7 @@ if __name__ == '__main__':
est = propeller.Learner(RankingErnieModel, run_config, hparams)
for qid, res in est.predict(predict_ds, ckpt=-1):
print('%d\t%d\t%.5f\t%.5f' % (qid[0], np.argmax(res), res[0], res[1]))
#for i in predict_ds:
# sen = i[0]
# for ss in np.squeeze(sen):
......
inference/README.md 0 → 100644
浏览文件 @ 4f1537e2
# ERNIE fast inference (C++)
ERNIE C++ fast inference API提供了一种更为高效的在线预测方案，可以直接联编译至生产环境以获取更好的性能。
其实现基于[fluid inference](https://www.paddlepaddle.org.cn/documentation/docs/zh/advanced_usage/deploy/inference/native_infer.html).
**请确保您的 fluid inference 版本高于 1.6.3 以获得正确的预测结果。**
本页面提供了一个ERNIE C++ fast inference 的 demo benchmark.
## 准备工作
demo 数据取自XNLI数据集test集合，位于./data 中。采用明文id格式，一行代表一个 batch, 包含四个字段：
```text
src_ids, pos_ids, sent_ids, self_attn_mask
```
字段之间按照分号(;)分隔；各字段内部包含 `shape` 和 `data` 两部分，按照冒号(:)分隔； `shape` 和 `data` 内部按空格分隔；`self_attn_mask` 为 FLOAT32 类型，其余字段为 INT64 类型。
ERNIE fast inference 需要输入 inference\_model 格式的模型，可以参考[这里](../README.zh.md#生成inference_model)生成 inference\_model .
**使用propeller产出的 inference\_model 只需要`src_ids`，`sent_ids` 两个字段，因此需要适当修改数据文件**
## 编译和运行
为了编译本 demo，c++ 编译器需要支持 C++11 标准。
下载对应的 [fluid_inference库](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/advanced_usage/deploy/inference/build_and_install_lib_cn.html) , 根据使用的 paddle 的版本和配置状况 (是否使用 avx, mkl, 以及 cuda, cudnn 版本) 选择下载对应的版本并解压，会得到 `fluid_inference` 文件夹，将其放在与`inference.cc`同一级目录。
用以下命令编译：
``` bash
cd ./gpu # cd ./cpu
mkdir build
cd build
cmake ..
make
```
用以下命令运行：
```
./run.sh ../data/sample /path/to/inference_mode_dir
```
## 性能测试
测试样本：XNLI test集合，输入BatchSize=1, SequenceLength=128.
重复5遍取平均值。
| 测试环境 | 延迟(ms) |
| ----- | ----- |
| CPU（Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz (20 线程)） | 29.8818|
| GPU （P4） | 8.5 |
inference/cpu/CMakeLists.txt 0 → 100644
浏览文件 @ 4f1537e2
CMAKE_MINIMUM_REQUIRED(VERSION 3.2)
PROJECT(inference_demo)
SET(CMAKE_C_COMPILER gcc)
SET(CMAKE_CXX_COMPILER g++)
ADD_COMPILE_OPTIONS(-std=c++11 -g)
SET(FLUID_INFER_LIB fluid_inference)
SET(FLUID_INC_PATH ${FLUID_INFER_LIB}/paddle/include)
SET(FLUID_LIB_PATH ${FLUID_INFER_LIB}/paddle/lib)
SET(GLOG_INC_PATH ${FLUID_INFER_LIB}/third_party/install/glog/include)
SET(GLOG_LIB_PATH ${FLUID_INFER_LIB}/third_party/install/glog/lib)
SET(GFLAGS_INC_PATH ${FLUID_INFER_LIB}/third_party/install/gflags/include)
SET(GFLAGS_LIB_PATH ${FLUID_INFER_LIB}/third_party/install/gflags/lib)
SET(MKLDNN_LIB_PATH ${FLUID_INFER_LIB}/third_party/install/mkldnn/lib)
SET(MKLML_LIB_PATH ${FLUID_INFER_LIB}/third_party/install/mklml/lib)
INCLUDE_DIRECTORIES(${FLUID_INC_PATH})
INCLUDE_DIRECTORIES(${GLOG_INC_PATH})
INCLUDE_DIRECTORIES(${GFLAGS_INC_PATH})
LINK_DIRECTORIES(${FLUID_LIB_PATH})
LINK_DIRECTORIES(${GLOG_LIB_PATH})
LINK_DIRECTORIES(${GFLAGS_LIB_PATH})
LINK_DIRECTORIES(${MKLML_LIB_PATH})
LINK_DIRECTORIES(${MKLDNN_LIB_PATH})
ADD_EXECUTABLE(inference inference.cc)
TARGET_LINK_LIBRARIES(inference dl paddle_fluid glog gflags pthread)
inference/cpu/inference.cc 0 → 100644
浏览文件 @ 4f1537e2
// Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <gflags/gflags.h>
#include <glog/logging.h>
#include <paddle_inference_api.h>
#include <chrono>
#include <fstream>
#include <iostream>
#include <numeric>
#include <sstream>
#include <string>
#include <vector>
DEFINE_string(model_dir, "", "model directory");
DEFINE_string(data, "", "input data path");
DEFINE_int32(repeat, 1, "repeat");
DEFINE_bool(output_prediction, false, "Whether to output the prediction results.");
DEFINE_bool(use_gpu, false, "Whether to use GPU for prediction.");
DEFINE_int32(device, 0, "device.");
template <typename T>
void GetValueFromStream(std::stringstream *ss, T *t) {
(*ss) >> (*t);
}
template <>
void GetValueFromStream<std::string>(std::stringstream *ss, std::string *t) {
*t = ss->str();
}
// Split string to vector
template <typename T>
void Split(const std::string &line, char sep, std::vector<T> *v) {
std::stringstream ss;
T t;
for (auto c : line) {
if (c != sep) {
ss << c;
} else {
GetValueFromStream<T>(&ss, &t);
v->push_back(std::move(t));
ss.str({});
ss.clear();
}
}
if (!ss.str().empty()) {
GetValueFromStream<T>(&ss, &t);
v->push_back(std::move(t));
ss.str({});
ss.clear();
}
}
template <typename T>
constexpr paddle::PaddleDType GetPaddleDType();
template <>
constexpr paddle::PaddleDType GetPaddleDType<int64_t>() {
return paddle::PaddleDType::INT64;
}
template <>
constexpr paddle::PaddleDType GetPaddleDType<float>() {
return paddle::PaddleDType::FLOAT32;
}
// Parse tensor from string
template <typename T>
bool ParseTensor(const std::string &field, paddle::PaddleTensor *tensor) {
std::vector<std::string> data;
Split(field, ':', &data);
if (data.size() < 2) return false;
std::string shape_str = data[0];
std::vector<int> shape;
Split(shape_str, ' ', &shape);
std::string mat_str = data[1];
std::vector<T> mat;
Split(mat_str, ' ', &mat);
tensor->shape = shape;
auto size =
std::accumulate(shape.begin(), shape.end(), 1, std::multiplies<int>()) *
sizeof(T);
tensor->data.Resize(size);
std::copy(mat.begin(), mat.end(), static_cast<T *>(tensor->data.data()));
tensor->dtype = GetPaddleDType<T>();
return true;
}
// Parse input tensors from string
bool ParseLine(const std::string &line,
std::vector<paddle::PaddleTensor> *tensors) {
std::vector<std::string> fields;
Split(line, ';', &fields);
if (fields.size() <= 2) return false;
tensors->clear();
tensors->reserve(4);
int i = 0;
// src_ids
paddle::PaddleTensor src_ids;
ParseTensor<int64_t>(fields[i++], &src_ids);
src_ids.name = "eval_placeholder_0";
tensors->push_back(src_ids);
// sent_ids
paddle::PaddleTensor sent_ids;
ParseTensor<int64_t>(fields[i++], &sent_ids);
sent_ids.name = "eval_placeholder_1";
tensors->push_back(sent_ids);
// pos_ids
paddle::PaddleTensor pos_ids;
ParseTensor<int64_t>(fields[i++], &pos_ids);
pos_ids.name = "eval_placeholder_2";
tensors->push_back(pos_ids);
// input_mask
paddle::PaddleTensor input_mask;
ParseTensor<float>(fields[i++], &input_mask);
input_mask.name = "eval_placeholder_3";
tensors->push_back(input_mask);
return true;
}
// Print outputs to log
void PrintOutputs(const std::vector<paddle::PaddleTensor> &outputs) {
//LOG(INFO) << "example_id\tcontradiction\tentailment\tneutral";
for (size_t i = 0; i < outputs.front().data.length() / sizeof(float) / 3; i += 1) {
std::cout << static_cast<float *>(outputs[0].data.data())[3 * i] << "\t"
<< static_cast<float *>(outputs[0].data.data())[3 * i + 1] << "\t"
<< static_cast<float *>(outputs[0].data.data())[3 * i + 2] << std::endl;
}
}
bool LoadInputData(std::vector<std::vector<paddle::PaddleTensor>> *inputs) {
if (FLAGS_data.empty()) {
LOG(ERROR) << "please set input data path";
return false;
}
std::ifstream fin(FLAGS_data);
std::string line;
int lineno = 0;
while (std::getline(fin, line)) {
std::vector<paddle::PaddleTensor> feed_data;
if (!ParseLine(line, &feed_data)) {
LOG(ERROR) << "Parse line[" << lineno << "] error!";
} else {
inputs->push_back(std::move(feed_data));
}
}
return true;
}
// ernie inference demo
// Options:
// --model_dir: ernie model file directory
// --data: data path
// --repeat: repeat num
// --use_gpu: use gpu
int main(int argc, char *argv[]) {
google::InitGoogleLogging(*argv);
gflags::ParseCommandLineFlags(&argc, &argv, true);
if (FLAGS_model_dir.empty()) {
LOG(ERROR) << "please set model dir";
return -1;
}
paddle::AnalysisConfig config;
config.SetModel(FLAGS_model_dir);
config.DisableGpu();
config.SwitchIrOptim();
config.EnableMKLDNN();
config.SetCpuMathLibraryNumThreads(20);
//config.EnableMemoryOptim();
auto predictor = CreatePaddlePredictor(config);
std::vector<std::vector<paddle::PaddleTensor>> inputs;
if (!LoadInputData(&inputs)) {
LOG(ERROR) << "load input data error!";
return -1;
}
std::vector<paddle::PaddleTensor> fetch;
int total_time{0};
// auto predict_timer = []()
int num_samples{0};
int count{0};
for (int i = 0; i < FLAGS_repeat; i++) {
for (auto feed : inputs) {
fetch.clear();
auto start = std::chrono::system_clock::now();
predictor->Run(feed, &fetch);
if (FLAGS_output_prediction && i == 0) {
PrintOutputs(fetch);
}
auto end = std::chrono::system_clock::now();
count += 1;
if (!fetch.empty()) {
total_time +=
std::chrono::duration_cast<std::chrono::milliseconds>(end - start)
.count();
//num_samples += fetch.front().data.length() / 2 / sizeof(float);
num_samples += fetch.front().data.length() / (sizeof(float) * 2);
}
}
}
auto per_sample_ms =
static_cast<float>(total_time) / num_samples;
LOG(INFO) << "Run " << num_samples
<< " samples, average latency: " << per_sample_ms
<< "ms per sample.";
LOG(INFO) << count;
return 0;
}
inference/cpu/run.sh 0 → 100755
浏览文件 @ 4f1537e2
set -x
(($# != 2)) && echo "${0} data model" && exit -1
export LD_LIBRARY_PATH=fluid_inference/third_party/install/mkldnn/lib:fluid_inference/third_party/install/mklml/lib:fluid_inference/paddle/lib/:/home/work/cuda-9.0/lib64/:/home/work/cudnn/cudnn_v7_3_1_cuda9.0/lib64/:$LD_LIBRARY_PATH \
./build/inference --logtostderr \
--model_dir $2 \
--data $1 \
--repeat 5 \
--output_prediction true \
--use_gpu true \
--device 0 \
inference/data/sample 0 → 100644
浏览文件 @ 4f1537e2
此差异已折叠。
inference/gpu/CMakeLists.txt 0 → 100644
浏览文件 @ 4f1537e2
CMAKE_MINIMUM_REQUIRED(VERSION 3.2)
PROJECT(inference_demo)
SET(CMAKE_C_COMPILER gcc)
SET(CMAKE_CXX_COMPILER g++)
ADD_COMPILE_OPTIONS(-std=c++11 -g)
SET(FLUID_INFER_LIB fluid_inference)
SET(FLUID_INC_PATH ${FLUID_INFER_LIB}/paddle/include)
SET(FLUID_LIB_PATH ${FLUID_INFER_LIB}/paddle/lib)
SET(GLOG_INC_PATH ${FLUID_INFER_LIB}/third_party/install/glog/include)
SET(GLOG_LIB_PATH ${FLUID_INFER_LIB}/third_party/install/glog/lib)
SET(GFLAGS_INC_PATH ${FLUID_INFER_LIB}/third_party/install/gflags/include)
SET(GFLAGS_LIB_PATH ${FLUID_INFER_LIB}/third_party/install/gflags/lib)
SET(MKLDNN_LIB_PATH ${FLUID_INFER_LIB}/third_party/install/mkldnn/lib)
SET(MKLML_LIB_PATH ${FLUID_INFER_LIB}/third_party/install/mklml/lib)
INCLUDE_DIRECTORIES(${FLUID_INC_PATH})
INCLUDE_DIRECTORIES(${GLOG_INC_PATH})
INCLUDE_DIRECTORIES(${GFLAGS_INC_PATH})
LINK_DIRECTORIES(${FLUID_LIB_PATH})
LINK_DIRECTORIES(${GLOG_LIB_PATH})
LINK_DIRECTORIES(${GFLAGS_LIB_PATH})
LINK_DIRECTORIES(${MKLML_LIB_PATH})
LINK_DIRECTORIES(${MKLDNN_LIB_PATH})
ADD_EXECUTABLE(inference inference.cc)
TARGET_LINK_LIBRARIES(inference dl paddle_fluid glog gflags pthread)
inference/gpu/inference.cc 0 → 100644
浏览文件 @ 4f1537e2
// Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <gflags/gflags.h>
#include <glog/logging.h>
#include <paddle_inference_api.h>
#include <chrono>
#include <fstream>
#include <iostream>
#include <numeric>
#include <sstream>
#include <string>
#include <vector>
DEFINE_string(model_dir, "", "model directory");
DEFINE_string(data, "", "input data path");
DEFINE_int32(repeat, 1, "repeat");
DEFINE_bool(output_prediction, false, "Whether to output the prediction results.");
DEFINE_bool(use_gpu, false, "Whether to use GPU for prediction.");
DEFINE_int32(device, 0, "device.");
template <typename T>
void GetValueFromStream(std::stringstream *ss, T *t) {
(*ss) >> (*t);
}
template <>
void GetValueFromStream<std::string>(std::stringstream *ss, std::string *t) {
*t = ss->str();
}
// Split string to vector
template <typename T>
void Split(const std::string &line, char sep, std::vector<T> *v) {
std::stringstream ss;
T t;
for (auto c : line) {
if (c != sep) {
ss << c;
} else {
GetValueFromStream<T>(&ss, &t);
v->push_back(std::move(t));
ss.str({});
ss.clear();
}
}
if (!ss.str().empty()) {
GetValueFromStream<T>(&ss, &t);
v->push_back(std::move(t));
ss.str({});
ss.clear();
}
}
template <typename T>
constexpr paddle::PaddleDType GetPaddleDType();
template <>
constexpr paddle::PaddleDType GetPaddleDType<int64_t>() {
return paddle::PaddleDType::INT64;
}
template <>
constexpr paddle::PaddleDType GetPaddleDType<float>() {
return paddle::PaddleDType::FLOAT32;
}
// Parse tensor from string
template <typename T>
bool ParseTensor(const std::string &field, paddle::PaddleTensor *tensor) {
std::vector<std::string> data;
Split(field, ':', &data);
if (data.size() < 2) return false;
std::string shape_str = data[0];
std::vector<int> shape;
Split(shape_str, ' ', &shape);
std::string mat_str = data[1];
std::vector<T> mat;
Split(mat_str, ' ', &mat);
tensor->shape = shape;
auto size =
std::accumulate(shape.begin(), shape.end(), 1, std::multiplies<int>()) *
sizeof(T);
tensor->data.Resize(size);
std::copy(mat.begin(), mat.end(), static_cast<T *>(tensor->data.data()));
tensor->dtype = GetPaddleDType<T>();
return true;
}
// Parse input tensors from string
bool ParseLine(const std::string &line,
std::vector<paddle::PaddleTensor> *tensors) {
std::vector<std::string> fields;
Split(line, ';', &fields);
if (fields.size() <= 2) return false;
tensors->clear();
tensors->reserve(4);
int i = 0;
paddle::PaddleTensor src_ids;
ParseTensor<int64_t>(fields[i++], &src_ids);
src_ids.name = "eval_placeholder_0";
tensors->push_back(src_ids);
// sent_ids
paddle::PaddleTensor sent_ids;
ParseTensor<int64_t>(fields[i++], &sent_ids);
sent_ids.name = "eval_placeholder_1";
tensors->push_back(sent_ids);
// pos_ids
paddle::PaddleTensor pos_ids;
ParseTensor<int64_t>(fields[i++], &pos_ids);
pos_ids.name = "eval_placeholder_2";
tensors->push_back(pos_ids);
// input_mask
paddle::PaddleTensor input_mask;
ParseTensor<float>(fields[i++], &input_mask);
input_mask.name = "eval_placeholder_3";
tensors->push_back(input_mask);
return true;
}
// Print outputs to log
void PrintOutputs(const std::vector<paddle::PaddleTensor> &outputs) {
//LOG(INFO) << "example_id\tcontradiction\tentailment\tneutral";
for (size_t i = 0; i < outputs.front().data.length() / sizeof(float) / 3; i += 1) {
std::cout << static_cast<float *>(outputs[0].data.data())[3 * i] << "\t"
<< static_cast<float *>(outputs[0].data.data())[3 * i + 1] << "\t"
<< static_cast<float *>(outputs[0].data.data())[3 * i + 2] << std::endl;
}
}
bool LoadInputData(std::vector<std::vector<paddle::PaddleTensor>> *inputs) {
if (FLAGS_data.empty()) {
LOG(ERROR) << "please set input data path";
return false;
}
std::ifstream fin(FLAGS_data);
std::string line;
int lineno = 0;
while (std::getline(fin, line)) {
std::vector<paddle::PaddleTensor> feed_data;
if (!ParseLine(line, &feed_data)) {
LOG(ERROR) << "Parse line[" << lineno << "] error!";
} else {
inputs->push_back(std::move(feed_data));
}
}
return true;
}
// ernie inference demo
// Options:
// --model_dir: ernie model file directory
// --data: data path
// --repeat: repeat num
// --use_gpu: use gpu
int main(int argc, char *argv[]) {
google::InitGoogleLogging(*argv);
gflags::ParseCommandLineFlags(&argc, &argv, true);
if (FLAGS_model_dir.empty()) {
LOG(ERROR) << "please set model dir";
return -1;
}
paddle::AnalysisConfig config;
config.SetModel(FLAGS_model_dir);
config.EnableUseGpu(100, 0);
config.SwitchSpecifyInputNames(true);
config.EnableCUDNN();
config.SwitchIrOptim(true);
config.EnableMemoryOptim();
auto predictor = CreatePaddlePredictor(config);
std::vector<std::vector<paddle::PaddleTensor>> inputs;
if (!LoadInputData(&inputs)) {
LOG(ERROR) << "load input data error!";
return -1;
}
std::vector<paddle::PaddleTensor> fetch;
int total_time{0};
// auto predict_timer = []()
int num_samples{0};
int count{0};
for (int i = 0; i < FLAGS_repeat; i++) {
for (auto feed : inputs) {
fetch.clear();
auto start = std::chrono::system_clock::now();
predictor->Run(feed, &fetch);
if (FLAGS_output_prediction && i == 0) {
PrintOutputs(fetch);
}
auto end = std::chrono::system_clock::now();
count += 1;
if (!fetch.empty()) {
total_time +=
std::chrono::duration_cast<std::chrono::milliseconds>(end - start)
.count();
//num_samples += fetch.front().data.length() / 2 / sizeof(float);
num_samples += fetch.front().data.length() / (sizeof(float) * 2);
}
}
}
auto per_sample_ms =
static_cast<float>(total_time) / num_samples;
LOG(INFO) << "Run " << num_samples
<< " samples, average latency: " << per_sample_ms
<< "ms per sample.";
LOG(INFO) << count;
return 0;
}
inference/gpu/run.sh 0 → 100755
浏览文件 @ 4f1537e2
set -x
(($# != 2)) && echo "${0} data model" && exit -1
export LD_LIBRARY_PATH=fluid_inference/third_party/install/mkldnn/lib:fluid_inference/third_party/install/mklml/lib:fluid_inference/paddle/lib/:/home/work/cuda-9.0/lib64/:/home/work/cudnn/cudnn_v7_3_1_cuda9.0/lib64/:$LD_LIBRARY_PATH
./build/inference --logtostderr \
--model_dir $2 \
--data $1 \
--repeat 5 \
--output_prediction true \
--use_gpu true \
--device 0 \
propeller/README.en.md
浏览文件 @ 4f1537e2
......@@ -18,11 +18,6 @@ Propeller provide the following benefits:
- create checkpoint files and recover from failures
- save visualizable results
## install
```script
pip install --user .
```
## Getting Started
```python
......@@ -71,6 +66,7 @@ pip install --user .
# Start training!
propeller.train_and_eval(BowModel, hparams, run_config, train_ds, eval_ds)
```
More detail see example/toy/
## Main Feature
1. train_and_eval
......
propeller/README.md
浏览文件 @ 4f1537e2
......@@ -19,10 +19,6 @@ Propeller 具有下列优势：
- 创建检查点文件并从故障中恢复
- 保存可视化的摘要结果
## install|安装
pip install --user .
## Getting Started|快速开始
```python
......@@ -70,6 +66,7 @@ pip install --user .
# 开始训练！
propeller.train_and_eval(BowModel, hparams, run_config, train_ds, eval_ds)
```
详细详细请见example/toy/
## 主要构件
1. train_and_eval
......
propeller/__init__.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Propeller"""
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals
......
propeller/data/__init__.py
浏览文件 @ 4f1537e2
......@@ -11,3 +11,6 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
doc
"""
propeller/data/functional.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Basic Dataset API"""
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals
......@@ -39,7 +40,7 @@ __all__ = ['Dataset']
@contextmanager
def open_file(filename, format=None):
def _open_file(filename, format=None):
if format is None:
fd = open(filename, 'rb')
elif format == 'GZIP':
......@@ -50,9 +51,9 @@ def open_file(filename, format=None):
fd.close()
def open_record(filename):
def gen():
with open_file(filename, format='GZIP') as f:
def _open_record(filename):
def _gen():
with _open_file(filename, format='GZIP') as f:
while True:
data = f.read(struct.calcsize('i'))
if not len(data):
......@@ -61,11 +62,11 @@ def open_record(filename):
data = f.read(l)
yield data
return gen
return _gen
def shuffle_func(dataset, buffer_size):
def gen():
def _shuffle_func(dataset, buffer_size):
def _gen():
buf = []
iterable = dataset()
try:
......@@ -82,11 +83,11 @@ def shuffle_func(dataset, buffer_size):
for i in buf:
yield i
return gen
return _gen
def interleave_func(iterable, map_fn, cycle_length, block_length):
def gen():
def _interleave_func(iterable, map_fn, cycle_length, block_length):
def _gen():
ls = itertools.tee(iterable(), cycle_length)
buf = []
for i, j in enumerate(ls):
......@@ -99,11 +100,11 @@ def interleave_func(iterable, map_fn, cycle_length, block_length):
for ii in (i for i in tup if i is not None):
yield ii
return gen
return _gen
def repeat_func(dataset, n):
def gen():
def _repeat_func(dataset, n):
def _gen():
iterable = dataset()
if n >= 0:
ret = itertools.chain(*itertools.tee(iterable, n))
......@@ -113,11 +114,11 @@ def repeat_func(dataset, n):
for i in ret:
yield i
return gen
return _gen
def filter_func(dataset, fn):
def gen():
def _filter_func(dataset, fn):
def _gen():
for i in dataset():
if isinstance(i, tuple) or isinstance(i, list):
if fn(*i) is True:
......@@ -126,41 +127,41 @@ def filter_func(dataset, fn):
if fn(i) is True:
yield i
return gen
return _gen
def map_func(dataset, fn):
def gen():
def _map_func(dataset, fn):
def _gen():
for i in dataset():
if isinstance(i, tuple) or isinstance(i, list):
yield fn(*i)
else:
yield fn(i)
return gen
return _gen
def shard_func(dataset, num_shards, index):
def gen():
def _shard_func(dataset, num_shards, index):
def _gen():
iterable = dataset()
ret = itertools.islice(iterable, index, None, num_shards)
for i in ret:
yield i
return gen
return _gen
def take_func(dataset, count):
def gen():
def _take_func(dataset, count):
def _gen():
iterable = dataset()
ret = itertools.islice(iterable, count)
for i in ret:
yield i
return gen
return _gen
def buffered_func(dataset, size):
def _buffered_func(dataset, size):
"""
Creates a buffered data reader.
......@@ -176,21 +177,21 @@ def buffered_func(dataset, size):
:returns: the buffered data reader.
"""
class EndSignal():
class _EndSignal(object):
pass
end = EndSignal()
end = _EndSignal()
def read_worker(r, q):
def _read_worker(r, q):
for d in r:
q.put(d)
q.put(end)
def data_reader():
def _data_reader():
r = dataset()
q = multiprocessing.Queue(maxsize=size)
t = multiprocessing.Process(
target=read_worker, args=(
target=_read_worker, args=(
r,
q, ))
t.daemon = True
......@@ -200,14 +201,14 @@ def buffered_func(dataset, size):
yield e
e = q.get()
return data_reader
return _data_reader
def padded_batch_func(dataset, batch_size, pad_value=0, max_seqlen=None):
def _padded_batch_func(dataset, batch_size, pad_value=0, max_seqlen=None):
if not isinstance(batch_size, int):
raise ValueError('unknown batch_size: %s' % repr(batch_size))
def gen():
def _gen():
iterable = dataset()
pad_value_t = pad_value
while True:
......@@ -226,71 +227,86 @@ def padded_batch_func(dataset, batch_size, pad_value=0, max_seqlen=None):
if (not np.isscalar(elem)) and elem.shape != ():
max_len = max(map(len,
e)) if max_seqlen is None else max_seqlen
e = map(lambda i: np.pad(i, [0, max_len - len(i)], 'constant', constant_values=pv) if max_len >= len(i) else i[: max_len], e)
def _fn(i):
if max_len >= len(i):
return np.pad(i, [0, max_len - len(i)],
'constant',
constant_values=pv)
else:
return i[:max_len]
e = map(_fn, e)
padded.append(np.stack(list(e)))
yield padded
return gen
return _gen
class Dataset(object):
"""Python Wrapper for PyReader"""
@classmethod
def from_generator_func(cls, gen, data_shapes=None, data_types=None):
if not inspect.isgeneratorfunction(gen):
raise ValueError('expect generator function, got %s' % repr(gen))
def from_generator_func(cls, _gen, data_shapes=None, data_types=None):
"""doc"""
if not inspect.isgeneratorfunction(_gen):
raise ValueError('expect generator function, got %s' % repr(_gen))
def wrapper(): #compat to py3.7
def _wrapper(): #compat to py3.7
try:
for item in gen():
for item in _gen():
yield item
except RuntimeError as e:
if str(e) != 'generator raised StopIteration':
raise e
ret = cls()
ret.generator = wrapper
ret.generator = _wrapper
ret.data_shapes = data_shapes
ret.data_types = data_types
return ret
@classmethod
def from_file(cls, filename, format=None):
"""doc"""
if os.path.getsize(filename) == 0:
raise RuntimeError('%s is empty' % filename)
def gen():
with open_file(filename, format) as f:
def _gen():
with _open_file(filename, format) as f:
for line in f:
yield line
ret = cls()
ret.generator = gen
ret.generator = _gen
ret.data_shapes = []
ret.data_types = str
return ret
@classmethod
def from_record_file(cls, filename):
"""doc"""
if os.path.getsize(filename) == 0:
raise RuntimeError('%s is empty' % filename)
gen = open_record(filename)
_gen = _open_record(filename)
ret = cls()
ret.generator = gen
ret.generator = _gen
ret.data_shapes = []
ret.data_types = str
return ret
@classmethod
def from_list(cls, ls):
"""doc"""
if not isinstance(ls, list):
raise ValueError('expect list, got %s' % repr(ls))
def gen():
def _gen():
for i in ls:
yield i
ret = cls()
ret.generator = gen
ret.generator = _gen
ret.data_shapes = []
ret.data_types = str
return ret
......@@ -339,6 +355,7 @@ class Dataset(object):
@property
def data_shapes(self):
"""doc"""
if self._data_shapes is None:
self._infer_shapes_and_types()
return self._data_shapes
......@@ -347,10 +364,12 @@ class Dataset(object):
@data_shapes.setter
def data_shapes(self, val):
"""doc"""
self._data_shapes = val
@property
def data_types(self):
"""doc"""
if self._data_types is None:
self._infer_shapes_and_types()
return self._data_types
......@@ -359,9 +378,11 @@ class Dataset(object):
@data_types.setter
def data_types(self, val):
"""doc"""
self._data_types = val
def apply(self, transform_func):
"""apply transform func to datasets"""
#input_shapes = transform_func.input_shapes
#input_types = transform_func.input_types
#data_shapes = transform_func.data_shapes
......@@ -377,46 +398,55 @@ class Dataset(object):
return ret
def shuffle(self, buffer_size):
func = functools.partial(shuffle_func, buffer_size=buffer_size)
"""doc"""
func = functools.partial(_shuffle_func, buffer_size=buffer_size)
return self.apply(func)
def repeat(self, n=-1):
func = functools.partial(repeat_func, n=n)
"""doc"""
func = functools.partial(_repeat_func, n=n)
return self.apply(func)
def map(self, fn):
func = functools.partial(map_func, fn=fn)
"""doc"""
func = functools.partial(_map_func, fn=fn)
return self.apply(func)
def filter(self, fn):
func = functools.partial(filter_func, fn=fn)
"""doc"""
func = functools.partial(_filter_func, fn=fn)
return self.apply(func)
def shard(self, num_shards, index):
"""doc"""
func = functools.partial(
shard_func, num_shards=num_shards, index=index)
_shard_func, num_shards=num_shards, index=index)
return self.apply(func)
def interleave(self, map_fn, cycle_length, block_length):
"""doc"""
func = functools.partial(
interleave_func,
_interleave_func,
map_fn=map_fn,
cycle_length=cycle_length,
block_length=block_length)
return self.apply(func)
def padded_batch(self, batch_size, pad_value=0, max_seqlen=None):
"""doc"""
func = functools.partial(
padded_batch_func,
_padded_batch_func,
batch_size=batch_size,
pad_value=pad_value,
max_seqlen=max_seqlen)
return self.apply(func)
def take(self, count=1):
func = functools.partial(take_func, count=count)
"""doc"""
func = functools.partial(_take_func, count=count)
return self.apply(func)
def buffered(self, size=10):
func = functools.partial(buffered_func, size=size)
"""doc"""
func = functools.partial(_buffered_func, size=size)
return self.apply(func)
propeller/paddle/__init__.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,9 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
doc
"""
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals
......
propeller/paddle/collection.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""global collections"""
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals
......@@ -43,13 +45,16 @@ class Collections(object):
_global_collection = None
def add(self, key, val):
"""doc"""
self.col.setdefault(key, []).append(val)
def get(self, key):
"""doc"""
return self.col.get(key, None)
def default_collection():
"""return global collection"""
global _global_collection
if _global_collection is None:
_global_collection = Collections()
......
propeller/paddle/data/__init__.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,9 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
doc
"""
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals
......
propeller/paddle/data/feature_column.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""FeatureColumns and many Column"""
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals
......@@ -30,7 +31,7 @@ import numpy as np
from glob import glob
from propeller.paddle.train import distribution
from propeller.data.functional import interleave_func
from propeller.data.functional import _interleave_func
from propeller.paddle.data.functional import Dataset
from propeller.paddle.data import example_pb2, feature_pb2
......@@ -43,35 +44,47 @@ __all__ = [
def basic_tokenizer(sen):
"""doc"""
seg = sen.split(b' ')
seg = filter(lambda i: i != b' ', seg)
return seg
class Column():
class Column(object):
"""doc"""
def __init__(self, name):
"""doc"""
pass
def raw_to_proto(self, raw):
"""doc"""
return feature_pb2.Feature()
@property
def output_shapes(self):
"""doc"""
pass
@property
def output_types(self):
"""doc"""
pass
def proto_to_instance(self, proto):
"""doc"""
raise NotImplementedError()
def raw_to_instance(self, raw):
"""doc"""
raise NotImplementedError()
class LabelColumn(Column):
"""doc"""
def __init__(self, name, vocab_dict=None, vocab_file=None):
"""doc"""
self.name = name
self.vocab = None
if vocab_file:
......@@ -84,13 +97,16 @@ class LabelColumn(Column):
@property
def output_shapes(self):
"""doc"""
return [1]
@property
def output_types(self):
"""doc"""
return 'int64'
def raw_to_proto(self, raw):
"""doc"""
if self.vocab is None:
ids = [int(raw)]
else:
......@@ -99,10 +115,12 @@ class LabelColumn(Column):
return fe
def proto_to_instance(self, feature):
"""doc"""
ret = np.array(feature.int64_list.value[0], dtype=np.int64)
return ret
def raw_to_instance(self, raw):
"""doc"""
if self.vocab is None:
ids = int(raw)
else:
......@@ -111,6 +129,8 @@ class LabelColumn(Column):
class TextColumn(Column):
"""doc"""
def __init__(self,
name,
unk_id,
......@@ -132,63 +152,75 @@ class TextColumn(Column):
@property
def output_shapes(self):
"""doc"""
return [-1]
@property
def output_types(self):
"""doc"""
return 'int64'
def raw_to_proto(self, raw):
"""doc"""
ids = [self.vocab.get(s, self.unk_id) for s in self.tokenizer(raw)]
fe = feature_pb2.Feature(int64_list=feature_pb2.Int64List(value=ids))
return fe
def proto_to_instance(self, feature):
"""doc"""
ret = np.array(feature.int64_list.value, dtype=np.int64)
return ret
def raw_to_instance(self, raw):
"""doc"""
ids = [self.vocab.get(s, self.unk_id) for s in self.tokenizer(raw)]
return np.array(ids, dtype=np.int64)
class TextIDColumn(Column):
"""doc"""
def __init__(self, name):
"""doc"""
self.name = name
@property
def output_shapes(self):
"""doc"""
return [-1]
@property
def output_types(self):
"""doc"""
return 'int64'
def raw_to_proto(self, raw):
"""doc"""
ids = [int(s) for s in raw.split(b' ')]
fe = feature_pb2.Feature(int64_list=feature_pb2.Int64List(value=ids))
return fe
def proto_to_instance(self, feature):
"""doc"""
ret = np.array(feature.int64_list.value, dtype=np.int64)
return ret
def raw_to_instance(self, raw):
"""doc"""
ret = np.array([int(i) for i in raw.split(b' ')], dtype=np.int64)
return ret
class FeatureColumns(object):
def __init__(self, columns, pad_id=0):
self._columns = columns
def _list_files(raw_dir):
return [os.path.join(raw_dir, p) for p in os.listdir(raw_dir)]
def raw_files(self, raw_dir):
return [os.path.join(raw_dir, p) for p in os.listdir(raw_dir)]
class FeatureColumns(object):
"""A Dataset Factory object"""
def gz_files(self, gz_dir):
return None if gz_dir is None else [
os.path.join(gz_dir, p) for p in os.listdir(gz_dir)
]
def __init__(self, columns):
"""doc"""
self._columns = columns
def _make_gz_dataset(self, raw_dir, gz_dir):
assert raw_dir or gz_dir, 'data_dir not specified when using gz mode'
......@@ -237,7 +269,7 @@ class FeatureColumns(object):
if shuffle:
dataset = dataset.shuffle(buffer_size=len(gz_files))
fn = partial(
interleave_func,
_interleave_func,
map_fn=lambda filename: Dataset.from_record_file(filename),
cycle_length=len(gz_files),
block_length=1)
......@@ -271,7 +303,7 @@ class FeatureColumns(object):
dataset = dataset.shuffle(buffer_size=len(data_files))
fn = partial(
interleave_func,
_interleave_func,
map_fn=lambda filename: Dataset.from_file(filename),
cycle_length=len(data_files),
block_length=1)
......@@ -294,9 +326,9 @@ class FeatureColumns(object):
def _read_stdin_dataset(self, encoding='utf8', shuffle=False, **kwargs):
log.info('reading raw files stdin')
def gen():
def _gen():
if six.PY3:
source = sys.stdin.buffer
source = sys.stdin.buffer
else:
source = sys.stdin
while True:
......@@ -305,12 +337,12 @@ class FeatureColumns(object):
break
yield line,
dataset = Dataset.from_generator_func(gen)
dataset = Dataset.from_generator_func(_gen)
if shuffle:
dataset = dataset.shuffle(buffer_size=1000)
def _parse_stdin(record_str):
'''function that takes python_str as input'''
"""function that takes python_str as input"""
features = record_str.strip(b'\n').split(b'\t')
ret = [
column.raw_to_instance(feature)
......@@ -346,13 +378,17 @@ class FeatureColumns(object):
gz_dir=None,
data_file=None,
**kwargs):
"""
build `Dataset` from `data_dir` or `data_file`
if `use_gz`, will try to convert data_files to gz format and save to `gz_dir`, if `gz_dir` not given, will create one.
"""
if use_gz:
gz_dir = self._make_gz_dataset(data_dir, gz_dir)
gz_files = self.gz_files(gz_dir)
gz_files = _list_files(gz_dir) if gz_dir is not None else gz_dir
ds = self._read_gz_dataset(gz_files, **kwargs)
else:
if data_dir is not None:
data_files = self.raw_files(data_dir)
data_files = _list_files(data_dir)
elif data_file is not None:
data_files = [data_file]
else:
......@@ -362,6 +398,7 @@ class FeatureColumns(object):
return ds
def build_dataset_from_stdin(self, name, **kwargs):
"""doc"""
ds = self._read_stdin_dataset(**kwargs)
ds.name = name
return ds
......
propeller/paddle/data/functional.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Pyreader based Dataset"""
import sys
import numpy as np
......@@ -25,7 +26,10 @@ log = logging.getLogger(__name__)
class Dataset(DatasetBase):
"""Pyreader based Dataset"""
def placeholders(self):
"""doc"""
if self.name is None:
raise ValueError('can not get feature from unnamed Dataset')
......@@ -41,7 +45,7 @@ class Dataset(DatasetBase):
return ret
def features(self):
'''start point of net building. call this in a program scope'''
"""start point of net building. call this in a program scope"""
if self.name is None:
raise ValueError('can not get feature from unnamed Dataset')
......@@ -51,9 +55,13 @@ class Dataset(DatasetBase):
(repr(self._data_shapes), repr(self._data_types)))
return self.placeholders()
def start(self, places=F.cuda_places()):
def start(self, places=None):
"""start Pyreader"""
if places is None:
places = F.cuda_places() if F.core.is_compiled_with_cuda(
) else F.cpu_places()
#assert self.pyreader is not None, 'use Dataset.features to build net first, then start dataset'
def gen():
def _gen():
try:
for idx, i in enumerate(self.generator()):
yield i
......@@ -63,5 +71,5 @@ class Dataset(DatasetBase):
r = F.io.PyReader(
feed_list=self.placeholders(), capacity=50, iterable=True)
r.decorate_batch_generator(gen, places=places)
r.decorate_batch_generator(_gen, places=places)
return r()
propeller/paddle/service/__init__.py 已删除 100644 → 0
浏览文件 @ b6697bf0
# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
propeller/paddle/service/server.py 已删除 100644 → 0
浏览文件 @ b6697bf0
# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import division
from __future__ import absolute_import
from __future__ import print_function
from __future__ import unicode_literals
import logging
import six
import asyncio
import threading
import grpc
from propeller.service import interface_pb2
from propeller.service import interface_pb2_grpc
import propeller.paddle.service.utils as serv_utils
from concurrent.futures import ThreadPoolExecutor
import paddle.fluid as F
from time import sleep, time
log = logging.getLogger(__name__)
def profile(msg):
def decfn(fn):
def retfn(*args, **kwargs):
start = time()
ret = fn(*args, **kwargs)
end = time()
log.debug('%s timecost: %.5f' % (msg, end - start))
return ret
return retfn
return decfn
def serve(model_dir, host, num_concurrent=None):
if six.PY2:
raise RuntimeError('propeller service work in python3 only')
num_worker = len(F.cuda_places(
)) if num_concurrent is None else num_concurrent
pool = ThreadPoolExecutor(num_worker)
class Predictor(object):
def __init__(self, did):
log.debug('create predictor on card %d' % did)
config = F.core.AnalysisConfig(model_dir)
config.enable_use_gpu(5000, did)
self._predictor = F.core.create_paddle_predictor(config)
@profile('paddle')
def __call__(self, args):
for i, a in enumerate(args):
a.name = 'placeholder_%d' % i
res = self._predictor.run(args)
return res
predictor_context = {}
class InferenceService(interface_pb2_grpc.InferenceServicer):
@profile('service')
def Infer(self, request, context):
try:
slots = request.slots
current_thread = threading.current_thread()
log.debug('%d slots received dispatch to thread %s' %
(len(slots), current_thread))
if current_thread not in predictor_context:
did = list(pool._threads).index(current_thread)
log.debug('spawning worker thread %d' % did)
predictor = Predictor(did)
predictor_context[current_thread] = predictor
else:
predictor = predictor_context[current_thread]
slots = [serv_utils.slot_to_paddlearray(s) for s in slots]
ret = predictor(slots)
response = [serv_utils.paddlearray_to_slot(r) for r in ret]
except Exception as e:
log.exception(e)
raise e
return interface_pb2.Slots(slots=response)
server = grpc.server(pool)
interface_pb2_grpc.add_InferenceServicer_to_server(InferenceService(),
server)
server.add_insecure_port(host)
server.start()
log.info('server started on %s...' % host)
try:
while True:
sleep(100000)
except KeyboardInterrupt as e:
pass
log.info('server stoped...')
if __name__ == '__main__':
from propeller import log
log.setLevel(logging.DEBUG)
serve(
'/home/work/chenxuyi/playground/grpc_play/ernie2.0/',
'10.255.138.19:8334',
num_concurrent=3)
propeller/paddle/service/utils.py 已删除 100644 → 0
浏览文件 @ b6697bf0
# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import division
from __future__ import absolute_import
from __future__ import print_function
from __future__ import unicode_literals
import struct
from propeller.service import interface_pb2
from propeller.service import interface_pb2_grpc
import paddle.fluid.core as core
def slot_to_paddlearray(slot):
if slot.type == interface_pb2.Slot.FP32:
type_str = 'f'
dtype = core.PaddleDType.FLOAT32
elif slot.type == interface_pb2.Slot.INT32:
type_str = 'i'
dtype = core.PaddleDType.INT32
elif slot.type == interface_pb2.Slot.INT64:
type_str = 'q'
dtype = core.PaddleDType.INT64
else:
raise RuntimeError('know type %s' % slot.type)
ret = core.PaddleTensor()
ret.shape = slot.dims
ret.dtype = dtype
num = len(slot.data) // struct.calcsize(type_str)
arr = struct.unpack('%d%s' % (num, type_str), slot.data)
ret.data = core.PaddleBuf(arr)
return ret
def paddlearray_to_slot(arr):
if arr.dtype == core.PaddleDType.FLOAT32:
dtype = interface_pb2.Slot.FP32
type_str = 'f'
arr_data = arr.data.float_data()
elif arr.dtype == core.PaddleDType.INT32:
dtype = interface_pb2.Slot.INT32
type_str = 'i'
arr_data = arr.data.int32_data()
elif arr.dtype == core.PaddleDType.INT64:
dtype = interface_pb2.Slot.INT64
type_str = 'q'
arr_data = arr.data.int64_data()
else:
raise RuntimeError('know type %s' % arr.dtype)
data = struct.pack('%d%s' % (len(arr_data), type_str), *arr_data)
pb = interface_pb2.Slot(type=dtype, dims=list(arr.shape), data=data)
return pb
propeller/paddle/summary.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""record summary tensor in a collection scope"""
from __future__ import print_function
from __future__ import absolute_import
......@@ -23,6 +24,7 @@ from propeller.paddle.collection import default_collection, Key
def scalar(name, tensor):
"""scalar summary"""
if not isinstance(tensor, F.framework.Variable):
raise ValueError('expect paddle Variable, got %s' % repr(tensor))
tensor.persistable = True
......@@ -30,6 +32,7 @@ def scalar(name, tensor):
def histogram(name, tensor):
"""histogram summary"""
if not isinstance(tensor, F.framework.Variable):
raise ValueError('expect paddle Variable, got %s' % repr(tensor))
tensor.persistable = True
......
propeller/paddle/train/__init__.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Propeller training"""
from __future__ import print_function
from __future__ import absolute_import
......
propeller/paddle/train/distribution.py
浏览文件 @ 4f1537e2
......@@ -128,6 +128,7 @@ def init_distribuition_env(program):
elif status.mode == DistributionMode.NCCL:
config = F.DistributeTranspilerConfig()
config.mode = "nccl2"
config.nccl_comm_num = 1
F.DistributeTranspiler(config=config).transpile(
status.replica_id,
trainers=','.join(status._env),
......
propeller/paddle/train/exporter.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,9 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
exporters
"""
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals
......@@ -19,6 +22,7 @@ import sys
import os
import itertools
import six
import inspect
import abc
import logging
......@@ -28,24 +32,36 @@ import paddle.fluid.layers as L
from propeller.paddle.train import Saver
from propeller.types import InferenceSpec
from propeller.train.model import Model
from propeller.paddle.train.trainer import _build_net
from propeller.paddle.train.trainer import _build_model_fn
from propeller.types import RunMode
from propeller.types import ProgramPair
log = logging.getLogger(__name__)
@six.add_metaclass(abc.ABCMeta)
class Exporter():
class Exporter(object):
"""base exporter"""
@abc.abstractmethod
def export(self, exe, program, eval_result, state):
"""export"""
raise NotImplementedError()
class BestExporter(Exporter):
"""export saved model accordingto `cmp_fn`"""
def __init__(self, export_dir, cmp_fn):
"""doc"""
self._export_dir = export_dir
self._best = None
self.cmp_fn = cmp_fn
def export(self, exe, program, eval_model_spec, eval_result, state):
"""doc"""
log.debug('New evaluate result: %s \nold: %s' %
(repr(eval_result), repr(self._best)))
if self._best is None or self.cmp_fn(old=self._best, new=eval_result):
......@@ -65,40 +81,85 @@ class BestExporter(Exporter):
class BestInferenceModelExporter(Exporter):
def __init__(self, export_dir, cmp_fn):
"""export inference model accordingto `cmp_fn`"""
def __init__(self,
export_dir,
cmp_fn,
model_class_or_model_fn=None,
hparams=None,
dataset=None):
"""doc"""
self._export_dir = export_dir
self._best = None
self.cmp_fn = cmp_fn
self.model_class_or_model_fn = model_class_or_model_fn
self.hparams = hparams
self.dataset = dataset
def export(self, exe, program, eval_model_spec, eval_result, state):
"""doc"""
if self.model_class_or_model_fn is not None and self.hparams is not None \
and self.dataset is not None:
log.info('Building program by user defined model function')
if issubclass(self.model_class_or_model_fn, Model):
_model_fn = _build_model_fn(self.model_class_or_model_fn)
elif inspect.isfunction(self.model_class_or_model_fn):
_model_fn = self.model_class_or_model_fn
else:
raise ValueError('unknown model %s' %
self.model_class_or_model_fn)
# build net
infer_program = F.Program()
startup_prog = F.Program()
with F.program_guard(infer_program, startup_prog):
#share var with Train net
with F.unique_name.guard():
log.info('Building Infer Graph')
infer_fea = self.dataset.features()
# run_config is None
self.model_spec = _build_net(_model_fn, infer_fea,
RunMode.PREDICT, self.hparams,
None)
log.info('Done')
infer_program = infer_program.clone(for_test=True)
self.program = ProgramPair(
train_program=infer_program, startup_program=startup_prog)
else:
self.program = program
self.model_spec = eval_model_spec
log.debug('New evaluate result: %s \nold: %s' %
(repr(eval_result), repr(self._best)))
if self._best is None or self.cmp_fn(old=self._best, new=eval_result):
log.debug('[Best Exporter]: export to %s' % self._export_dir)
if eval_model_spec.inference_spec is None:
if self.model_spec.inference_spec is None:
raise ValueError('model_fn didnt return InferenceSpec')
inf_sepc_dict = eval_model_spec.inference_spec
if not isinstance(inf_sepc_dict, dict):
inf_sepc_dict = {'inference': inf_sepc_dict}
for inf_sepc_name, inf_sepc in six.iteritems(inf_sepc_dict):
if not isinstance(inf_sepc, InferenceSpec):
raise ValueError('unkonw inference spec type: %s' % v)
inf_spec_dict = self.model_spec.inference_spec
if not isinstance(inf_spec_dict, dict):
inf_spec_dict = {'inference': inf_spec_dict}
for inf_spec_name, inf_spec in six.iteritems(inf_spec_dict):
if not isinstance(inf_spec, InferenceSpec):
raise ValueError('unknow inference spec type: %s' %
inf_spec)
save_dir = os.path.join(self._export_dir, inf_sepc_name)
save_dir = os.path.join(self._export_dir, inf_spec_name)
log.debug('[Best Exporter]: save inference model: "%s" to %s' %
(inf_sepc_name, save_dir))
feed_var = [i.name for i in inf_sepc.inputs]
fetch_var = inf_sepc.outputs
(inf_spec_name, save_dir))
feed_var = [i.name for i in inf_spec.inputs]
fetch_var = inf_spec.outputs
eval_program = program.train_program
infer_program = self.program.train_program
startup_prog = F.Program()
F.io.save_inference_model(
save_dir,
feed_var,
fetch_var,
exe,
main_program=eval_program)
main_program=infer_program)
self._best = eval_result
else:
log.debug('[Best Exporter]: skip step %s' % state.gstep)
propeller/paddle/train/hooks.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""train hooks"""
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals
......@@ -38,44 +39,56 @@ log = logging.getLogger(__name__)
class RunHook(object):
"""RunHook Base class"""
def __init__(self):
"""doc"""
pass
def before_train(self):
def before_train(self, program):
"""doc"""
pass
def before_run(self, state):
"""doc"""
return []
def after_run(self, res_list, state):
"""doc"""
pass
def should_stop(self, state):
"""doc"""
return False
def after_train(self):
"""doc"""
pass
class TqdmProgressBarHook(RunHook):
"""show a progress bar when training"""
def __init__(self, max_steps, desc=None):
"""doc"""
self.tqdm = None
import tqdm
from propeller import log as main_log
hdl = main_log.handlers[0]
class TqdmLogginHandler(logging.Handler):
class _TqdmLogginHandler(logging.Handler):
def emit(self, record):
"""doc"""
try:
msg = self.format(record)
tqdm.tqdm.write(msg, file=sys.stderr)
self.flush()
except (KeyboardInterrupt, SystemExit):
raise
except (KeyboardInterrupt, SystemExit) as e:
raise e
except:
self.handleError(record)
tqdm_hdl = TqdmLogginHandler()
tqdm_hdl = _TqdmLogginHandler()
tqdm_hdl.setFormatter(hdl.formatter)
main_log.removeHandler(hdl)
main_log.addHandler(tqdm_hdl)
......@@ -91,46 +104,55 @@ class TqdmProgressBarHook(RunHook):
class TqdmNotebookProgressBarHook(RunHook):
"""show a progress bar when training"""
def __init__(self, max_steps, desc=None):
"""doc"""
self.tqdm = None
import tqdm
from propeller import log as main_log
hdl = main_log.handlers[0]
class TqdmLogginHandler(logging.Handler):
class _TqdmLogginHandler(logging.Handler):
def emit(self, record):
"""doc"""
try:
msg = self.format(record)
tqdm.tqdm.write(msg, file=sys.stderr)
self.flush()
except (KeyboardInterrupt, SystemExit):
raise
except (KeyboardInterrupt, SystemExit) as e:
raise e
except:
self.handleError(record)
tqdm_hdl = TqdmLogginHandler()
tqdm_hdl = _TqdmLogginHandler()
tqdm_hdl.setFormatter(hdl.formatter)
main_log.removeHandler(hdl)
main_log.addHandler(tqdm_hdl)
self.tqdm = tqdm.tqdm_notebook(total=max_steps, desc=None)
def before_run(self, state):
"""doc"""
self.tqdm.n = state.gstep
self.tqdm.refresh()
return []
def __del__(self):
"""doc"""
if self.tqdm:
self.tqdm.close()
class LoggingHook(RunHook):
"""log tensor in to screan and tensorboard"""
def __init__(self,
loss,
per_step=10,
skip_step=100,
summary_writer=None,
summary_record=None):
"""doc"""
if per_step is None or skip_step is None:
raise ValueError('wrong step argument, per step: %d skip_step %d' %
(per_step, skip_step))
......@@ -141,7 +163,8 @@ class LoggingHook(RunHook):
self.writer = summary_writer
self.last_state = None
def before_train(self):
def before_train(self, program):
"""doc"""
if self.summary_record:
if self.summary_record.scalar:
self.s_name, self.s_tolog = zip(*self.summary_record.scalar)
......@@ -154,6 +177,7 @@ class LoggingHook(RunHook):
self.h_name, self.h_tolog = [], []
def before_run(self, state):
"""doc"""
if state.gstep % self.per_step == 0 and state.step > self.skip_step:
ret = [self.loss]
if self.summary_record:
......@@ -164,6 +188,7 @@ class LoggingHook(RunHook):
return []
def after_run(self, res_list, state):
"""doc"""
if state.gstep % self.per_step == 0 and state.step > self.skip_step:
if not self.summary_record:
return
......@@ -209,11 +234,15 @@ class LoggingHook(RunHook):
class StopAtStepHook(RunHook):
"""stop training at some step"""
def __init__(self, stop_global_step, stop_step):
"""doc"""
self._stop_gstep = stop_global_step
self._stop_step = stop_step
def should_stop(self, state):
"""doc"""
if (self._stop_gstep and state.gstep >= self._stop_gstep) or \
(self._stop_step and state.step >= self._stop_step):
log.info('StopAtStepHook called stop')
......@@ -226,6 +255,7 @@ class EvalHook(RunHook):
"""hook this on a eval Executor"""
def __init__(self, metrics, summary_writer=None):
"""doc"""
self.writer = summary_writer
self._result = None
......@@ -244,11 +274,13 @@ class EvalHook(RunHook):
else:
self.names, self.metrics = [], []
def before_train(self):
def before_train(self, program):
"""doc"""
for m in self.metrics:
m.reset()
def before_run(self, state):
"""doc"""
ls = [m.tensor for m in self.metrics]
for i in ls:
if not (isinstance(i, list) or isinstance(i, tuple)):
......@@ -265,15 +297,18 @@ class EvalHook(RunHook):
return ls_flt
def after_run(self, res_list, state):
"""doc"""
res = util.unflatten(res_list, self.schema)
for r, m in zip(res, self.metrics):
m.update(r)
@property
def result(self):
"""doc"""
return self._result
def after_train(self):
"""doc"""
printable = []
self._result = {}
for n, m in zip(self.names, self.metrics):
......@@ -284,12 +319,16 @@ class EvalHook(RunHook):
class CheckpointSaverHook(RunHook):
"""Save checkpoint every n step"""
def __init__(self, saver, per_step=10, skip_step=100):
"""doc"""
self.saver = saver
self.per_step = per_step
self.skip_step = skip_step
def after_run(self, res_list, state):
"""doc"""
if state.gstep % self.per_step == 0 and \
state.step > self.skip_step:
self.saver.save(state)
propeller/paddle/train/metrics.py
浏览文件 @ 4f1537e2
......@@ -11,9 +11,12 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""predefined metrics"""
import sys
import os
import six
import numpy as np
import itertools
import logging
......@@ -31,98 +34,132 @@ __all__ = [
class Metrics(object):
"""Metrics base class"""
def __init__(self):
"""doc"""
self.saver = []
@property
def tensor(self):
"""doc"""
pass
def update(self, *args):
"""doc"""
pass
def eval(self):
"""doc"""
pass
class Mean(Metrics):
"""doc"""
def __init__(self, t):
"""doc"""
self.t = t
self.reset()
def reset(self):
"""doc"""
self.saver = np.array([])
@property
def tensor(self):
"""doc"""
self.t.persistable = True
return self.t,
def update(self, args):
"""doc"""
t, = args
t = t.reshape([-1])
self.saver = np.concatenate([self.saver, t])
def eval(self):
"""doc"""
return self.saver.mean()
class Ppl(Mean):
"""doc"""
def eval(self):
"""doc"""
return np.exp(self.saver.mean())
class Acc(Mean):
"""doc"""
def __init__(self, label, pred):
"""doc"""
self.eq = L.equal(pred, label)
self.reset()
@property
def tensor(self):
"""doc"""
self.eq.persistable = True
return self.eq,
class MSE(Mean):
"""doc"""
def __init__(self, label, pred):
"""doc"""
diff = pred - label
self.mse = diff * diff
self.reset()
@property
def tensor(self):
"""doc"""
self.mse.persistable = True
return self.mse,
class Cosine(Mean):
"""doc"""
def __init__(self, label, pred):
"""doc"""
self.cos = L.cos_sim(label, pred)
self.reset()
@property
def tensor(self):
"""doc"""
self.cos.persistable = True
return self.cos,
class Precision(Metrics):
"""doc"""
def __init__(self, label, pred):
"""doc"""
self.label = label
self.pred = pred
self.reset()
def reset(self):
"""doc"""
self.label_saver = np.array([], dtype=np.bool)
self.pred_saver = np.array([], dtype=np.bool)
@property
def tensor(self):
"""doc"""
self.label.persistable = True
self.pred.persistable = True
return self.label, self.pred
def update(self, args):
"""doc"""
label, pred = args
label = label.reshape([-1]).astype(np.bool)
pred = pred.reshape([-1]).astype(np.bool)
......@@ -134,20 +171,27 @@ class Precision(Metrics):
self.pred_saver = np.concatenate([self.pred_saver, pred])
def eval(self):
"""doc"""
tp = (self.label_saver & self.pred_saver).astype(np.int64).sum()
t = self.label_saver.astype(np.int64).sum()
return tp / t
class Recall(Precision):
"""doc"""
def eval(self):
"""doc"""
tp = (self.label_saver & self.pred_saver).astype(np.int64).sum()
p = (self.label_saver).astype(np.int64).sum()
return tp / p
class F1(Precision):
"""doc"""
def eval(self):
"""doc"""
tp = (self.label_saver & self.pred_saver).astype(np.int64).sum()
t = self.label_saver.astype(np.int64).sum()
p = self.pred_saver.astype(np.int64).sum()
......@@ -157,22 +201,28 @@ class F1(Precision):
class Auc(Metrics):
"""doc"""
def __init__(self, label, pred):
"""doc"""
self.pred = pred
self.label = label
self.reset()
def reset(self):
"""doc"""
self.pred_saver = np.array([], dtype=np.float32)
self.label_saver = np.array([], dtype=np.bool)
@property
def tensor(self):
"""doc"""
self.pred.persistable = True
self.label.persistable = True
return [self.pred, self.label]
def update(self, args):
"""doc"""
pred, label = args
pred = pred.reshape([-1]).astype(np.float32)
label = label.reshape([-1]).astype(np.bool)
......@@ -180,6 +230,7 @@ class Auc(Metrics):
self.label_saver = np.concatenate([self.label_saver, label])
def eval(self):
"""doc"""
fpr, tpr, thresholds = sklearn.metrics.roc_curve(
self.label_saver.astype(np.int64), self.pred_saver)
auc = sklearn.metrics.auc(fpr, tpr)
......@@ -187,11 +238,15 @@ class Auc(Metrics):
class RecallAtPrecision(Auc):
"""doc"""
def __init__(self, label, pred, precision=0.9):
"""doc"""
super(RecallAtPrecision, self).__init__(label, pred)
self.precision = precision
def eval(self):
"""doc"""
self.pred_saver = self.pred_saver.reshape(
[self.label_saver.size, -1])[:, -1]
precision, recall, thresholds = sklearn.metrics.precision_recall_curve(
......@@ -202,11 +257,15 @@ class RecallAtPrecision(Auc):
class PrecisionAtThreshold(Auc):
"""doc"""
def __init__(self, label, pred, threshold=0.5):
"""doc"""
super().__init__(label, pred)
self.threshold = threshold
def eval(self):
"""doc"""
infered = self.pred_saver > self.threshold
correct_num = np.array(infered & self.label_saver).sum()
infer_num = infered.sum()
......@@ -214,25 +273,31 @@ class PrecisionAtThreshold(Auc):
class Mrr(Metrics):
"""doc"""
def __init__(self, qid, label, pred):
"""doc"""
self.qid = qid
self.label = label
self.pred = pred
self.reset()
def reset(self):
"""doc"""
self.qid_saver = np.array([], dtype=np.int64)
self.label_saver = np.array([], dtype=np.int64)
self.pred_saver = np.array([], dtype=np.float32)
@property
def tensor(self):
"""doc"""
self.qid.persistable = True
self.label.persistable = True
self.pred.persistable = True
return [self.qid, self.label, self.pred]
def update(self, args):
"""doc"""
qid, label, pred = args
if not (qid.shape[0] == label.shape[0] == pred.shape[0]):
raise ValueError(
......@@ -246,10 +311,12 @@ class Mrr(Metrics):
[self.pred_saver, pred.reshape([-1]).astype(np.float32)])
def eval(self):
def key_func(tup):
"""doc"""
def _key_func(tup):
return tup[0]
def calc_func(tup):
def _calc_func(tup):
ranks = [
1. / (rank + 1.)
for rank, (_, l, p) in enumerate(
......@@ -262,19 +329,22 @@ class Mrr(Metrics):
return 0.
mrr_for_qid = [
calc_func(tup)
_calc_func(tup)
for _, tup in itertools.groupby(
sorted(
zip(self.qid_saver, self.label_saver, self.pred_saver),
key=key_func),
key=key_func)
key=_key_func),
key=_key_func)
]
mrr = np.float32(sum(mrr_for_qid) / len(mrr_for_qid))
return mrr
class ChunkF1(Metrics):
"""doc"""
def __init__(self, label, pred, seqlen, num_label):
"""doc"""
self.label = label
self.pred = pred
self.seqlen = seqlen
......@@ -327,18 +397,21 @@ class ChunkF1(Metrics):
return chunks
def reset(self):
"""doc"""
self.label_cnt = 0
self.pred_cnt = 0
self.correct_cnt = 0
@property
def tensor(self):
"""doc"""
self.pred.persistable = True
self.label.persistable = True
self.seqlen.persistable = True
return [self.pred, self.label, self.seqlen]
def update(self, args):
"""doc"""
pred, label, seqlen = args
pred = pred.reshape([-1]).astype(np.int32).tolist()
label = label.reshape([-1]).astype(np.int32).tolist()
......@@ -374,6 +447,7 @@ class ChunkF1(Metrics):
label_index += 1
def eval(self):
"""doc"""
if self.pred_cnt == 0:
precision = 0.0
else:
......@@ -393,23 +467,29 @@ class ChunkF1(Metrics):
class PNRatio(Metrics):
"""doc"""
def __init__(self, qid, label, pred):
"""doc"""
self.qid = qid
self.label = label
self.pred = pred
self.saver = {}
def reset(self):
"""doc"""
self.saver = {}
@property
def tensor(self):
"""doc"""
self.qid.persistable = True
self.label.persistable = True
self.pred.persistable = True
return [self.qid, self.label, self.pred]
def update(self, args):
"""doc"""
qid, label, pred = args
if not (qid.shape[0] == label.shape[0] == pred.shape[0]):
raise ValueError('dimention not match: qid[%s] label[%s], pred[%s]'
......@@ -424,6 +504,7 @@ class PNRatio(Metrics):
self.saver[q].append((l, p))
def eval(self):
"""doc"""
p = 0
n = 0
for qid, outputs in self.saver.items():
......@@ -446,10 +527,14 @@ class PNRatio(Metrics):
class BinaryPNRatio(PNRatio):
"""doc"""
def __init__(self, qid, label, pred):
"""doc"""
super(BinaryPNRatio, self).__init__(qid, label, pred)
def eval(self):
"""doc"""
p = 0
n = 0
for qid, outputs in self.saver.items():
......@@ -474,7 +559,10 @@ class BinaryPNRatio(PNRatio):
class PrecisionAtK(Metrics):
"""doc"""
def __init__(self, qid, label, pred, k=1):
"""doc"""
self.qid = qid
self.label = label
self.pred = pred
......@@ -482,16 +570,19 @@ class PrecisionAtK(Metrics):
self.saver = {}
def reset(self):
"""doc"""
self.saver = {}
@property
def tensor(self):
"""doc"""
self.qid.persistable = True
self.label.persistable = True
self.pred.persistable = True
return [self.qid, self.label, self.pred]
def update(self, args):
"""doc"""
qid, label, pred = args
if not (qid.shape[0] == label.shape[0] == pred.shape[0]):
raise ValueError('dimention not match: qid[%s] label[%s], pred[%s]'
......@@ -507,6 +598,7 @@ class PrecisionAtK(Metrics):
self.saver[q].append((l, p))
def eval(self):
"""doc"""
right = 0
total = 0
for v in self.saver.values():
......
propeller/paddle/train/monitored_executor.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,10 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
doc
"""
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals
......@@ -37,9 +41,17 @@ log = logging.getLogger(__name__)
__all__ = ['MonitoredExecutor', 'Saver']
def _get_one_place():
return F.cuda_places()[0] if F.core.is_compiled_with_cuda(
) else F.cpu_places()[0]
class RunState(object):
"""serializable Run state object"""
@classmethod
def from_str(cls, s):
"""doc"""
j = json.loads(s)
ret = RunState()
ret._gstep = j['global_step']
......@@ -48,29 +60,36 @@ class RunState(object):
return ret
def __init__(self):
"""doc"""
self._gstep = 0
self._step = 0
self._time = time()
@property
def gstep(self):
"""doc"""
return self._gstep
@property
def step(self):
"""doc"""
return self._step
@property
def time(self):
"""doc"""
return self._time
def __repr__(self):
"""doc"""
return repr({'global_step': self._gstep, 'time': self._time})
def serialize(self):
"""doc"""
return json.dumps({'global_step': self._gstep, 'time': self._time})
def next(self):
"""doc"""
ret = RunState()
ret._gstep = self._gstep + 1
ret._step = self._step + 1
......@@ -79,12 +98,15 @@ class RunState(object):
class Saver(object):
"""checkpoint saver and manager"""
def __init__(self,
save_dir,
exe,
program,
save_prefix='model',
max_ckpt_to_keep=None):
"""doc"""
if exe is not None:
assert isinstance(
exe, F.Executor
......@@ -108,9 +130,11 @@ class Saver(object):
@property
def last_ckpt(self):
"""doc"""
return self.ckpt_list[-1] if len(self.ckpt_list) else None
def save(self, state):
"""doc"""
save_name = '%s_%d' % (self._save_prefix, state.gstep)
save_dir = os.path.join(self._save_dir, save_name)
tmp_dir = os.path.join(self._save_dir, 'tmp')
......@@ -139,28 +163,26 @@ class Saver(object):
open(self.ckpt_info_path, 'w').write('\n'.join(self.ckpt_list))
def restore(self, ckpt=-1):
if not isinstance(ckpt, (int, ) + six.string_types):
raise ValueError('ckpt type not understood %s' % repr(ckpt))
"""doc"""
if isinstance(ckpt, int):
try:
ckpt = self.ckpt_list[ckpt]
path = os.path.join(self._save_dir, self.ckpt_list[ckpt])
except IndexError:
raise ValueError('invalid restore ckpt number %d' % ckpt)
if isinstance(ckpt, six.string_types):
try:
ckpt = self.ckpt_list.index(ckpt)
except ValueError:
raise ValueError('ckpt: %s not in ckpt list: %s' %
(ckpt, self.ckpt_list))
elif isinstance(ckpt, six.string_types):
if not os.path.exists(ckpt):
raise ValueError('ckpt: %s not found' % ckpt)
path = ckpt
else:
raise ValueError('ckpt type not understood %s' % repr(ckpt))
path = os.path.join(self._save_dir, self.ckpt_list[ckpt])
meta_file = os.path.join(path, 'meta')
if not os.path.exists(meta_file):
raise RuntimeError('meta not found in restore dir: %s' % path)
state = RunState.from_str(open(meta_file).read())
log.info('restore from ckpt %s, ckpt-status: %s' % (path, repr(state)))
def fn(v):
def _fn(v):
vpath = os.path.join(path, v.name)
if F.io.is_persistable(v):
if os.path.exists(vpath):
......@@ -171,12 +193,12 @@ class Saver(object):
return False
F.io.load_vars(
self._exe, path, main_program=self._program, predicate=fn)
self._exe, path, main_program=self._program, predicate=_fn)
return state
class MonitoredExecutor(object):
"""A wrapper handling the train loop"""
"""An Executor wrapper handling the train loop"""
def __init__(
self,
......@@ -209,13 +231,18 @@ class MonitoredExecutor(object):
@property
def state(self):
"""doc"""
return self._state
def init_or_restore_variables(self):
def init_or_restore_variables(self, ckpt=-1):
"""
init vars or restore vars from model_dir
call before train
"""
# The order of this 2 steps really matters
# 1. init train
F.Executor(F.cuda_places()[0]).run(self._program.startup_program)
F.Executor(_get_one_place()).run(self._program.startup_program)
# 2. restore param
if self._warm_start_setting is not None:
if not os.path.exists(self._warm_start_setting.from_dir):
......@@ -224,29 +251,34 @@ class MonitoredExecutor(object):
log.info("warm start from %s" % self._warm_start_setting.from_dir)
if self._warm_start_setting.predicate_fn is not None:
def fn(v):
def _fn(v):
ret = self._warm_start_setting.predicate_fn(v)
if ret:
log.info('warm start: %s' % v.name)
return ret
F.io.load_vars(
F.Executor(F.cuda_places()[0]),
F.Executor(_get_one_place()),
self._warm_start_setting.from_dir,
main_program=self._program.train_program,
predicate=fn)
predicate=_fn)
else:
raise NotImplementedError()
self._saver = Saver(
self._model_dir,
F.Executor(F.cuda_places()[0]),
F.Executor(_get_one_place()),
program=self._program.train_program,
max_ckpt_to_keep=self._max_ckpt)
if self._saver.last_ckpt is not None:
self._state = self._saver.restore()
def freeze(self):
self._state = self._saver.restore(ckpt)
def _freeze(self):
"""
call before enter train loop
convert program to compiled program
will do nothing if loss is None i.e. not in train mode
"""
if self._loss is None:
log.debug('will not freeze a program without loss')
return
......@@ -278,8 +310,16 @@ class MonitoredExecutor(object):
startup_program=self._program.startup_program)
def __enter__(self):
"""
prepapre before enter train loop
"""
if F.core.is_compiled_with_cuda():
log.info('propeller runs in CUDA mode')
else:
log.info('propeller runs in CPU mode')
log.debug('freezing program')
self.freeze()
self._freeze()
log.debug('done freezing')
log.info('********** Start Loop ************')
# TODO init
......@@ -287,10 +327,13 @@ class MonitoredExecutor(object):
self.result = None
for h in self._hooks:
log.debug('train loop has hook %s' % h)
h.before_train()
h.before_train(self._program)
return self
def run(self, fetch_list=[], *args, **kwargs):
"""
wrapper for Executor.run
"""
#log.debug('Executor running step %d' % self._state.gstep)
if self._hooks:
fetch_list = [fetch_list]
......@@ -306,11 +349,12 @@ class MonitoredExecutor(object):
]
#if len(set(fetch_list)) != len(fetch_list):
# log.error('strange shit happend when fetch list has idetity tensors %s' % fetch_list)
#log.debug(fetch_list)
res = self._exe.run(self._program.train_program,
fetch_list=fetch_list,
*args,
**kwargs)
res = [self.merge_result(r) for r in res]
res = [self._merge_result(r) for r in res]
#log.debug(res)
res = util.unflatten(res, schema)
......@@ -330,6 +374,9 @@ class MonitoredExecutor(object):
return ret
def __exit__(self, err_type, err_value, trace):
"""
clean up things and report hook result when exit train loop
"""
if (err_type is None) or isinstance(err_value, (
F.core.EOFException, StopException, KeyboardInterrupt)):
try:
......@@ -344,7 +391,10 @@ class MonitoredExecutor(object):
log.exception('error occur during loop %s: %s' %
(err_type, err_value))
def merge_result(self, ls):
def _merge_result(self, ls):
"""
merge results from multi gpu cards
"""
dev_count = len(self._program.train_program._places) if isinstance(
self._program.train_program, F.compiler.CompiledProgram) else 1
if dev_count == 1:
......
propeller/paddle/train/trainer.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""common ML train and eval procedure"""
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals
......@@ -28,7 +29,8 @@ from time import time
import paddle.fluid as F
import paddle.fluid.layers as L
from propeller.types import RunMode, StopException, SummaryRecord, StopException, ModelSpec, InferenceSpec, ProgramPair, RunConfig
from propeller.types import RunMode, StopException, SummaryRecord, StopException
from propeller.types import ModelSpec, InferenceSpec, ProgramPair, RunConfig
from propeller.paddle import summary, collection
from propeller.paddle.data.functional import Dataset
from propeller.paddle.train import distribution
......@@ -43,7 +45,7 @@ log = logging.getLogger(__name__)
__all__ = ['train_and_eval', 'Learner']
def get_summary_writer(path):
def _get_summary_writer(path):
summary_writer = None
try:
from tensorboardX import SummaryWriter
......@@ -54,7 +56,12 @@ def get_summary_writer(path):
return summary_writer
def log_eval_result(name, eval_result, swriter, state):
def _get_one_place():
return F.cuda_places()[0] if F.core.is_compiled_with_cuda(
) else F.cpu_places()[0]
def _log_eval_result(name, eval_result, swriter, state):
log.debug(eval_result)
printable = []
for n, val in six.iteritems(eval_result):
......@@ -71,7 +78,7 @@ def log_eval_result(name, eval_result, swriter, state):
log.info('******************************')
def build_net(model_fn, features, mode, params, run_config):
def _build_net(model_fn, features, mode, params, run_config):
model_spec = model_fn(
features=features, mode=mode, params=params, run_config=run_config)
......@@ -97,12 +104,14 @@ def build_net(model_fn, features, mode, params, run_config):
class Learner(object):
"""A Learner can train / eval / predict on a Dataset"""
def __init__(self,
model_class_or_model_fn,
run_config,
params=None,
warm_start_setting=None):
'''
"""
model_class_or_model_fn(callable|propeller.train.Model): `model_class_or_model_fn` be specified in 2 ways:
1. subclass of propeller.train.Model which implements:
1. \_\_init\_\_ (hyper_param, mode, run_config)
......@@ -121,58 +130,23 @@ class Learner(object):
params: any python object, will pass to your `model_fn` or `propeller.train.Model`
run_config (propeller.RunConfig): run_config.max_steps should not be None.
warm_start_setting (propeller.WarmStartSetting): Optional. warm start variable will overwrite model variable.
'''
"""
if run_config.model_dir is None:
raise ValueError('model_dir should specified in run_config')
if issubclass(model_class_or_model_fn, Model):
def model_fn(features, mode, params, run_config):
if mode != RunMode.PREDICT:
fea, label = features[:-1], features[-1]
else:
fea = features
model = model_class_or_model_fn(
params, mode, run_config=run_config)
pred = model.forward(fea)
if isinstance(pred, F.framework.Variable):
prediction = [pred]
else:
prediction = pred
if mode == RunMode.TRAIN:
loss = model.loss(pred, label)
model.backward(loss)
return ModelSpec(
loss=loss, predictions=prediction, mode=mode)
elif mode == RunMode.EVAL:
loss = model.loss(pred, label)
me = model.metrics(pred, label)
inf_spec = InferenceSpec(inputs=fea, outputs=prediction)
if 'loss' not in me:
me['loss'] = metrics.Mean(loss)
return ModelSpec(
loss=loss,
predictions=prediction,
metrics=me,
mode=mode,
inference_spec=inf_spec)
elif mode == RunMode.PREDICT:
return ModelSpec(predictions=prediction, mode=mode)
else:
raise RuntimeError('unknown run mode %s' % mode)
_model_fn = _build_model_fn(model_class_or_model_fn)
elif inspect.isfunction(model_class_or_model_fn):
model_fn = model_class_or_model_fn
_model_fn = model_class_or_model_fn
else:
raise ValueError('unknown model %s' % model_class_or_model_fn)
self.model_fn = model_fn
self.model_fn = _model_fn
self.params = params
self.run_config = run_config
self.warm_start_setting = warm_start_setting
def build_for_train(self, train_dataset):
def _build_for_train(self, train_dataset):
train_dataset.name = 'train'
train_program = F.Program()
startup_prog = F.Program()
......@@ -181,8 +155,8 @@ class Learner(object):
with collection.Collections() as collections:
log.info('Building Train Graph...')
fea = train_dataset.features()
model_spec = build_net(self.model_fn, fea, RunMode.TRAIN,
self.params, self.run_config)
model_spec = _build_net(self.model_fn, fea, RunMode.TRAIN,
self.params, self.run_config)
log.info('Building Train Graph: Done')
scalars = collections.get(collection.Key.SUMMARY_SCALAR)
......@@ -208,7 +182,7 @@ class Learner(object):
train_program=train_program,
startup_program=startup_prog), model_spec, summary_record
def build_for_eval(self, ds):
def _build_for_eval(self, ds):
ds.name = 'eval'
program = F.Program()
startup_prog = F.Program()
......@@ -217,8 +191,8 @@ class Learner(object):
with F.unique_name.guard():
log.info('Building Eval Graph')
fea = ds.features()
model_spec = build_net(self.model_fn, fea, RunMode.EVAL,
self.params, self.run_config)
model_spec = _build_net(self.model_fn, fea, RunMode.EVAL,
self.params, self.run_config)
log.info('Done')
program = program.clone(for_test=True)
log.info(
......@@ -227,7 +201,7 @@ class Learner(object):
return ProgramPair(
train_program=program, startup_program=startup_prog), model_spec
def build_for_predict(self, ds):
def _build_for_predict(self, ds):
ds.name = 'predict'
program = F.Program()
startup_prog = F.Program()
......@@ -236,8 +210,8 @@ class Learner(object):
with F.unique_name.guard():
log.info('Building Predict Graph')
fea = ds.features()
model_spec = build_net(self.model_fn, fea, RunMode.PREDICT,
self.params, self.run_config)
model_spec = _build_net(self.model_fn, fea, RunMode.PREDICT,
self.params, self.run_config)
log.info('Done')
program = program.clone(for_test=True)
......@@ -249,11 +223,12 @@ class Learner(object):
train_program=program, startup_program=startup_prog), model_spec
def train(self, train_ds, train_hooks=[]):
"""train on a `Dataset`"""
if not isinstance(train_ds, Dataset):
raise ValueError('expect dataset to be instance of Dataset, got %s'
% repr(train_ds))
train_program, model_spec, summary_record = self.build_for_train(
train_program, model_spec, summary_record = self._build_for_train(
train_ds)
train_run_hooks = [
hooks.StopAtStepHook(self.run_config.max_steps,
......@@ -261,13 +236,16 @@ class Learner(object):
hooks.LoggingHook(
model_spec.loss,
summary_record=summary_record,
summary_writer=get_summary_writer(
summary_writer=_get_summary_writer(
os.path.join(self.run_config.model_dir, 'train_history')),
per_step=self.run_config.log_steps,
skip_step=self.run_config.skip_steps),
]
if model_spec.train_hooks is not None:
train_run_hooks.extend(model_spec.train_hooks)
train_run_hooks.extend(train_hooks)
train_executor = F.Executor(F.cuda_places()[0])
train_executor = F.Executor(_get_one_place())
mon_exe = MonitoredExecutor(
train_executor,
......@@ -297,24 +275,29 @@ class Learner(object):
return mon_exe.result
def evaluate(self, eval_dataset, eval_hooks=[]):
"""eval on a `Dataset`"""
if not isinstance(eval_dataset, Dataset):
raise ValueError('expect dataset to be instance of Dataset, got %s'
% repr(eval_dataset))
program, model_spec = self.build_for_eval(eval_dataset)
single_card_place = F.cuda_places()[0]
program, model_spec = self._build_for_eval(eval_dataset)
single_card_place = _get_one_place()
eval_executor = F.Executor(single_card_place)
eval_hooks = [
eval_run_hooks = [
hooks.StopAtStepHook(self.run_config.eval_max_steps,
self.run_config.eval_max_steps),
hooks.EvalHook(model_spec.metrics, )
]
if model_spec.eval_hooks is not None:
eval_run_hooks.extend(model_spec.eval_hooks)
eval_run_hooks.extend(eval_hooks)
mon_exe = MonitoredExecutor(
eval_executor,
program,
run_config=self.run_config,
run_hooks=eval_hooks)
run_hooks=eval_run_hooks)
mon_exe.init_or_restore_variables()
try:
......@@ -326,32 +309,43 @@ class Learner(object):
_, eval_result = mon_exe.result
summary_writer = get_summary_writer(
summary_writer = _get_summary_writer(
os.path.join(self.run_config.model_dir, 'eval_history'))
log_eval_result('eval', eval_result, summary_writer, mon_exe.state)
_log_eval_result('eval', eval_result, summary_writer, mon_exe.state)
return mon_exe.result
def predict(self, predict_dataset, ckpt=None, steps=-1, split_batch=True):
'''
def predict(self,
predict_dataset,
ckpt=-1,
ckpt_path=None,
steps=-1,
split_batch=True):
"""
Perform predictoin
will call `model_fn` and initiate user-specifed model in `propeller.RunMode.PREDICT` mode
Args:
infer_dataset (propeller.data.Dataset): should not `shuffle` or `repeat`
steps (int): steps to predict, if -1 is specifed, will stop when `StopException` is raised in `infer_dataset`
steps (int): steps to predict, if None is specifed,
will stop when `StopException` is raised in `infer_dataset`
ckpt_path (None|str): Path of a specific checkpoint to predict.
If None, the latest checkpoint in model_dir is used.
If there are no checkpoints in model_dir,
prediction is run with newly initialized Variables instead of ones restored from checkpoint.
ckpt (int): deprecated args
split_batch (bool): if True, prediction of each example in a batch is returned.
Yields:
Evaluated values of predictions tensors.
'''
"""
if not isinstance(predict_dataset, Dataset):
raise ValueError('expect dataset to be instance of Dataset, got %s'
% repr(predict_dataset))
program, model_spec = self.build_for_predict(predict_dataset)
single_card_place = F.cuda_places()[0]
program, model_spec = self._build_for_predict(predict_dataset)
single_card_place = _get_one_place()
executor = F.Executor(single_card_place)
pred_run_config = RunConfig(
run_steps=steps if steps == -1 else None,
......@@ -360,11 +354,12 @@ class Learner(object):
executor,
program,
run_config=pred_run_config, )
mon_exe.init_or_restore_variables()
mon_exe.init_or_restore_variables(ckpt
if ckpt_path is None else ckpt_path)
try:
with mon_exe:
log.info('Runining predict from dir: %s' % repr(mon_exe.state))
single_card_place = F.cuda_places()[0]
single_card_place = _get_one_place()
for data in predict_dataset.start(places=[single_card_place]):
res = mon_exe.run(fetch_list=model_spec.predictions,
feed=data)
......@@ -379,7 +374,7 @@ class Learner(object):
pass
def train_and_eval(_shit=None,
def train_and_eval(_placeholder=None,
model_class_or_model_fn=None,
params=None,
run_config=None,
......@@ -389,36 +384,27 @@ def train_and_eval(_shit=None,
train_hooks=[],
eval_hooks=[],
exporters=[]):
'''
"""
Perform train and evaluate procesure.
will call `model_fn` and initiate user-specifed model in `propeller.RunMode.PREDICT` mode
Args:
model_class_or_model_fn(callable|propeller.train.Model): `model_class_or_model_fn` be specified in 2 ways:
1. subclass of propeller.train.Model which implements:
1. \_\_init\_\_ (hyper_param, mode, run_config)
2. forward (features) => (prediction)
3. backword (loss) => None
4. loss (predictoin) => (loss)
5. metrics (optional) (prediction) => (dict of propeller.Metrics)
2. a model_fn takes following args:
1. features
2. param
3. mode
4. run_config(optional)
1. subclass of propeller.train.Model
2. a model_fn takes following args: 1. features; 2. param; 3. mode; 4. run_config(optional)
and returns a `propeller.ModelSpec`
params: any python object, will pass to your `model_fn` or `propeller.train.Model`
run_config (propeller.RunConfig): run_config.max_steps should not be None.
train_dataset (propeller.paddle.data.Dataset): training will stop if global_step > run_config.max_steps.
eval_dataset (propeller.paddle.data.Dataset|dict): Optional, if Dict of propeller.data.Dataset were specified, will perform evluatation on every evaluation sets and report results.
eval_dataset (propeller.paddle.data.Dataset|dict): Optional, if Dict of propeller.data.Dataset were specified,
will perform evluatation on every evaluation sets and report results.
warm_start_setting (propeller.WarmStartSetting): Optional. warm start variable will overwrite model variable.
train_hooks (list of propeller.paddle.train.RunHook): Optional.
eval_hooks (list of propeller.paddle.train.RunHook): Optional.
exporters (list of propeller.paddle.train.Exporter): Optional.
'''
if _shit is not None:
"""
if _placeholder is not None:
raise ValueError('specify keyword args to this function')
if model_class_or_model_fn is None or params is None or run_config is None or train_dataset is None:
raise ValueError(
......@@ -454,13 +440,13 @@ def train_and_eval(_shit=None,
params,
warm_start_setting=warm_start_setting)
class EvalHookOnTrainLoop(hooks.RunHook):
class _EvalHookOnTrainLoop(hooks.RunHook):
def __init__(self):
self.program, self.model_spec = est.build_for_eval(
self.program, self.model_spec = est._build_for_eval(
list(eval_dataset.values())[
0]) #eval_datasets must have same output shapes
self.summary_writers = {
ds_name: get_summary_writer(
ds_name: _get_summary_writer(
os.path.join(
os.path.join(run_config.model_dir, 'eval_history'),
ds_name))
......@@ -468,6 +454,7 @@ def train_and_eval(_shit=None,
}
def after_run(self, _, state):
"""doc"""
if state.step > run_config.skip_steps and state.gstep % run_config.eval_steps == 0:
eval_results = {}
for name, ds in six.iteritems(eval_dataset):
......@@ -478,7 +465,7 @@ def train_and_eval(_shit=None,
self.model_spec.metrics,
summary_writer=self.summary_writers[name], )
]
single_card_place = F.cuda_places()[0]
single_card_place = _get_one_place()
eval_executor = F.Executor(single_card_place)
mon_exe = MonitoredExecutor(
eval_executor,
......@@ -495,8 +482,8 @@ def train_and_eval(_shit=None,
eval_res = hook_results[
1] # hook_results: [StopAtStepHook, EvalHook, ...]
eval_results[name] = eval_res
log_eval_result(name, eval_res, self.summary_writers[name],
state)
_log_eval_result(name, eval_res,
self.summary_writers[name], state)
for exporter in exporters:
exporter.export(eval_executor, self.program,
self.model_spec, eval_results, state)
......@@ -505,6 +492,46 @@ def train_and_eval(_shit=None,
return eval_results
if distribution.status.is_master:
train_hooks.append(EvalHookOnTrainLoop())
train_hooks.append(_EvalHookOnTrainLoop())
res = est.train(train_dataset, train_hooks=train_hooks)
return res
def _build_model_fn(model_class):
def _model_fn(features, mode, params, run_config):
if mode != RunMode.PREDICT:
fea, label = features[:-1], features[-1]
else:
fea = features
model = model_class(params, mode, run_config=run_config)
pred = model.forward(fea)
if isinstance(pred, F.framework.Variable):
prediction = [pred]
else:
prediction = pred
if mode == RunMode.TRAIN:
loss = model.loss(pred, label)
model.backward(loss)
return ModelSpec(loss=loss, predictions=prediction, mode=mode)
elif mode == RunMode.EVAL:
loss = model.loss(pred, label)
me = model.metrics(pred, label)
inf_spec = InferenceSpec(inputs=fea, outputs=prediction)
if 'loss' not in me:
me['loss'] = metrics.Mean(loss)
return ModelSpec(
loss=loss,
predictions=prediction,
metrics=me,
mode=mode,
inference_spec=inf_spec)
elif mode == RunMode.PREDICT:
inf_spec = InferenceSpec(inputs=fea, outputs=prediction)
return ModelSpec(
predictions=prediction, mode=mode, inference_spec=inf_spec)
else:
raise RuntimeError('unknown run mode %s' % mode)
return _model_fn
propeller/service/__init__.py
浏览文件 @ 4f1537e2
......@@ -11,3 +11,4 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""server"""
propeller/service/server.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,9 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
Never Never Never import paddle.fluid in main process, or any module would import fluid.
"""
from __future__ import division
from __future__ import absolute_import
......@@ -24,27 +27,27 @@ from time import sleep, time
import multiprocessing
import zmq
""" Never Never Never import paddle.fluid in main process, or any module would import fluid.
"""
log = logging.getLogger(__name__)
def profile(msg):
def decfn(fn):
def retfn(*args, **kwargs):
def _profile(msg):
def _decfn(fn):
def _retfn(*args, **kwargs):
start = time()
ret = fn(*args, **kwargs)
end = time()
log.debug('%s timecost: %.5f' % (msg, end - start))
return ret
return retfn
return _retfn
return decfn
return _decfn
class Predictor(object):
"""paddle predictor wrapper"""
def __init__(self, model_dir, device_idx=0):
import paddle.fluid as F
log.debug('create predictor on card %d' % device_idx)
......@@ -52,7 +55,7 @@ class Predictor(object):
config.enable_use_gpu(5000, device_idx)
self._predictor = F.core.create_paddle_predictor(config)
@profile('paddle')
@_profile('paddle')
def __call__(self, args):
for i, a in enumerate(args):
a.name = 'placeholder_%d' % i
......@@ -61,6 +64,7 @@ class Predictor(object):
def run_worker(model_dir, device_idx, endpoint="ipc://worker.ipc"):
"""worker process entrence"""
try:
log.debug("run_worker %s" % device_idx)
os.environ["CUDA_VISIBLE_DEVICES"] = os.getenv(
......@@ -97,6 +101,8 @@ def run_worker(model_dir, device_idx, endpoint="ipc://worker.ipc"):
class InferencePredictor(object):
"""control Predictor for multi gpu card"""
def __init__(self, backend_addr, model_dir, n_devices=1):
self.backend_addr = backend_addr
self.model_dir = model_dir
......@@ -104,6 +110,7 @@ class InferencePredictor(object):
self.children = []
def start(self):
"""doc"""
for device_idx in range(self.n_devices):
p = multiprocessing.Process(
target=run_worker,
......@@ -113,21 +120,27 @@ class InferencePredictor(object):
return self
def join(self):
"""doc"""
for p in self.children:
p.join()
def term(self):
"""doc"""
for p in self.children:
log.debug("terminating children %s" % repr(p))
p.terminate()
class InferenceProxy(object):
"""zmq proxy"""
def __init__(self):
"""doc"""
self.backend = None
self.frontend = None
def listen(self, frontend_addr, backend_addr):
"""doc"""
log.info("InferenceProxy starting...")
try:
context = zmq.Context(1)
......@@ -152,11 +165,15 @@ class InferenceProxy(object):
class InferenceServer(object):
"""start InferencePredictor and InferenceProxy"""
def __init__(self, model_dir, n_devices):
"""doc"""
self.model_dir = model_dir
self.n_devices = n_devices
def listen(self, port):
"""doc"""
frontend_addr = "tcp://*:%s" % port
backend_addr = "ipc://backend.ipc"
predictor = InferencePredictor(backend_addr, self.model_dir,
......
propeller/service/utils.py
浏览文件 @ 4f1537e2
......@@ -13,6 +13,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""utils for server"""
from __future__ import division
from __future__ import absolute_import
......@@ -26,6 +27,7 @@ from propeller.service import interface_pb2
def slot_to_numpy(slot):
"""doc"""
if slot.type == interface_pb2.Slot.FP32:
dtype = np.float32
type_str = 'f'
......@@ -45,6 +47,7 @@ def slot_to_numpy(slot):
def numpy_to_slot(arr):
"""doc"""
if arr.dtype == np.float32:
dtype = interface_pb2.Slot.FP32
elif arr.dtype == np.int32:
......@@ -59,28 +62,27 @@ def numpy_to_slot(arr):
def slot_to_paddlearray(slot):
"""doc"""
import paddle.fluid.core as core
if slot.type == interface_pb2.Slot.FP32:
dtype = np.float32
type_str = 'f'
dtype = core.PaddleDType.FLOAT32
elif slot.type == interface_pb2.Slot.INT32:
dtype = np.int32
type_str = 'i'
dtype = core.PaddleDType.INT32
elif slot.type == interface_pb2.Slot.INT64:
dtype = np.int64
type_str = 'q'
dtype = core.PaddleDType.INT64
else:
raise RuntimeError('know type %s' % slot.type)
ret = core.PaddleTensor()
ret.shape = slot.dims
ret.dtype = dtype
num = len(slot.data) // struct.calcsize(type_str)
arr = struct.unpack('%d%s' % (num, type_str), slot.data)
ret.data = core.PaddleBuf(arr)
ret = core.PaddleTensor(data=np.array(arr, dtype=dtype).reshape(slot.dims))
return ret
def paddlearray_to_slot(arr):
"""doc"""
import paddle.fluid.core as core
if arr.dtype == core.PaddleDType.FLOAT32:
dtype = interface_pb2.Slot.FP32
......@@ -102,12 +104,14 @@ def paddlearray_to_slot(arr):
def nparray_list_serialize(arr_list):
"""doc"""
slot_list = [numpy_to_slot(arr) for arr in arr_list]
slots = interface_pb2.Slots(slots=slot_list)
return slots.SerializeToString()
def nparray_list_deserialize(string):
"""doc"""
slots = interface_pb2.Slots()
slots.ParseFromString(string)
return [slot_to_numpy(slot) for slot in slots.slots]
propeller/tools/ckpt_inspector.py
浏览文件 @ 4f1537e2
......@@ -72,6 +72,10 @@ def parse(filename):
elif proto.data_type == framework_pb2.VarType.INT8:
arr = np.array(
gen_arr(f.read(), 'B'), dtype=np.int8).reshape(proto.dims)
elif proto.data_type == framework_pb2.VarType.FP16:
arr = np.array(
gen_arr(f.read(), 'H'),
dtype=np.uint16).view(np.float16).reshape(proto.dims)
else:
raise RuntimeError('Unknown dtype %s' % proto.data_type)
......
propeller/train/__init__.py
浏览文件 @ 4f1537e2
......@@ -11,3 +11,6 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
doc
"""
propeller/train/model.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,10 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
Model template
"""
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals
......@@ -26,7 +30,11 @@ import numpy as np
@six.add_metaclass(abc.ABCMeta)
class Model():
class Model(object):
"""
template
"""
def __init__(self, config, mode):
"""
Args:
......@@ -39,9 +47,9 @@ class Model():
def forward(self, features):
"""
Args:
features (list of Tensor): depends on your Dataset.output_shapes
features (list of Tensor): inputs features that depends on your Dataset.output_shapes
Returns:
return (Tensor):
return (Tensor): prediction
"""
pass
......@@ -53,8 +61,6 @@ class Model():
label (Tensor): depends on your Dataset.output_shapes
Returns:
return (paddle scalar): loss
"""
pass
......
propeller/types.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Basic types"""
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals
......@@ -21,12 +23,15 @@ from collections import namedtuple
class RunMode(object):
"""model_fn will be called in 3 modes"""
TRAIN = 1
PREDICT = 2
EVAL = 3
class HParams(object):
"""Hyper paramerter"""
def __init__(self, **kwargs):
for k, v in kwargs.items():
self.__dict__[k] = v
......@@ -45,30 +50,36 @@ class HParams(object):
def __setitem__(self, key, val):
self.__dict__[key] = val
@staticmethod
def from_json(self, json_str):
@classmethod
def from_json(cls, json_str):
"""doc"""
d = json.loads(json_str)
if type(d) != dict:
raise ValueError('json object must be dict.')
return HParams.from_dict(d)
def get(self, key, default=None):
"""doc"""
return self.__dict__.get(key, default)
@staticmethod
def from_dict(self, d):
@classmethod
def from_dict(cls, d):
"""doc"""
if type(d) != dict:
raise ValueError('input must be dict.')
hp = HParams(**d)
return hp
def to_json(self):
"""doc"""
return json.dumps(self.__dict__)
def to_dict(self):
"""doc"""
return self.__dict__
def join(self, other):
"""doc"""
if not isinstance(other, HParams):
raise ValueError('input must be HParams instance.')
self.__dict__.update(**other.__dict__)
......@@ -95,9 +106,12 @@ ModelSpec = namedtuple('ModelSpec', [
'metrics',
'mode',
'inference_spec',
'train_hooks',
'eval_hooks',
])
ModelSpec.__new__.__defaults__ = (None, ) * len(ModelSpec._fields)
class StopException(Exception):
"""doc"""
pass
propeller/util.py
浏览文件 @ 4f1537e2
......@@ -11,6 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""global utils"""
from __future__ import print_function
from __future__ import absolute_import
from __future__ import unicode_literals
......@@ -31,6 +32,7 @@ log = logging.getLogger(__name__)
def ArgumentParser(name):
"""predefined argparser"""
parser = argparse.ArgumentParser('propeller model')
parser.add_argument('--run_config', type=str, default='')
parser.add_argument(
......@@ -59,6 +61,7 @@ def _get_dict_from_environ_or_json_or_file(args, env_name):
def parse_file(filename):
"""useless api"""
d = _get_dict_from_environ_or_json_or_file(filename, None)
if d is None:
raise ValueError('file(%s) not found' % filename)
......@@ -66,6 +69,7 @@ def parse_file(filename):
def parse_runconfig(args=None):
"""get run_config from env or file"""
d = _get_dict_from_environ_or_json_or_file(args.run_config,
'PROPELLER_RUNCONFIG')
if d is None:
......@@ -74,6 +78,7 @@ def parse_runconfig(args=None):
def parse_hparam(args=None):
"""get hparam from env or file"""
if args is not None:
hparam_strs = reduce(list.__add__, args.hparam)
else:
......@@ -91,6 +96,7 @@ def parse_hparam(args=None):
def flatten(s):
"""doc"""
assert is_struture(s)
schema = [len(ss) for ss in s]
flt = list(itertools.chain(*s))
......@@ -98,6 +104,7 @@ def flatten(s):
def unflatten(structure, schema):
"""doc"""
start = 0
res = []
for _range in schema:
......@@ -107,10 +114,12 @@ def unflatten(structure, schema):
def is_struture(s):
"""doc"""
return isinstance(s, list) or isinstance(s, tuple)
def map_structure(func, s):
"""same sa tf.map_structure"""
if isinstance(s, list) or isinstance(s, tuple):
return [map_structure(func, ss) for ss in s]
elif isinstance(s, dict):
......
requirements.txt
浏览文件 @ 4f1537e2
......@@ -5,3 +5,5 @@ scikit-learn==0.20.3
scipy==1.2.1
six==1.11.0
sklearn==0.0
sentencepiece==0.1.8
paddlepaddle-gpu==1.6.3.post107
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或