Cherry pick PaddleNLP emotion_detection and ernie from release1.6 to develop (#3815)

* update PaddleNLP emotion_detection and ernie for Release/1.6 (#3608)

* emotion-detection => 1.6

* ERNIE => 1.6

* [PaddleNLP] update emotion_detection readme

* [PaddleNLP] emotion_detection add download.py (#3649)

* emotion-detection => 1.6

* ERNIE => 1.6

* [PaddleNLP] update emotion_detection readme

* [PaddleNLP] emotion_detection add download.py for windows user

* [PaddleNLP] fix emotion_detection open problem, add paddlehub version (#3706)

* update emotion-detection readme and fix open problem

* fix ernie

PaddleNLP/emotion_detection/README.md

@@ -25,7 +25,7 @@
 | BERT | 93.6% | 92.3%  | 78.6%  |
 | ERNIE | 94.4% | 94.0% | 80.6% |
 
-同时推荐用户参考[IPython Notebook demo](https://aistudio.baidu.com/aistudio/projectDetail/122291)
+同时推荐用户参考[IPython Notebook demo](https://aistudio.baidu.com/aistudio/projectDetail/122291)。
 
 ## 快速开始
 
@@ -33,7 +33,7 @@
 
 1. PaddlePaddle 安装
 
-   本项目依赖于 PaddlePaddle Fluid 1.3.2 及以上版本，请参考 [安装指南](http://www.paddlepaddle.org/#quick-start) 进行安装
+   本项目依赖于 PaddlePaddle Fluid 1.6 及以上版本，请参考 [安装指南](http://www.paddlepaddle.org/#quick-start) 进行安装。
 
 2. 代码安装
 
@@ -46,7 +46,7 @@
 
 3. 环境依赖
 
-   请参考 PaddlePaddle [安装说明](https://www.paddlepaddle.org.cn/documentation/docs/zh/1.5/beginners_guide/install/index_cn.html) 部分的内容
+   Python 2 的版本要求 2.7.15+，Python 3 的版本要求 3.5.1+/3.6/3.7，其它环境请参考 PaddlePaddle [安装说明](https://www.paddlepaddle.org.cn/documentation/docs/zh/1.5/beginners_guide/install/index_cn.html) 部分的内容。
 
 ### 代码结构说明
 
@@ -56,6 +56,7 @@
 .
 ├── config.json             # 配置文件
 ├── config.py               # 配置文件读取接口
+├── download.py             # 下载数据及预训练模型脚本
 ├── inference_model.py      # 保存 inference_model 的脚本
 ├── reader.py               # 数据读取接口
 ├── run_classifier.py       # 项目的主程序入口，包括训练、预测、评估
@@ -86,7 +87,7 @@ python tokenizer.py --test_data_dir ./test.txt.utf8 --batch_size 1 > test.txt.ut
 
 #### 公开数据集
 
-这里我们提供一份已标注的、经过分词预处理的机器人聊天数据集，只需运行数据下载脚本 ```sh download_data.sh```，运行成功后，会生成文件夹 ```data```，其目录结构如下：
+这里我们提供一份已标注的、经过分词预处理的机器人聊天数据集，只需运行数据下载脚本 ```sh download_data.sh```，或者 ```python download.py dataset``` 运行成功后，会生成文件夹 ```data```，其目录结构如下：
 
 ```text
 .
@@ -181,6 +182,8 @@ tar xvf emotion_detection_ernie_finetune-1.0.0.tar.gz
 
 ```shell
 sh download_model.sh
+# 或者
+python download.py model
 ```
 
 以上两种方式会将预训练的 TextCNN 模型和 ERNIE模型，保存在```pretrain_models```目录下，可直接修改```run.sh```脚本中的```init_checkpoint```参数进行评估、预测。
@@ -302,7 +305,7 @@ Final test result:
 
 我们也提供了使用 PaddleHub 加载 ERNIE 模型的选项，PaddleHub 是 PaddlePaddle 的预训练模型管理工具，可以一行代码完成预训练模型的加载，简化预训练模型的使用和迁移学习。更多相关的介绍，可以查看 [PaddleHub](https://github.com/PaddlePaddle/PaddleHub)
 
-注意：使用该选项需要先安装PaddleHub，安装命令如下
+注意：使用该选项需要先安装PaddleHub >= 1.2.0，安装命令如下
 ```shell
 pip install paddlehub
 ```
@@ -333,6 +336,8 @@ sh run_ernie.sh infer
 
 ## 版本更新
 
+2019/10/21 PaddlePaddle1.6适配，添加download.py脚本。
+
 2019/08/26 规范化配置的使用，对模块内数据处理代码进行了重构，更新README结构，提高易用性。
 
 2019/06/13 添加PaddleHub调用ERNIE方式。

PaddleNLP/emotion_detection/config.py

@@ -19,6 +19,7 @@ from __future__ import absolute_import
 from __future__ import division
 from __future__ import print_function
 
+import io
 import os
 import six
 import json
@@ -122,7 +123,7 @@ class PDConfig(object):
             return
 
         try:
-            with open(file_path, "r") as fin:
+            with io.open(file_path, "r") as fin:
                 self.json_config = json.load(fin)
         except Exception as e:
             raise IOError("Error in parsing json config file '%s'" % file_path)

PaddleNLP/emotion_detection/download.py

+# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Download script, download dataset and pretrain models.
+"""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import io
+import os
+import sys
+import time
+import hashlib
+import tarfile
+import requests
+
+
+def usage():
+    desc = ("\nDownload datasets and pretrained models for EmotionDetection task.\n"
+    "Usage:\n"
+    "   1. python download.py dataset\n"
+    "   2. python download.py model\n")
+    print(desc)
+
+
+def md5file(fname):
+    hash_md5 = hashlib.md5()
+    with io.open(fname, "rb") as fin:
+        for chunk in iter(lambda: fin.read(4096), b""):
+            hash_md5.update(chunk)
+    return hash_md5.hexdigest()
+
+
+def extract(fname, dir_path):
+    """
+    Extract tar.gz file
+    """
+    try:
+        tar = tarfile.open(fname, "r:gz")
+        file_names = tar.getnames()
+        for file_name in file_names:
+            tar.extract(file_name, dir_path)
+            print(file_name)
+        tar.close()
+    except Exception as e:
+        raise e
+
+
+def download(url, filename, md5sum):
+    """
+    Download file and check md5
+    """
+    retry = 0
+    retry_limit = 3
+    chunk_size = 4096
+    while not (os.path.exists(filename) and md5file(filename) == md5sum):
+        if retry < retry_limit:
+            retry += 1
+        else:
+            raise RuntimeError("Cannot download dataset ({0}) with retry {1} times.".
+                    format(url, retry_limit))
+        try:
+            start = time.time()
+            size = 0
+            res = requests.get(url, stream=True)
+            filesize = int(res.headers['content-length'])
+            if res.status_code == 200:
+                print("[Filesize]: %0.2f MB" % (filesize / 1024 / 1024))
+                # save by chunk
+                with io.open(filename, "wb") as fout:
+                    for chunk in res.iter_content(chunk_size=chunk_size):
+                        if chunk:
+                            fout.write(chunk)
+                            size += len(chunk)
+                            pr = '>' * int(size * 50 / filesize)
+                            print('\r[Process ]: %s%.2f%%' % (pr, float(size / filesize*100)), end='')
+            end = time.time()
+            print("\n[CostTime]: %.2f s" % (end - start))
+        except Exception as e:
+            print(e)
+
+
+def download_dataset(dir_path):
+    BASE_URL = "https://baidu-nlp.bj.bcebos.com/"
+    DATASET_NAME = "emotion_detection-dataset-1.0.0.tar.gz"
+    DATASET_MD5 = "512d256add5f9ebae2c101b74ab053e9"
+    file_path = os.path.join(dir_path, DATASET_NAME)
+    url = BASE_URL + DATASET_NAME
+
+    if not os.path.exists(dir_path):
+        os.makedirs(dir_path)
+    # download dataset
+    print("Downloading dataset: %s" % url)
+    download(url, file_path, DATASET_MD5)
+    # extract dataset
+    print("Extracting dataset: %s" % file_path)
+    extract(file_path, dir_path)
+    os.remove(file_path)
+
+
+def download_model(dir_path):
+    MODELS = {}
+    BASE_URL = "https://baidu-nlp.bj.bcebos.com/"
+    CNN_NAME = "emotion_detection_textcnn-1.0.0.tar.gz"
+    CNN_MD5 = "b7ee648fcd108835c880a5f5fce0d8ab"
+    ERNIE_NAME = "emotion_detection_ernie_finetune-1.0.0.tar.gz"
+    ERNIE_MD5 = "dfeb68ddbbc87f466d3bb93e7d11c03a"
+    MODELS[CNN_NAME] = CNN_MD5
+    MODELS[ERNIE_NAME] = ERNIE_MD5
+
+    if not os.path.exists(dir_path):
+        os.makedirs(dir_path)
+
+    for model in MODELS:
+        url = BASE_URL + model
+        model_path = os.path.join(dir_path, model)
+        print("Downloading model: %s" % url)
+        # download model
+        download(url, model_path, MODELS[model])
+        # extract model.tar.gz
+        print("Extracting model: %s" % model_path)
+        extract(model_path, dir_path)
+        os.remove(model_path)
+
+
+if __name__ == '__main__':
+    if len(sys.argv) != 2:
+        usage()
+        sys.exit(1)
+
+    if sys.argv[1] == "dataset":
+        pwd = os.path.join(os.path.dirname(__file__), './')
+        download_dataset(pwd)
+    elif sys.argv[1] == "model":
+        pwd = os.path.join(os.path.dirname(__file__), './pretrain_models')
+        download_model(pwd)
+    else:
+        usage()
+

PaddleNLP/emotion_detection/download_model.sh

 #!/bin/bash
 
-mkdir -p models
-cd models
+mkdir -p pretrain_models
+cd pretrain_models
 
 # download pretrain model file to ./models/
 MODEL_CNN=https://baidu-nlp.bj.bcebos.com/emotion_detection_textcnn-1.0.0.tar.gz

PaddleNLP/emotion_detection/inference_model.py

-# -*- encoding: utf8 -*-
+# -*- coding: UTF-8 -*-
 # Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
@@ -44,9 +44,8 @@ def do_save_inference_model(args):
 
     with fluid.program_guard(test_prog, startup_prog):
         with fluid.unique_name.guard():
-            infer_pyreader, probs, feed_target_names = create_model(
+            infer_loader, probs, feed_target_names = create_model(
                 args,
-                pyreader_name='infer_reader',
                 num_labels=args.num_labels,
                 is_prediction=True)
 
@@ -79,20 +78,7 @@ def test_inference_model(args, texts):
         dev_count = int(os.environ.get('CPU_NUM', 1))
         place = fluid.CPUPlace()
 
-    test_prog = fluid.default_main_program()
-    startup_prog = fluid.default_startup_program()
-
-    with fluid.program_guard(test_prog, startup_prog):
-        with fluid.unique_name.guard():
-            infer_pyreader, probs, feed_target_names = create_model(
-                args,
-                pyreader_name='infer_reader',
-                num_labels=args.num_labels,
-                is_prediction=True)
-
-    test_prog = test_prog.clone(for_test=True)
     exe = fluid.Executor(place)
-    exe.run(startup_prog)
 
     assert (args.inference_model_dir)
     infer_program, feed_names, fetch_targets = fluid.io.load_inference_model(
@@ -107,9 +93,8 @@ def test_inference_model(args, texts):
         wids, seq_len = utils.pad_wid(wids)
         data.append(wids)
         seq_lens.append(seq_len)
-    batch_size = len(data)
-    data = np.array(data).reshape((batch_size, 128, 1))
-    seq_lens = np.array(seq_lens).reshape((batch_size, 1))
+    data = np.array(data)
+    seq_lens = np.array(seq_lens)
 
     pred = exe.run(infer_program,
                 feed={

PaddleNLP/emotion_detection/reader.py

@@ -96,16 +96,16 @@ class EmoTectProcessor(object):
         Generate data for train, dev or test
         """
         if phase == "train":
-            return paddle.batch(
+            return fluid.io.batch(
                 self.get_train_examples(self.data_dir, epoch, self.max_seq_len), batch_size)
         elif phase == "dev":
-            return paddle.batch(
+            return fluid.io.batch(
                 self.get_dev_examples(self.data_dir, epoch, self.max_seq_len), batch_size)
         elif phase == "test":
-            return paddle.batch(
+            return fluid.io.batch(
                 self.get_test_examples(self.data_dir, epoch, self.max_seq_len), batch_size)
         elif phase == "infer":
-            return paddle.batch(
+            return fluid.io.batch(
                 self.get_infer_examples(self.data_dir, epoch, self.max_seq_len), batch_size)
         else:
             raise ValueError(

PaddleNLP/emotion_detection/run_classifier.py

@@ -32,30 +32,30 @@ import numpy as np
 
 from models.classification import nets
 from models.model_check import check_cuda
+from models.model_check import check_version
 from config import PDConfig
 import reader
 import utils
 
 
 def create_model(args,
-                 pyreader_name,
                  num_labels,
                  is_prediction=False):
     """
     Create Model for Emotion Detection
     """
-    data = fluid.layers.data(name="words", shape=[-1, args.max_seq_len, 1], dtype="int64")
-    label = fluid.layers.data(name="label", shape=[-1, 1], dtype="int64")
-    seq_len = fluid.layers.data(name="seq_len", shape=[-1, 1], dtype="int64")
+    data = fluid.data(name="words", shape=[-1, args.max_seq_len], dtype="int64")
+    label = fluid.data(name="label", shape=[-1, 1], dtype="int64")
+    seq_len = fluid.data(name="seq_len", shape=[-1], dtype="int64")
 
     if is_prediction:
-        pyreader = fluid.io.PyReader(
+        loader = fluid.io.DataLoader.from_generator(
             feed_list=[data, seq_len],
             capacity=16,
             iterable=False,
             return_list=False)
     else:
-        pyreader = fluid.io.PyReader(
+        loader = fluid.io.DataLoader.from_generator(
             feed_list=[data, label, seq_len],
             capacity=16,
             iterable=False,
@@ -78,19 +78,19 @@ def create_model(args,
 
     if is_prediction:
         probs = network(data, seq_len, None, args.vocab_size, class_dim=num_labels, is_prediction=True)
-        return pyreader, probs, [data.name, seq_len.name]
+        return loader, probs, [data.name, seq_len.name]
 
     avg_loss, probs = network(data, seq_len, label, args.vocab_size, class_dim=num_labels)
     num_seqs = fluid.layers.create_tensor(dtype='int64')
     accuracy = fluid.layers.accuracy(input=probs, label=label, total=num_seqs)
-    return pyreader, avg_loss, accuracy, num_seqs
+    return loader, avg_loss, accuracy, num_seqs
 
 
-def evaluate(exe, test_program, test_pyreader, fetch_list, eval_phase):
+def evaluate(exe, test_program, test_loader, fetch_list, eval_phase):
     """
     Evaluation Function
     """
-    test_pyreader.start()
+    test_loader.start()
     total_cost, total_acc, total_num_seqs = [], [], []
     time_begin = time.time()
     while True:
@@ -105,7 +105,7 @@ def evaluate(exe, test_program, test_pyreader, fetch_list, eval_phase):
             total_acc.extend(np_acc * np_num_seqs)
             total_num_seqs.extend(np_num_seqs)
         except fluid.core.EOFException:
-            test_pyreader.reset()
+            test_loader.reset()
             break
     time_end = time.time()
     print("[%s evaluation] avg loss: %f, avg acc: %f, elapsed time: %f s" %
@@ -113,8 +113,8 @@ def evaluate(exe, test_program, test_pyreader, fetch_list, eval_phase):
            np.sum(total_acc) / np.sum(total_num_seqs), time_end - time_begin))
 
 
-def infer(exe, infer_program, infer_pyreader, fetch_list, infer_phase):
-    infer_pyreader.start()
+def infer(exe, infer_program, infer_loader, fetch_list, infer_phase):
+    infer_loader.start()
     time_begin = time.time()
     while True:
         try:
@@ -125,7 +125,7 @@ def infer(exe, infer_program, infer_pyreader, fetch_list, infer_phase):
                 print("%d\t%f\t%f\t%f" %
                       (np.argmax(probs), probs[0], probs[1], probs[2]))
         except fluid.core.EOFException as e:
-            infer_pyreader.reset()
+            infer_loader.reset()
             break
     time_end = time.time()
     print("[%s] elapsed time: %f s" % (infer_phase, time_end - time_begin))
@@ -172,9 +172,8 @@ def main(args):
 
         with fluid.program_guard(train_program, startup_prog):
             with fluid.unique_name.guard():
-                train_pyreader, loss, accuracy, num_seqs = create_model(
+                train_loader, loss, accuracy, num_seqs = create_model(
                     args,
-                    pyreader_name='train_reader',
                     num_labels=num_labels,
                     is_prediction=False)
 
@@ -202,9 +201,8 @@ def main(args):
         test_prog = fluid.Program()
         with fluid.program_guard(test_prog, startup_prog):
             with fluid.unique_name.guard():
-                test_pyreader, loss, accuracy, num_seqs = create_model(
+                test_loader, loss, accuracy, num_seqs = create_model(
                     args,
-                    pyreader_name='test_reader',
                     num_labels=num_labels,
                     is_prediction=False)
         test_prog = test_prog.clone(for_test=True)
@@ -218,9 +216,8 @@ def main(args):
         test_prog = fluid.Program()
         with fluid.program_guard(test_prog, startup_prog):
             with fluid.unique_name.guard():
-                infer_pyreader, probs, _ = create_model(
+                infer_loader, probs, _ = create_model(
                     args,
-                    pyreader_name='infer_reader',
                     num_labels=num_labels,
                     is_prediction=True)
         test_prog = test_prog.clone(for_test=True)
@@ -239,18 +236,18 @@ def main(args):
 
     if args.do_train:
         train_exe = exe
-        train_pyreader.decorate_sample_list_generator(train_data_generator)
+        train_loader.set_sample_list_generator(train_data_generator)
     else:
         train_exe = None
     if args.do_val:
         test_exe = exe
-        test_pyreader.decorate_sample_list_generator(test_data_generator)
+        test_loader.set_sample_list_generator(test_data_generator)
     if args.do_infer:
         test_exe = exe
-        infer_pyreader.decorate_sample_list_generator(infer_data_generator)
+        infer_loader.set_sample_list_generator(infer_data_generator)
 
     if args.do_train:
-        train_pyreader.start()
+        train_loader.start()
         steps = 0
         total_cost, total_acc, total_num_seqs = [], [], []
         time_begin = time.time()
@@ -276,7 +273,7 @@ def main(args):
                     total_num_seqs.extend(np_num_seqs)
 
                     if args.verbose:
-                        verbose = "train pyreader queue size: %d, " % train_pyreader.queue.size(
+                        verbose = "train loader queue size: %d, " % train_loader.queue.size(
                         )
                         print(verbose)
 
@@ -301,20 +298,20 @@ def main(args):
                 if steps % args.validation_steps == 0:
                     # evaluate on dev set
                     if args.do_val:
-                        evaluate(test_exe, test_prog, test_pyreader,
+                        evaluate(test_exe, test_prog, test_loader,
                                  [loss.name, accuracy.name, num_seqs.name],
                                  "dev")
 
             except fluid.core.EOFException:
                 print("final step: %d " % steps)
                 if args.do_val:
-                    evaluate(test_exe, test_prog, test_pyreader,
+                    evaluate(test_exe, test_prog, test_loader,
                         [loss.name, accuracy.name, num_seqs.name],
                         "dev")
 
                 save_path = os.path.join(args.save_checkpoint_dir, "step_" + str(steps))
                 fluid.io.save_persistables(exe, save_path, train_program)
-                train_pyreader.reset()
+                train_loader.reset()
                 break
 
     if args.do_train and args.enable_ce:
@@ -336,14 +333,14 @@ def main(args):
     # evaluate on test set
     if not args.do_train and args.do_val:
         print("Final test result:")
-        evaluate(test_exe, test_prog, test_pyreader,
+        evaluate(test_exe, test_prog, test_loader,
                  [loss.name, accuracy.name, num_seqs.name],
                  "test")
 
     # infer
     if args.do_infer:
         print("Final infer result:")
-        infer(test_exe, test_prog, infer_pyreader,
+        infer(test_exe, test_prog, infer_loader,
              [probs.name],
              "infer")
 
@@ -361,4 +358,5 @@ if __name__ == "__main__":
     args.build()
     args.print_arguments()
     check_cuda(args.use_cuda)
+    check_version()
     main(args)

PaddleNLP/emotion_detection/run_ernie_classifier.py

@@ -305,7 +305,7 @@ def main(args):
 
     if args.do_train:
         train_exe = exe
-        train_pyreader.decorate_tensor_provider(train_data_generator)
+        train_pyreader.set_batch_generator(train_data_generator)
     else:
         train_exe = None
     if args.do_val or args.do_infer:
@@ -355,7 +355,7 @@ def main(args):
                 if steps % args.validation_steps == 0:
                     # evaluate dev set
                     if args.do_val:
-                        test_pyreader.decorate_tensor_provider(
+                        test_pyreader.set_batch_generator(
                             reader.data_generator(
                                 input_file=args.dev_set,
                                 batch_size=args.batch_size,
@@ -375,7 +375,7 @@ def main(args):
 
     # eval on test set
     if not args.do_train and args.do_val:
-        test_pyreader.decorate_tensor_provider(
+        test_pyreader.set_batch_generator(
             reader.data_generator(
                 input_file=args.test_set,
                 batch_size=args.batch_size, phase='test', epoch=1,
@@ -386,7 +386,7 @@ def main(args):
 
     # infer on infer set
     if args.do_infer:
-        infer_pyreader.decorate_tensor_provider(
+        infer_pyreader.set_batch_generator(
             reader.data_generator(
                 input_file=args.infer_set,
                 batch_size=args.batch_size,

PaddleNLP/models/classification/nets.py

@@ -17,7 +17,7 @@ def bow_net(data,
     Bow net
     """
     # embedding layer
-    emb = fluid.layers.embedding(input=data, size=[dict_dim, emb_dim])
+    emb = fluid.embedding(input=data, size=[dict_dim, emb_dim])
     emb = fluid.layers.sequence_unpad(emb, length=seq_len)
     # bow layer
     bow = fluid.layers.sequence_pool(input=emb, pool_type='sum')
@@ -50,7 +50,7 @@ def cnn_net(data,
     Conv net
     """
     # embedding layer
-    emb = fluid.layers.embedding(input=data, size=[dict_dim, emb_dim])
+    emb = fluid.embedding(input=data, size=[dict_dim, emb_dim])
     emb = fluid.layers.sequence_unpad(emb, length=seq_len)
     # convolution layer
     conv_3 = fluid.nets.sequence_conv_pool(
@@ -87,7 +87,7 @@ def lstm_net(data,
     Lstm net
     """
     # embedding layer
-    emb = fluid.layers.embedding(
+    emb = fluid.embedding(
         input=data,
         size=[dict_dim, emb_dim],
         param_attr=fluid.ParamAttr(learning_rate=emb_lr))
@@ -129,7 +129,7 @@ def bilstm_net(data,
     Bi-Lstm net
     """
     # embedding layer
-    emb = fluid.layers.embedding(
+    emb = fluid.embedding(
         input=data,
         size=[dict_dim, emb_dim],
         param_attr=fluid.ParamAttr(learning_rate=emb_lr))
@@ -175,7 +175,7 @@ def gru_net(data,
     """
     gru net
     """
-    emb = fluid.layers.embedding(
+    emb = fluid.embedding(
         input=data,
         size=[dict_dim, emb_dim],
         param_attr=fluid.ParamAttr(learning_rate=emb_lr))
@@ -216,7 +216,7 @@ def textcnn_net(data,
         win_sizes = [1, 2, 3]
 
     # embedding layer
-    emb = fluid.layers.embedding(input=data, size=[dict_dim, emb_dim])
+    emb = fluid.embedding(input=data, size=[dict_dim, emb_dim])
     emb = fluid.layers.sequence_unpad(emb, length=seq_len)
     # convolution layer
     convs = []

PaddleNLP/models/model_check.py

@@ -33,37 +33,6 @@ def check_cuda(use_cuda, err = \
     except Exception as e:
         pass
 
-def check_version():
-        """
-        Log error and exit when the installed version of paddlepaddle is
-        not satisfied.
-        """
-        err = "PaddlePaddle version 1.6 or higher is required, " \
-            "or a suitable develop version is satisfied as well. \n" \
-            "Please make sure the version is good with your code." \
-
-        try:
-            fluid.require_version('1.6.0')
-        except Exception as e:
-            print(err)
-            sys.exit(1)
-
-
-def check_version():
-    """
-    Log error and exit when the installed version of paddlepaddle is
-    not satisfied.
-    """
-    err = "PaddlePaddle version 1.6 or higher is required, " \
-          "or a suitable develop version is satisfied as well. \n" \
-          "Please make sure the version is good with your code." \
-
-    try:
-        fluid.require_version('1.6.0')
-    except Exception as e:
-        print(err)
-        sys.exit(1)
-
 
 def check_version():
     """

PaddleNLP/models/representation/ernie.py

@@ -30,19 +30,19 @@ from models.transformer_encoder import encoder, pre_process_layer
 
 def ernie_pyreader(args, pyreader_name):
     """define standard ernie pyreader"""
-    pyreader = fluid.layers.py_reader(
+    src_ids = fluid.data(name='1', shape=[-1, args.max_seq_len, 1], dtype='int64')
+    sent_ids = fluid.data(name='2', shape=[-1, args.max_seq_len, 1], dtype='int64')
+    pos_ids = fluid.data(name='3', shape=[-1, args.max_seq_len, 1], dtype='int64')
+    input_mask = fluid.data(name='4', shape=[-1, args.max_seq_len, 1], dtype='float32')
+    labels = fluid.data(name='5', shape=[-1, 1], dtype='int64')
+    seq_lens = fluid.data(name='6', shape=[-1], dtype='int64')
+
+    pyreader = fluid.io.DataLoader.from_generator(
+        feed_list=[src_ids, sent_ids, pos_ids, input_mask, labels, seq_lens],
         capacity=50,
-        shapes=[[-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1],
-                [-1, args.max_seq_len, 1], [-1, args.max_seq_len, 1], [-1, 1],
-                [-1, 1]],
-        dtypes=['int64', 'int64', 'int64', 'float32', 'int64', 'int64'],
-        lod_levels=[0, 0, 0, 0, 0, 0],
-        name=pyreader_name,
+        iterable=False,
         use_double_buffer=True)
 
-    (src_ids, sent_ids, pos_ids, input_mask, labels,
-     seq_lens) = fluid.layers.read_file(pyreader)
-
     ernie_inputs = {
         "src_ids": src_ids,
         "sent_ids": sent_ids,
@@ -80,6 +80,7 @@ def ernie_encoder_with_paddle_hub(ernie_inputs, max_seq_len):
     embeddings = {
         "sentence_embeddings": cls_feats,
         "token_embeddings": unpad_enc_out,
+        "padded_token_embeddings": enc_out
     }
 
     for k, v in embeddings.items():
@@ -106,6 +107,7 @@ def ernie_encoder(ernie_inputs, ernie_config):
     embeddings = {
         "sentence_embeddings": cls_feats,
         "token_embeddings": unpad_enc_out,
+        "padded_token_embeddings": enc_out
     }
 
     for k, v in embeddings.items():