diff --git a/ernie-gen/README.md b/ernie-gen/README.md index 12751f3b7a0cfe5042f95abe821fbc0f339b08ba..3da01df1708bdf633bba9124cfe750e46075efc3 100644 --- a/ernie-gen/README.md +++ b/ernie-gen/README.md @@ -26,7 +26,6 @@ For technical description of the algorithm, please see our paper: >Accepted by **IJCAI-2020** -![ERNIE-GEN](https://img.shields.io/badge/Pretraining-Generation-green) ![Gigaword](https://img.shields.io/badge/Abstractive Summarization-Gigaword-yellow) ![Gigaword](https://img.shields.io/badge/Abstractive Summarization-CNN/Daily Mail-blue) ![SQuAD](https://img.shields.io/badge/Question Generation-SQuAD-green) ![Personal-Chat](https://img.shields.io/badge/Dialogue Response-Personal Chat-yellowgreen) ![CoQA](https://img.shields.io/badge/Generative Question Answering-CoQA-orange) --- **[ERNIE-GEN](https://arxiv.org/abs/2001.11314.pdf) is a multi-flow language generation framework for both pre-training and fine-tuning.** We propose a novel **span-by-span generation** pre-training task to enable the model to **generate a semantically-complete span** at each step rather than a word, in light of the fact that entities, phrases in human writing are organized in a coherent manner. An **infilling generation mechanism** and a **noise-aware generation method** are incorporated into both pre-training and fine-tuning to alleviate **the problem of exposure bias**. In the pre-training phase, ERNIE-GEN adopts a **multi-granularity target fragments sampling** strategy to force decoder to rely more on the encoder representations other than the previous generated words to enhancing the correlation between encoder and decoder. diff --git a/ernie-gen/README.zh.md b/ernie-gen/README.zh.md index 2ee9cd4ad2e4e16472c8dc2b5a73dc7056f31236..b58673b3b83bd7f54112d3a3382bc40c32ff6f18 100644 --- a/ernie-gen/README.zh.md +++ b/ernie-gen/README.zh.md @@ -26,7 +26,6 @@ >Accepted by **IJCAI-2020** -![ERNIE-GEN](https://img.shields.io/badge/预训练-生成-green) ![Gigaword](https://img.shields.io/badge/生成式摘要-Gigaword-yellow) ![Gigaword](https://img.shields.io/badge/生成式摘要-CNN/Daily Mail-blue) ![SQuAD](https://img.shields.io/badge/问题生成-SQuAD-green) ![Personal-Chat](https://img.shields.io/badge/多轮对话-Personal Chat-yellowgreen) ![CoQA](https://img.shields.io/badge/多轮问答-CoQA-orange) --- **ERNIE-GEN 是面向生成任务的预训练-微调框架**,首次在预训练阶段加入**span-by-span 生成**任务,让模型每次能够生成一个语义完整的片段。在预训练和微调中通过**填充式生成机制**和**噪声感知机制**来缓解曝光偏差问题。此外, ERNIE-GEN 采样**多片段-多粒度目标文本采样**策略, 增强源文本和目标文本的关联性,加强了编码器和解码器的交互。