From cedfa0802c949cc8baa4a92003b8a41df3b57b0e Mon Sep 17 00:00:00 2001 From: zhanghan17 Date: Tue, 19 May 2020 23:44:59 +0800 Subject: [PATCH] del tag --- ernie-gen/README.md | 1 - ernie-gen/README.zh.md | 1 - 2 files changed, 2 deletions(-) diff --git a/ernie-gen/README.md b/ernie-gen/README.md index 12751f3..3da01df 100644 --- a/ernie-gen/README.md +++ b/ernie-gen/README.md @@ -26,7 +26,6 @@ For technical description of the algorithm, please see our paper: >Accepted by **IJCAI-2020** -![ERNIE-GEN](https://img.shields.io/badge/Pretraining-Generation-green) ![Gigaword](https://img.shields.io/badge/Abstractive Summarization-Gigaword-yellow) ![Gigaword](https://img.shields.io/badge/Abstractive Summarization-CNN/Daily Mail-blue) ![SQuAD](https://img.shields.io/badge/Question Generation-SQuAD-green) ![Personal-Chat](https://img.shields.io/badge/Dialogue Response-Personal Chat-yellowgreen) ![CoQA](https://img.shields.io/badge/Generative Question Answering-CoQA-orange) --- **[ERNIE-GEN](https://arxiv.org/abs/2001.11314.pdf) is a multi-flow language generation framework for both pre-training and fine-tuning.** We propose a novel **span-by-span generation** pre-training task to enable the model to **generate a semantically-complete span** at each step rather than a word, in light of the fact that entities, phrases in human writing are organized in a coherent manner. An **infilling generation mechanism** and a **noise-aware generation method** are incorporated into both pre-training and fine-tuning to alleviate **the problem of exposure bias**. In the pre-training phase, ERNIE-GEN adopts a **multi-granularity target fragments sampling** strategy to force decoder to rely more on the encoder representations other than the previous generated words to enhancing the correlation between encoder and decoder. diff --git a/ernie-gen/README.zh.md b/ernie-gen/README.zh.md index 2ee9cd4..b58673b 100644 --- a/ernie-gen/README.zh.md +++ b/ernie-gen/README.zh.md @@ -26,7 +26,6 @@ >Accepted by **IJCAI-2020** -![ERNIE-GEN](https://img.shields.io/badge/预训练-生成-green) ![Gigaword](https://img.shields.io/badge/生成式摘要-Gigaword-yellow) ![Gigaword](https://img.shields.io/badge/生成式摘要-CNN/Daily Mail-blue) ![SQuAD](https://img.shields.io/badge/问题生成-SQuAD-green) ![Personal-Chat](https://img.shields.io/badge/多轮对话-Personal Chat-yellowgreen) ![CoQA](https://img.shields.io/badge/多轮问答-CoQA-orange) --- **ERNIE-GEN 是面向生成任务的预训练-微调框架**，首次在预训练阶段加入**span-by-span 生成**任务，让模型每次能够生成一个语义完整的片段。在预训练和微调中通过**填充式生成机制**和**噪声感知机制**来缓解曝光偏差问题。此外, ERNIE-GEN 采样**多片段-多粒度目标文本采样**策略, 增强源文本和目标文本的关联性，加强了编码器和解码器的交互。 -- GitLab