From f978801f9080e121f9e2a8ce72a35e4849f4d60c Mon Sep 17 00:00:00 2001 From: Zhong Hui Date: Thu, 17 Dec 2020 22:38:52 +0800 Subject: [PATCH] add pretrained model reference (#5098) --- PaddleNLP/docs/transformers.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/PaddleNLP/docs/transformers.md b/PaddleNLP/docs/transformers.md index 98a5b966..7735bac7 100644 --- a/PaddleNLP/docs/transformers.md +++ b/PaddleNLP/docs/transformers.md @@ -58,3 +58,8 @@ for batch in train_data_loader: 4. 定义训练所需的优化器，loss函数等，就可以开始进行模型fine-tune任务。 更多详细使用方法，请参考[examples](https://github.com/PaddlePaddle/models/tree/develop/PaddleNLP/examples)。 + +## 参考资料： +- 部分中文预训练模型来自：https://github.com/ymcui/Chinese-BERT-wwm +- Sun, Yu, et al. "Ernie: Enhanced representation through knowledge integration." arXiv preprint arXiv:1904.09223 (2019). +- Cui, Yiming, et al. "Pre-training with whole word masking for chinese bert." arXiv preprint arXiv:1906.08101 (2019). -- GitLab