[model_config.json](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/tokenizer_config.json)
[vocab.json](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/vocab.json)
[vocab.txt](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/vocab.txt) | - -也可以通过`paddlenlp` cli 工具来下载对应的模型权重，使用步骤如下所示： - -* 安装paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* 下载命令行 - -```shell -paddlenlp download --cache-dir ./pretrained_models csarron/roberta-base-squad-v1 -``` - -有任何下载的问题都可以到[PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP)中发Issue提问。 \ No newline at end of file diff --git a/modelcenter/community/csarron/roberta-base-squad-v1/download_en.md b/modelcenter/community/csarron/roberta-base-squad-v1/download_en.md deleted file mode 100644 index bbdc2aebe7f11f9802a0f37ea5fb64cd57a2b5d9..0000000000000000000000000000000000000000 --- a/modelcenter/community/csarron/roberta-base-squad-v1/download_en.md +++ /dev/null @@ -1,23 +0,0 @@ -# model list - -## - -| model | description | model_size | download | -| --- | --- | --- | --- | -|csarron/roberta-base-squad-v1| | 475.51MB | [merges.txt](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/merges.txt)
[model_config.json](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/tokenizer_config.json)
[vocab.json](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/vocab.json)
[vocab.txt](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/vocab.txt) | - -or you can download all of model file with the following steps: - -* install paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* download model with cli tool - -```shell -paddlenlp download --cache-dir ./pretrained_models csarron/roberta-base-squad-v1 -``` - -If you have any problems with it, you can post issue on [PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP) to get support. diff --git a/modelcenter/community/csarron/roberta-base-squad-v1/info.yaml b/modelcenter/community/csarron/roberta-base-squad-v1/info.yaml deleted file mode 100644 index 1095b2a22715b39a60d723b4e76648f4c9eab5f9..0000000000000000000000000000000000000000 --- a/modelcenter/community/csarron/roberta-base-squad-v1/info.yaml +++ /dev/null @@ -1,24 +0,0 @@ -Model_Info: - name: "csarron/roberta-base-squad-v1" - description: "RoBERTa-base fine-tuned on SQuAD v1" - description_en: "RoBERTa-base fine-tuned on SQuAD v1" - icon: "" - from_repo: "https://huggingface.co/csarron/roberta-base-squad-v1" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Question Answering" - sub_tag: "回答问题" - -Example: - -Datasets: "squad" -Publisher: "csarron" -License: "mit" -Language: "English" -Paper: - - title: 'RoBERTa: A Robustly Optimized BERT Pretraining Approach' - url: 'http://arxiv.org/abs/1907.11692v1' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file diff --git a/modelcenter/community/dbmdz/bert-base-german-cased/info.yaml b/modelcenter/community/dbmdz/bert-base-german-cased/info.yaml index 2eb9d8d9aa7eb32c7371b54ddf8217903f47450c..ba61f65725d34d0729497a8fabaffeeb71d86cd8 100644 --- a/modelcenter/community/dbmdz/bert-base-german-cased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-german-cased/info.yaml @@ -1,23 +1,19 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: German +License: mit Model_Info: - name: "dbmdz/bert-base-german-cased" - description: "🤗 + 📚 dbmdz German BERT models" - description_en: "🤗 + 📚 dbmdz German BERT models" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-german-cased" - + description: 🤗 + 📚 dbmdz German BERT models + description_en: 🤗 + 📚 dbmdz German BERT models + from_repo: https://huggingface.co/dbmdz/bert-base-german-cased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-german-cased +Paper: null +Publisher: dbmdz Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "dbmdz" -License: "mit" -Language: "German" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dbmdz/bert-base-german-cased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-german-cased/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..2a4843b0a54da647797ee926991813ff5a29c371 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-german-cased/introduction_cn.ipynb @@ -0,0 +1,95 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "9fb28341", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz German BERT models\n", + "\n", + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources another German BERT models 🎉\n", + "\n", + "# German BERT\n", + "\n", + "## Stats\n", + "\n", + "In addition to the recently released [German BERT](https://deepset.ai/german-bert)\n", + "model by [deepset](https://deepset.ai/) we provide another German-language model.\n", + "\n", + "The source data for the model consists of a recent Wikipedia dump, EU Bookshop corpus,\n", + "Open Subtitles, CommonCrawl, ParaCrawl and News Crawl. This results in a dataset with\n", + "a size of 16GB and 2,350,234,427 tokens.\n", + "\n", + "For sentence splitting, we use [spacy](https://spacy.io/). Our preprocessing steps\n", + "(sentence piece model for vocab generation) follow those used for training\n", + "[SciBERT](https://github.com/allenai/scibert). The model is trained with an initial\n", + "sequence length of 512 subwords and was performed for 1.5M steps." + ] + }, + { + "cell_type": "markdown", + "id": "589fadf4", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "646e12d4", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5935d3e0", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-german-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "b05add24", + "metadata": {}, + "source": [ + "# Reference\n", + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-german-cased](https://huggingface.co/dbmdz/bert-base-german-cased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-german-cased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-german-cased/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..8f9e678c5c0373c788f456ae1d513755c55f370f --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-german-cased/introduction_en.ipynb @@ -0,0 +1,93 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "e875e0cc", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz German BERT models\n", + "\n", + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources another German BERT models 🎉\n", + "\n", + "# German BERT\n", + "\n", + "## Stats\n", + "\n", + "In addition to the recently released [German BERT](https://deepset.ai/german-bert)\n", + "model by [deepset](https://deepset.ai/) we provide another German-language model.\n", + "\n", + "The source data for the model consists of a recent Wikipedia dump, EU Bookshop corpus,\n", + "Open Subtitles, CommonCrawl, ParaCrawl and News Crawl. This results in a dataset with\n", + "a size of 16GB and 2,350,234,427 tokens.\n", + "\n", + "For sentence splitting, we use [spacy](https://spacy.io/). Our preprocessing steps\n", + "(sentence piece model for vocab generation) follow those used for training\n", + "[SciBERT](https://github.com/allenai/scibert). The model is trained with an initial\n", + "sequence length of 512 subwords and was performed for 1.5M steps." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8dcad967", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c7c65281", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-german-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "1b52feb8", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "bc00304a", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-german-cased](https://huggingface.co/dbmdz/bert-base-german-cased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-german-uncased/info.yaml b/modelcenter/community/dbmdz/bert-base-german-uncased/info.yaml index 519dcaa077a88073ab53bb7be0d58f478bfc296f..da3e16292df4d852acf399a6641c0e76ded1e1bc 100644 --- a/modelcenter/community/dbmdz/bert-base-german-uncased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-german-uncased/info.yaml @@ -1,23 +1,19 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: German +License: mit Model_Info: - name: "dbmdz/bert-base-german-uncased" - description: "🤗 + 📚 dbmdz German BERT models" - description_en: "🤗 + 📚 dbmdz German BERT models" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-german-uncased" - + description: 🤗 + 📚 dbmdz German BERT models + description_en: 🤗 + 📚 dbmdz German BERT models + from_repo: https://huggingface.co/dbmdz/bert-base-german-uncased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-german-uncased +Paper: null +Publisher: dbmdz Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "dbmdz" -License: "mit" -Language: "German" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dbmdz/bert-base-german-uncased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-german-uncased/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..9490ad2084d98551dced766897ad028f18959032 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-german-uncased/introduction_cn.ipynb @@ -0,0 +1,102 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "46b7bbb6", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz German BERT models\n", + "\n", + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources another German BERT models 🎉\n", + "\n", + "# German BERT\n", + "\n", + "## Stats\n", + "\n", + "In addition to the recently released [German BERT](https://deepset.ai/german-bert)\n", + "model by [deepset](https://deepset.ai/) we provide another German-language model.\n", + "\n", + "The source data for the model consists of a recent Wikipedia dump, EU Bookshop corpus,\n", + "Open Subtitles, CommonCrawl, ParaCrawl and News Crawl. This results in a dataset with\n", + "a size of 16GB and 2,350,234,427 tokens.\n", + "\n", + "For sentence splitting, we use [spacy](https://spacy.io/). Our preprocessing steps\n", + "(sentence piece model for vocab generation) follow those used for training\n", + "[SciBERT](https://github.com/allenai/scibert). The model is trained with an initial\n", + "sequence length of 512 subwords and was performed for 1.5M steps." + ] + }, + { + "cell_type": "markdown", + "id": "bc37d3e3", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2afff18c", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "967f058e", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-german-uncased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "483dbced", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "04e50d8c", + "metadata": {}, + "source": [ + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-german-uncased](https://huggingface.co/dbmdz/bert-base-german-uncased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-german-uncased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-german-uncased/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..3c19f2ff3c29a2f5d5206047d08c0f5151ed781f --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-german-uncased/introduction_en.ipynb @@ -0,0 +1,101 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "5e0d446c", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz German BERT models\n", + "\n", + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources another German BERT models 🎉\n", + "\n", + "# German BERT\n", + "\n", + "## Stats\n", + "\n", + "In addition to the recently released [German BERT](https://deepset.ai/german-bert)\n", + "model by [deepset](https://deepset.ai/) we provide another German-language model.\n", + "\n", + "The source data for the model consists of a recent Wikipedia dump, EU Bookshop corpus,\n", + "Open Subtitles, CommonCrawl, ParaCrawl and News Crawl. This results in a dataset with\n", + "a size of 16GB and 2,350,234,427 tokens.\n", + "\n", + "For sentence splitting, we use [spacy](https://spacy.io/). Our preprocessing steps\n", + "(sentence piece model for vocab generation) follow those used for training\n", + "[SciBERT](https://github.com/allenai/scibert). The model is trained with an initial\n", + "sequence length of 512 subwords and was performed for 1.5M steps." + ] + }, + { + "cell_type": "markdown", + "id": "524680d5", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "39332440", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "19cf118e", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-german-uncased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "fb81d709", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "747fd5d3", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-german-uncased](https://huggingface.co/dbmdz/bert-base-german-uncased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-italian-uncased/info.yaml b/modelcenter/community/dbmdz/bert-base-italian-uncased/info.yaml index 9aea3699b760462595d111702ff925806d94ac1e..1072ac17af37b9cdc0075d8f48e7be6d5b388d1d 100644 --- a/modelcenter/community/dbmdz/bert-base-italian-uncased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-italian-uncased/info.yaml @@ -1,23 +1,19 @@ +Datasets: wikipedia +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Italian +License: mit Model_Info: - name: "dbmdz/bert-base-italian-uncased" - description: "🤗 + 📚 dbmdz BERT and ELECTRA models" - description_en: "🤗 + 📚 dbmdz BERT and ELECTRA models" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-italian-uncased" - + description: 🤗 + 📚 dbmdz BERT and ELECTRA models + description_en: 🤗 + 📚 dbmdz BERT and ELECTRA models + from_repo: https://huggingface.co/dbmdz/bert-base-italian-uncased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-italian-uncased +Paper: null +Publisher: dbmdz Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "wikipedia" -Publisher: "dbmdz" -License: "mit" -Language: "Italian" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dbmdz/bert-base-italian-uncased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-italian-uncased/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..eebce5803b87659cf344be6c5cd12c6630b10a92 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-italian-uncased/introduction_cn.ipynb @@ -0,0 +1,152 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "dea2fc9e", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz BERT and ELECTRA models\n" + ] + }, + { + "cell_type": "markdown", + "id": "00744cbd", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources Italian BERT and ELECTRA models 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "d7106b74", + "metadata": {}, + "source": [ + "# Italian BERT\n" + ] + }, + { + "cell_type": "markdown", + "id": "7ee0fd67", + "metadata": {}, + "source": [ + "The source data for the Italian BERT model consists of a recent Wikipedia dump and\n", + "various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final\n", + "training corpus has a size of 13GB and 2,050,057,573 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "a3961910", + "metadata": {}, + "source": [ + "For sentence splitting, we use NLTK (faster compared to spacy).\n", + "Our cased and uncased models are training with an initial sequence length of 512\n", + "subwords for ~2-3M steps.\n" + ] + }, + { + "cell_type": "markdown", + "id": "480e4fea", + "metadata": {}, + "source": [ + "For the XXL Italian models, we use the same training data from OPUS and extend\n", + "it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/).\n", + "Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "d710804e", + "metadata": {}, + "source": [ + "Note: Unfortunately, a wrong vocab size was used when training the XXL models.\n", + "This explains the mismatch of the \"real\" vocab size of 31102, compared to the\n", + "vocab size specified in `config.json`. However, the model is working and all\n", + "evaluations were done under those circumstances.\n", + "See [this issue](https://github.com/dbmdz/berts/issues/7) for more information.\n" + ] + }, + { + "cell_type": "markdown", + "id": "2d9c79e5", + "metadata": {}, + "source": [ + "The Italian ELECTRA model was trained on the \"XXL\" corpus for 1M steps in total using a batch\n", + "size of 128. We pretty much following the ELECTRA training procedure as used for\n", + "[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra).\n" + ] + }, + { + "cell_type": "markdown", + "id": "3ee71cee", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9ffe9a93", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "82d327d4", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-italian-uncased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "56d92161", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "ad146f63", + "metadata": {}, + "source": [ + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-italian-uncased](https://huggingface.co/dbmdz/bert-base-italian-uncased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-italian-uncased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-italian-uncased/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..843ea2d291150fb6ef774f5acc299623bd782d7b --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-italian-uncased/introduction_en.ipynb @@ -0,0 +1,151 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "8601b7e0", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz BERT and ELECTRA models\n" + ] + }, + { + "cell_type": "markdown", + "id": "2e2ee06f", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources Italian BERT and ELECTRA models 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "a7b6e470", + "metadata": {}, + "source": [ + "# Italian BERT\n" + ] + }, + { + "cell_type": "markdown", + "id": "d1afb03c", + "metadata": {}, + "source": [ + "The source data for the Italian BERT model consists of a recent Wikipedia dump and\n", + "various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final\n", + "training corpus has a size of 13GB and 2,050,057,573 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "a900d41a", + "metadata": {}, + "source": [ + "For sentence splitting, we use NLTK (faster compared to spacy).\n", + "Our cased and uncased models are training with an initial sequence length of 512\n", + "subwords for ~2-3M steps.\n" + ] + }, + { + "cell_type": "markdown", + "id": "d4ea3425", + "metadata": {}, + "source": [ + "For the XXL Italian models, we use the same training data from OPUS and extend\n", + "it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/).\n", + "Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "f1d5804d", + "metadata": {}, + "source": [ + "Note: Unfortunately, a wrong vocab size was used when training the XXL models.\n", + "This explains the mismatch of the \"real\" vocab size of 31102, compared to the\n", + "vocab size specified in `config.json`. However, the model is working and all\n", + "evaluations were done under those circumstances.\n", + "See [this issue](https://github.com/dbmdz/berts/issues/7) for more information.\n" + ] + }, + { + "cell_type": "markdown", + "id": "cc4f3d3d", + "metadata": {}, + "source": [ + "The Italian ELECTRA model was trained on the \"XXL\" corpus for 1M steps in total using a batch\n", + "size of 128. We pretty much following the ELECTRA training procedure as used for\n", + "[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra).\n" + ] + }, + { + "cell_type": "markdown", + "id": "76e431e8", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1b014af1", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ca7904c6", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-italian-uncased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "261390e6", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "f5c0c815", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-italian-uncased](https://huggingface.co/dbmdz/bert-base-italian-uncased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/info.yaml b/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/info.yaml index 5a0420fcd805c01627cd1729ec6cd342bd0902b6..4ed571e88339e30d592945336c032ebcffa79f36 100644 --- a/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/info.yaml @@ -1,23 +1,19 @@ +Datasets: wikipedia +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Italian +License: mit Model_Info: - name: "dbmdz/bert-base-italian-xxl-cased" - description: "🤗 + 📚 dbmdz BERT and ELECTRA models" - description_en: "🤗 + 📚 dbmdz BERT and ELECTRA models" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-italian-xxl-cased" - + description: 🤗 + 📚 dbmdz BERT and ELECTRA models + description_en: 🤗 + 📚 dbmdz BERT and ELECTRA models + from_repo: https://huggingface.co/dbmdz/bert-base-italian-xxl-cased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-italian-xxl-cased +Paper: null +Publisher: dbmdz Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "wikipedia" -Publisher: "dbmdz" -License: "mit" -Language: "Italian" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..f96fe1ecf6b1cff8a991dfdeef567c250a4f9b4f --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/introduction_cn.ipynb @@ -0,0 +1,151 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "4e448d86", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz BERT and ELECTRA models\n" + ] + }, + { + "cell_type": "markdown", + "id": "9bcf089b", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources Italian BERT and ELECTRA models 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "fb9adbdd", + "metadata": {}, + "source": [ + "# Italian BERT\n" + ] + }, + { + "cell_type": "markdown", + "id": "e5a80c49", + "metadata": {}, + "source": [ + "The source data for the Italian BERT model consists of a recent Wikipedia dump and\n", + "various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final\n", + "training corpus has a size of 13GB and 2,050,057,573 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "3513aa96", + "metadata": {}, + "source": [ + "For sentence splitting, we use NLTK (faster compared to spacy).\n", + "Our cased and uncased models are training with an initial sequence length of 512\n", + "subwords for ~2-3M steps.\n" + ] + }, + { + "cell_type": "markdown", + "id": "ca0e58ee", + "metadata": {}, + "source": [ + "For the XXL Italian models, we use the same training data from OPUS and extend\n", + "it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/).\n", + "Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "744e0851", + "metadata": {}, + "source": [ + "Note: Unfortunately, a wrong vocab size was used when training the XXL models.\n", + "This explains the mismatch of the \"real\" vocab size of 31102, compared to the\n", + "vocab size specified in `config.json`. However, the model is working and all\n", + "evaluations were done under those circumstances.\n", + "See [this issue](https://github.com/dbmdz/berts/issues/7) for more information.\n" + ] + }, + { + "cell_type": "markdown", + "id": "3bb28396", + "metadata": {}, + "source": [ + "The Italian ELECTRA model was trained on the \"XXL\" corpus for 1M steps in total using a batch\n", + "size of 128. We pretty much following the ELECTRA training procedure as used for\n", + "[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra).\n" + ] + }, + { + "cell_type": "markdown", + "id": "4c0c2ecb", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e059cf91", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "95a883f8", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-italian-xxl-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "6b2c856e", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "ffdf7223", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-italian-xxl-cased](https://huggingface.co/dbmdz/bert-base-italian-xxl-cased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..cdd5de4e81c6931a8b415e35299b5dd3e863be12 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/introduction_en.ipynb @@ -0,0 +1,151 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "41ca2df0", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz BERT and ELECTRA models\n" + ] + }, + { + "cell_type": "markdown", + "id": "58e60a32", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources Italian BERT and ELECTRA models 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "c7b5f379", + "metadata": {}, + "source": [ + "# Italian BERT\n" + ] + }, + { + "cell_type": "markdown", + "id": "5bf65013", + "metadata": {}, + "source": [ + "The source data for the Italian BERT model consists of a recent Wikipedia dump and\n", + "various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final\n", + "training corpus has a size of 13GB and 2,050,057,573 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "a3aadc8d", + "metadata": {}, + "source": [ + "For sentence splitting, we use NLTK (faster compared to spacy).\n", + "Our cased and uncased models are training with an initial sequence length of 512\n", + "subwords for ~2-3M steps.\n" + ] + }, + { + "cell_type": "markdown", + "id": "4d670485", + "metadata": {}, + "source": [ + "For the XXL Italian models, we use the same training data from OPUS and extend\n", + "it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/).\n", + "Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "a366dc7d", + "metadata": {}, + "source": [ + "Note: Unfortunately, a wrong vocab size was used when training the XXL models.\n", + "This explains the mismatch of the \"real\" vocab size of 31102, compared to the\n", + "vocab size specified in `config.json`. However, the model is working and all\n", + "evaluations were done under those circumstances.\n", + "See [this issue](https://github.com/dbmdz/berts/issues/7) for more information.\n" + ] + }, + { + "cell_type": "markdown", + "id": "eaee3adf", + "metadata": {}, + "source": [ + "The Italian ELECTRA model was trained on the \"XXL\" corpus for 1M steps in total using a batch\n", + "size of 128. We pretty much following the ELECTRA training procedure as used for\n", + "[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra).\n" + ] + }, + { + "cell_type": "markdown", + "id": "c5151f8a", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f9e48e19", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c02e6f47", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-italian-xxl-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "13e03e4b", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "66705724", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-italian-xxl-cased](https://huggingface.co/dbmdz/bert-base-italian-xxl-cased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/info.yaml b/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/info.yaml index 3b0c895b02120abb1c31203e8c88e40ba1357ab4..2715cc513f4d1bed057c97947f3fd08da13c7240 100644 --- a/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/info.yaml @@ -1,19 +1,15 @@ -Model_Info: - name: "dbmdz/bert-base-turkish-128k-cased" - description: "🤗 + 📚 dbmdz Turkish BERT model" - description_en: "🤗 + 📚 dbmdz Turkish BERT model" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-turkish-128k-cased" - -Task: - -Example: - -Datasets: "" -Publisher: "dbmdz" -License: "mit" -Language: "Turkish" -Paper: - +Datasets: '' +Example: null +IfOnlineDemo: 0 IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +Language: Turkish +License: mit +Model_Info: + description: 🤗 + 📚 dbmdz Turkish BERT model + description_en: 🤗 + 📚 dbmdz Turkish BERT model + from_repo: https://huggingface.co/dbmdz/bert-base-turkish-128k-cased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-turkish-128k-cased +Paper: null +Publisher: dbmdz +Task: null diff --git a/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..a5568c662a25bcfee3015f33a87a832561904440 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/introduction_cn.ipynb @@ -0,0 +1,151 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "bda1db47", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz Turkish BERT model\n" + ] + }, + { + "cell_type": "markdown", + "id": "ba254a15", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources a cased model for Turkish 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "bf2818ba", + "metadata": {}, + "source": [ + "# 🇹🇷 BERTurk\n" + ] + }, + { + "cell_type": "markdown", + "id": "788f7baa", + "metadata": {}, + "source": [ + "BERTurk is a community-driven cased BERT model for Turkish.\n" + ] + }, + { + "cell_type": "markdown", + "id": "5e051a7d", + "metadata": {}, + "source": [ + "Some datasets used for pretraining and evaluation are contributed from the\n", + "awesome Turkish NLP community, as well as the decision for the model name: BERTurk.\n" + ] + }, + { + "cell_type": "markdown", + "id": "1edbcf52", + "metadata": {}, + "source": [ + "## Stats\n" + ] + }, + { + "cell_type": "markdown", + "id": "5b7c3ff4", + "metadata": {}, + "source": [ + "The current version of the model is trained on a filtered and sentence\n", + "segmented version of the Turkish [OSCAR corpus](https://traces1.inria.fr/oscar/),\n", + "a recent Wikipedia dump, various [OPUS corpora](http://opus.nlpl.eu/) and a\n", + "special corpus provided by [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/).\n" + ] + }, + { + "cell_type": "markdown", + "id": "9413f21a", + "metadata": {}, + "source": [ + "The final training corpus has a size of 35GB and 44,04,976,662 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "25593952", + "metadata": {}, + "source": [ + "For this model we use a vocab size of 128k.\n" + ] + }, + { + "cell_type": "markdown", + "id": "962cf00d", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d4a4e8e3", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7157d7da", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-turkish-128k-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "e47155ee", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "081638b2", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-turkish-128k-cased](https://huggingface.co/dbmdz/bert-base-turkish-128k-cased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..4979c965075ba1d231cbf1a0f651efe3b991e63e --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/introduction_en.ipynb @@ -0,0 +1,151 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "911a1be9", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz Turkish BERT model\n" + ] + }, + { + "cell_type": "markdown", + "id": "4f09b0f1", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources a cased model for Turkish 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "fa2a78a0", + "metadata": {}, + "source": [ + "# 🇹🇷 BERTurk\n" + ] + }, + { + "cell_type": "markdown", + "id": "8b2f8c68", + "metadata": {}, + "source": [ + "BERTurk is a community-driven cased BERT model for Turkish.\n" + ] + }, + { + "cell_type": "markdown", + "id": "fe23e365", + "metadata": {}, + "source": [ + "Some datasets used for pretraining and evaluation are contributed from the\n", + "awesome Turkish NLP community, as well as the decision for the model name: BERTurk.\n" + ] + }, + { + "cell_type": "markdown", + "id": "2da0ce24", + "metadata": {}, + "source": [ + "## Stats\n" + ] + }, + { + "cell_type": "markdown", + "id": "d3f6af43", + "metadata": {}, + "source": [ + "The current version of the model is trained on a filtered and sentence\n", + "segmented version of the Turkish [OSCAR corpus](https://traces1.inria.fr/oscar/),\n", + "a recent Wikipedia dump, various [OPUS corpora](http://opus.nlpl.eu/) and a\n", + "special corpus provided by [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/).\n" + ] + }, + { + "cell_type": "markdown", + "id": "0d8d60c1", + "metadata": {}, + "source": [ + "The final training corpus has a size of 35GB and 44,04,976,662 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "ce42504f", + "metadata": {}, + "source": [ + "For this model we use a vocab size of 128k.\n" + ] + }, + { + "cell_type": "markdown", + "id": "4815bfdb", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7a084604", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8d041c78", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-turkish-128k-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "da82079c", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "b6632d46", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-turkish-128k-cased](https://huggingface.co/dbmdz/bert-base-turkish-128k-cased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-turkish-cased/info.yaml b/modelcenter/community/dbmdz/bert-base-turkish-cased/info.yaml index 394d341ab1796ec9906a455109641d4a24735a36..619970861ef2728dbe52cbacc1c15475b703ac21 100644 --- a/modelcenter/community/dbmdz/bert-base-turkish-cased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-turkish-cased/info.yaml @@ -1,19 +1,15 @@ -Model_Info: - name: "dbmdz/bert-base-turkish-cased" - description: "🤗 + 📚 dbmdz Turkish BERT model" - description_en: "🤗 + 📚 dbmdz Turkish BERT model" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-turkish-cased" - -Task: - -Example: - -Datasets: "" -Publisher: "dbmdz" -License: "mit" -Language: "Turkish" -Paper: - +Datasets: '' +Example: null +IfOnlineDemo: 0 IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +Language: Turkish +License: mit +Model_Info: + description: 🤗 + 📚 dbmdz Turkish BERT model + description_en: 🤗 + 📚 dbmdz Turkish BERT model + from_repo: https://huggingface.co/dbmdz/bert-base-turkish-cased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-turkish-cased +Paper: null +Publisher: dbmdz +Task: null diff --git a/modelcenter/community/dbmdz/bert-base-turkish-cased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-turkish-cased/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..0e32f0526269707052bb3011e33756cef9d0ebb6 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-turkish-cased/introduction_cn.ipynb @@ -0,0 +1,118 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "e9075248", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz Turkish BERT model\n" + ] + }, + { + "cell_type": "markdown", + "id": "0f68224a", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources a cased model for Turkish 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "a800751f", + "metadata": {}, + "source": [ + "# 🇹🇷 BERTurk\n" + ] + }, + { + "cell_type": "markdown", + "id": "0f418bcc", + "metadata": {}, + "source": [ + "BERTurk is a community-driven cased BERT model for Turkish.\n" + ] + }, + { + "cell_type": "markdown", + "id": "059528cb", + "metadata": {}, + "source": [ + "Some datasets used for pretraining and evaluation are contributed from the\n", + "awesome Turkish NLP community, as well as the decision for the model name: BERTurk.\n" + ] + }, + { + "cell_type": "markdown", + "id": "ec8d00db", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1cb273ef", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "45fd943c", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-turkish-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "0653e10b", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "14b8d869", + "metadata": {}, + "source": [ + "\n", + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-turkish-cased](https://huggingface.co/dbmdz/bert-base-turkish-cased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-turkish-cased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-turkish-cased/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..f2d90a6ded8cb2dfad99f718cd13a0dfeb038e4d --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-turkish-cased/introduction_en.ipynb @@ -0,0 +1,117 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "22b9df2e", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz Turkish BERT model\n" + ] + }, + { + "cell_type": "markdown", + "id": "509b461d", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources a cased model for Turkish 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "84ab205a", + "metadata": {}, + "source": [ + "# 🇹🇷 BERTurk\n" + ] + }, + { + "cell_type": "markdown", + "id": "aafa4b5d", + "metadata": {}, + "source": [ + "BERTurk is a community-driven cased BERT model for Turkish.\n" + ] + }, + { + "cell_type": "markdown", + "id": "16251feb", + "metadata": {}, + "source": [ + "Some datasets used for pretraining and evaluation are contributed from the\n", + "awesome Turkish NLP community, as well as the decision for the model name: BERTurk.\n" + ] + }, + { + "cell_type": "markdown", + "id": "1bdf0158", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "aa2b4d91", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0f55d31d", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-turkish-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "9aae3e54", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "839b89b9", + "metadata": {}, + "source": [ + "\n", + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-turkish-cased](https://huggingface.co/dbmdz/bert-base-turkish-cased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-turkish-uncased/info.yaml b/modelcenter/community/dbmdz/bert-base-turkish-uncased/info.yaml index f771d3f68b52bf37c901a717c3d74a92a2156fb8..ffabf371a04554e26cf1b4a505a8d0347bd73c9a 100644 --- a/modelcenter/community/dbmdz/bert-base-turkish-uncased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-turkish-uncased/info.yaml @@ -1,19 +1,15 @@ -Model_Info: - name: "dbmdz/bert-base-turkish-uncased" - description: "🤗 + 📚 dbmdz Turkish BERT model" - description_en: "🤗 + 📚 dbmdz Turkish BERT model" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-turkish-uncased" - -Task: - -Example: - -Datasets: "" -Publisher: "dbmdz" -License: "mit" -Language: "Turkish" -Paper: - +Datasets: '' +Example: null +IfOnlineDemo: 0 IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +Language: Turkish +License: mit +Model_Info: + description: 🤗 + 📚 dbmdz Turkish BERT model + description_en: 🤗 + 📚 dbmdz Turkish BERT model + from_repo: https://huggingface.co/dbmdz/bert-base-turkish-uncased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-turkish-uncased +Paper: null +Publisher: dbmdz +Task: null diff --git a/modelcenter/community/dbmdz/bert-base-turkish-uncased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-turkish-uncased/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..c9332fb9eca2330d332aca6d1dfaa1ff855999df --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-turkish-uncased/introduction_cn.ipynb @@ -0,0 +1,118 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f3dbf349", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz Turkish BERT model\n" + ] + }, + { + "cell_type": "markdown", + "id": "479d8e10", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources an uncased model for Turkish 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "fc31d938", + "metadata": {}, + "source": [ + "# 🇹🇷 BERTurk\n" + ] + }, + { + "cell_type": "markdown", + "id": "c05c98f4", + "metadata": {}, + "source": [ + "BERTurk is a community-driven uncased BERT model for Turkish.\n" + ] + }, + { + "cell_type": "markdown", + "id": "116bbd89", + "metadata": {}, + "source": [ + "Some datasets used for pretraining and evaluation are contributed from the\n", + "awesome Turkish NLP community, as well as the decision for the model name: BERTurk.\n" + ] + }, + { + "cell_type": "markdown", + "id": "eab29ae1", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "531e7c2f", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c23a11a9", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-turkish-uncased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "1a4d2556", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "4e10e25f", + "metadata": {}, + "source": [ + "\n", + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-turkish-uncased](https://huggingface.co/dbmdz/bert-base-turkish-uncased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-turkish-uncased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-turkish-uncased/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..18d5794ee74f4d15ca5b050d057d4831ce9307be --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-turkish-uncased/introduction_en.ipynb @@ -0,0 +1,118 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f1968bb1", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz Turkish BERT model\n" + ] + }, + { + "cell_type": "markdown", + "id": "37119e6e", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources an uncased model for Turkish 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "e2428d3f", + "metadata": {}, + "source": [ + "# 🇹🇷 BERTurk\n" + ] + }, + { + "cell_type": "markdown", + "id": "455a98e2", + "metadata": {}, + "source": [ + "BERTurk is a community-driven uncased BERT model for Turkish.\n" + ] + }, + { + "cell_type": "markdown", + "id": "3c7b1272", + "metadata": {}, + "source": [ + "Some datasets used for pretraining and evaluation are contributed from the\n", + "awesome Turkish NLP community, as well as the decision for the model name: BERTurk.\n" + ] + }, + { + "cell_type": "markdown", + "id": "cdd8f852", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "81436ade", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bd223538", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-turkish-uncased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "7edb6fa7", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "95b108cb", + "metadata": {}, + "source": [ + "\n", + "\n", + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-turkish-uncased](https://huggingface.co/dbmdz/bert-base-turkish-uncased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/deepparag/Aeona/info.yaml b/modelcenter/community/deepparag/Aeona/info.yaml index 65ec5dc9572ca5e1e4180cc77a0dce91abd96c17..e9e858f766354f7f547ebfa1a07a2603038b93f8 100644 --- a/modelcenter/community/deepparag/Aeona/info.yaml +++ b/modelcenter/community/deepparag/Aeona/info.yaml @@ -1,23 +1,19 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: '' +License: mit Model_Info: - name: "deepparag/Aeona" - description: "Aeona | Chatbot" - description_en: "Aeona | Chatbot" - icon: "" - from_repo: "https://huggingface.co/deepparag/Aeona" - + description: Aeona | Chatbot + description_en: Aeona | Chatbot + from_repo: https://huggingface.co/deepparag/Aeona + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: deepparag/Aeona +Paper: null +Publisher: deepparag Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "" -Publisher: "deepparag" -License: "mit" -Language: "" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 文本生成 + sub_tag_en: Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/deepparag/Aeona/introduction_cn.ipynb b/modelcenter/community/deepparag/Aeona/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..7ccce23b10d222fda60c6923d7a261aee1b42464 --- /dev/null +++ b/modelcenter/community/deepparag/Aeona/introduction_cn.ipynb @@ -0,0 +1,116 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "9d9bb2aa", + "metadata": {}, + "source": [ + "# Aeona | Chatbot\n" + ] + }, + { + "cell_type": "markdown", + "id": "7361d804", + "metadata": {}, + "source": [ + "An generative AI made using [microsoft/DialoGPT-small](https://huggingface.co/microsoft/DialoGPT-small).\n" + ] + }, + { + "cell_type": "markdown", + "id": "008bcb8d", + "metadata": {}, + "source": [ + "Recommended to use along with an [AIML Chatbot](https://github.com/deepsarda/Aeona-Aiml) to reduce load, get better replies, add name and personality to your bot.\n", + "Using an AIML Chatbot will allow you to hardcode some replies also.\n" + ] + }, + { + "cell_type": "markdown", + "id": "b4bfb9cd", + "metadata": {}, + "source": [ + "## Evaluation\n", + "Below is a comparison of Aeona vs. other baselines on the mixed dataset given above using automatic evaluation metrics.\n" + ] + }, + { + "cell_type": "markdown", + "id": "4d478ffa", + "metadata": {}, + "source": [ + "| Model | Perplexity |\n", + "|---|---|\n", + "| Seq2seq Baseline [3] | 29.8 |\n", + "| Wolf et al. [5] | 16.3 |\n", + "| GPT-2 baseline | 99.5 |\n", + "| DialoGPT baseline | 56.6 |\n", + "| DialoGPT finetuned | 11.4 |\n", + "| PersonaGPT | 10.2 |\n", + "| **Aeona** | **7.9** |\n" + ] + }, + { + "cell_type": "markdown", + "id": "ebb927ce", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ea2a9d8e", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dc15795c", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"deepparag/Aeona\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "074bd20d", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/deepparag/Aeona](https://huggingface.co/deepparag/Aeona)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/deepparag/Aeona/introduction_en.ipynb b/modelcenter/community/deepparag/Aeona/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..e55b926e280f85db85fab4bed68273f9d2ada18a --- /dev/null +++ b/modelcenter/community/deepparag/Aeona/introduction_en.ipynb @@ -0,0 +1,116 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f8079990", + "metadata": {}, + "source": [ + "# Aeona | Chatbot\n" + ] + }, + { + "cell_type": "markdown", + "id": "6a69f81a", + "metadata": {}, + "source": [ + "An generative AI made using [microsoft/DialoGPT-small](https://huggingface.co/microsoft/DialoGPT-small).\n" + ] + }, + { + "cell_type": "markdown", + "id": "a65479b8", + "metadata": {}, + "source": [ + "Recommended to use along with an [AIML Chatbot](https://github.com/deepsarda/Aeona-Aiml) to reduce load, get better replies, add name and personality to your bot.\n", + "Using an AIML Chatbot will allow you to hardcode some replies also.\n" + ] + }, + { + "cell_type": "markdown", + "id": "ea390590", + "metadata": {}, + "source": [ + "## Evaluation\n", + "Below is a comparison of Aeona vs. other baselines on the mixed dataset given above using automatic evaluation metrics.\n" + ] + }, + { + "cell_type": "markdown", + "id": "5b64325a", + "metadata": {}, + "source": [ + "| Model | Perplexity |\n", + "|---|---|\n", + "| Seq2seq Baseline [3] | 29.8 |\n", + "| Wolf et al. [5] | 16.3 |\n", + "| GPT-2 baseline | 99.5 |\n", + "| DialoGPT baseline | 56.6 |\n", + "| DialoGPT finetuned | 11.4 |\n", + "| PersonaGPT | 10.2 |\n", + "| **Aeona** | **7.9** |\n" + ] + }, + { + "cell_type": "markdown", + "id": "bf7f0d0e", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "16b58290", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cf18c96e", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"deepparag/Aeona\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "fae16f6e", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/deepparag/Aeona](https://huggingface.co/deepparag/Aeona) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/deepparag/DumBot/download_cn.md b/modelcenter/community/deepparag/DumBot/download_cn.md deleted file mode 100644 index 4a06958c07563ed41bab7ee79f3debd41a8ec89d..0000000000000000000000000000000000000000 --- a/modelcenter/community/deepparag/DumBot/download_cn.md +++ /dev/null @@ -1,23 +0,0 @@ -# 模型列表 - -## deepparag/DumBot - -| 模型名称 | 模型介绍 | 模型大小 | 模型下载 | -| --- | --- | --- | --- | -|deepparag/DumBot| | 621.95MB | [merges.txt](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/merges.txt)
[model_config.json](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/tokenizer_config.json)
[vocab.json](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/vocab.json) | - -也可以通过`paddlenlp` cli 工具来下载对应的模型权重，使用步骤如下所示： - -* 安装paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* 下载命令行 - -```shell -paddlenlp download --cache-dir ./pretrained_models deepparag/DumBot -``` - -有任何下载的问题都可以到[PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP)中发Issue提问。 \ No newline at end of file diff --git a/modelcenter/community/deepparag/DumBot/download_en.md b/modelcenter/community/deepparag/DumBot/download_en.md deleted file mode 100644 index f2bd3c7436283f60730f9eb46958044c3a54eca3..0000000000000000000000000000000000000000 --- a/modelcenter/community/deepparag/DumBot/download_en.md +++ /dev/null @@ -1,23 +0,0 @@ -# model list - -## - -| model | description | model_size | download | -| --- | --- | --- | --- | -|deepparag/DumBot| | 621.95MB | [merges.txt](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/merges.txt)
[model_config.json](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/tokenizer_config.json)
[vocab.json](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/vocab.json) | - -or you can download all of model file with the following steps: - -* install paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* download model with cli tool - -```shell -paddlenlp download --cache-dir ./pretrained_models deepparag/DumBot -``` - -If you have any problems with it, you can post issue on [PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP) to get support. diff --git a/modelcenter/community/deepparag/DumBot/info.yaml b/modelcenter/community/deepparag/DumBot/info.yaml deleted file mode 100644 index 8f99ea06b4e1bc0e947407ff5c51bff15654c4c5..0000000000000000000000000000000000000000 --- a/modelcenter/community/deepparag/DumBot/info.yaml +++ /dev/null @@ -1,23 +0,0 @@ -Model_Info: - name: "deepparag/DumBot" - description: "THIS AI IS OUTDATED. See [Aeona](https://huggingface.co/deepparag/Aeona)" - description_en: "THIS AI IS OUTDATED. See [Aeona](https://huggingface.co/deepparag/Aeona)" - icon: "" - from_repo: "https://huggingface.co/deepparag/DumBot" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "" -Publisher: "deepparag" -License: "mit" -Language: "" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file diff --git a/modelcenter/community/deepset/roberta-base-squad2-distilled/info.yaml b/modelcenter/community/deepset/roberta-base-squad2-distilled/info.yaml index f351a6e3f7a836db46c759f7b962c17f240dbf02..2db8b542ffeb3de088e024b156e7fa1cdf744e50 100644 --- a/modelcenter/community/deepset/roberta-base-squad2-distilled/info.yaml +++ b/modelcenter/community/deepset/roberta-base-squad2-distilled/info.yaml @@ -1,23 +1,19 @@ +Datasets: squad_v2 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: mit Model_Info: - name: "deepset/roberta-base-squad2-distilled" - description: "Overview" - description_en: "Overview" - icon: "" - from_repo: "https://huggingface.co/deepset/roberta-base-squad2-distilled" - + description: Overview + description_en: Overview + from_repo: https://huggingface.co/deepset/roberta-base-squad2-distilled + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: deepset/roberta-base-squad2-distilled +Paper: null +Publisher: deepset Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Question Answering" - sub_tag: "回答问题" - -Example: - -Datasets: "squad_v2" -Publisher: "deepset" -License: "mit" -Language: "English" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 回答问题 + sub_tag_en: Question Answering + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/deepset/roberta-base-squad2-distilled/introduction_cn.ipynb b/modelcenter/community/deepset/roberta-base-squad2-distilled/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..c456482d7f23ddbe471aaea16b1a98e5add84810 --- /dev/null +++ b/modelcenter/community/deepset/roberta-base-squad2-distilled/introduction_cn.ipynb @@ -0,0 +1,98 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "85b7cc2e", + "metadata": {}, + "source": [ + "## Overview\n", + "\n", + "Language model: deepset/roberta-base-squad2-distilled\n", + "\n", + "Language: English\n", + "\n", + "Training data: SQuAD 2.0 training set Eval data: SQuAD 2.0 dev set Infrastructure: 4x V100 GPU\n", + "\n", + "Published: Dec 8th, 2021" + ] + }, + { + "cell_type": "markdown", + "id": "a455ff64", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d51fa907", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4590c7eb", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"deepset/roberta-base-squad2-distilled\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "ac6e34fd", + "metadata": {}, + "source": [ + "## Authors\n", + "- Timo Möller: `timo.moeller [at] deepset.ai`\n", + "- Julian Risch: `julian.risch [at] deepset.ai`\n", + "- Malte Pietsch: `malte.pietsch [at] deepset.ai`\n", + "- Michel Bartels: `michel.bartels [at] deepset.ai`\n", + "## About us\n", + "\n", + "We bring NLP to the industry via open source!\n", + "Our focus: Industry specific language models & large scale QA systems.\n" + ] + }, + { + "cell_type": "markdown", + "id": "3d22bf87", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/deepset/roberta-base-squad2-distilled](https://huggingface.co/deepset/roberta-base-squad2-distilled)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/deepset/roberta-base-squad2-distilled/introduction_en.ipynb b/modelcenter/community/deepset/roberta-base-squad2-distilled/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..fe7b0fa8216a4e048acb13216b0ba1d7e0e1363c --- /dev/null +++ b/modelcenter/community/deepset/roberta-base-squad2-distilled/introduction_en.ipynb @@ -0,0 +1,98 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "b917c220", + "metadata": {}, + "source": [ + "## Overview\n", + "\n", + "Language model: deepset/roberta-base-squad2-distilled\n", + "\n", + "Language: English\n", + "\n", + "Training data: SQuAD 2.0 training set Eval data: SQuAD 2.0 dev set Infrastructure: 4x V100 GPU\n", + "\n", + "Published: Dec 8th, 2021" + ] + }, + { + "cell_type": "markdown", + "id": "94e41c66", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8b2c9009", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b9472a8e", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"deepset/roberta-base-squad2-distilled\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "942ce61d", + "metadata": {}, + "source": [ + "## Authors\n", + "- Timo Möller: `timo.moeller [at] deepset.ai`\n", + "- Julian Risch: `julian.risch [at] deepset.ai`\n", + "- Malte Pietsch: `malte.pietsch [at] deepset.ai`\n", + "- Michel Bartels: `michel.bartels [at] deepset.ai`\n", + "## About us\n", + "\n", + "We bring NLP to the industry via open source!\n", + "Our focus: Industry specific language models & large scale QA systems.\n" + ] + }, + { + "cell_type": "markdown", + "id": "d65be46f", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/deepset/roberta-base-squad2-distilled](https://huggingface.co/deepset/roberta-base-squad2-distilled) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dslim/bert-base-NER/info.yaml b/modelcenter/community/dslim/bert-base-NER/info.yaml index 763642eb18d0b0b5cc0d87ae9be28b06d40e674d..29aea2ab9d6f208cdf160933c3831eb53801efc9 100644 --- a/modelcenter/community/dslim/bert-base-NER/info.yaml +++ b/modelcenter/community/dslim/bert-base-NER/info.yaml @@ -1,24 +1,21 @@ +Datasets: conll2003 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: mit Model_Info: - name: "dslim/bert-base-NER" - description: "bert-base-NER" - description_en: "bert-base-NER" - icon: "" - from_repo: "https://huggingface.co/dslim/bert-base-NER" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Token Classification" - sub_tag: "Token分类" - -Example: - -Datasets: "conll2003" -Publisher: "dslim" -License: "mit" -Language: "English" + description: bert-base-NER + description_en: bert-base-NER + from_repo: https://huggingface.co/dslim/bert-base-NER + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dslim/bert-base-NER Paper: - - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding' - url: 'http://arxiv.org/abs/1810.04805v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding' + url: http://arxiv.org/abs/1810.04805v2 +Publisher: dslim +Task: +- sub_tag: Token分类 + sub_tag_en: Token Classification + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dslim/bert-base-NER/introduction_cn.ipynb b/modelcenter/community/dslim/bert-base-NER/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..191633124b33be6710a112cb1980fe67e7ac3e10 --- /dev/null +++ b/modelcenter/community/dslim/bert-base-NER/introduction_cn.ipynb @@ -0,0 +1,125 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "4dd2d9a8", + "metadata": {}, + "source": [ + "# bert-base-NER\n" + ] + }, + { + "cell_type": "markdown", + "id": "39c0b4be", + "metadata": {}, + "source": [ + "## Model description\n" + ] + }, + { + "cell_type": "markdown", + "id": "0961b24f", + "metadata": {}, + "source": [ + "**bert-base-NER** is a fine-tuned BERT model that is ready to use for **Named Entity Recognition** and achieves **state-of-the-art performance** for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC).\n" + ] + }, + { + "cell_type": "markdown", + "id": "58641459", + "metadata": {}, + "source": [ + "Specifically, this model is a *bert-base-cased* model that was fine-tuned on the English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset.\n" + ] + }, + { + "cell_type": "markdown", + "id": "9da0ddda", + "metadata": {}, + "source": [ + "If you'd like to use a larger BERT-large model fine-tuned on the same dataset, a **bert-large-NER** version is also available.\n" + ] + }, + { + "cell_type": "markdown", + "id": "4d5adc68", + "metadata": {}, + "source": [ + "## How to use\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "88f3ea49", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "76ef1f0e", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dslim/bert-base-NER\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "137e381c", + "metadata": {}, + "source": [ + "## Citation\n", + "```\n", + "@inproceedings{tjong-kim-sang-de-meulder-2003-introduction,\n", + "title = \"Introduction to the {C}o{NLL}-2003 Shared Task: Language-Independent Named Entity Recognition\",\n", + "author = \"Tjong Kim Sang, Erik F. and\n", + "De Meulder, Fien\",\n", + "booktitle = \"Proceedings of the Seventh Conference on Natural Language Learning at {HLT}-{NAACL} 2003\",\n", + "year = \"2003\",\n", + "url = \"https://www.aclweb.org/anthology/W03-0419\",\n", + "pages = \"142--147\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3a632df2", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/dslim/bert-base-NER](https://huggingface.co/dslim/bert-base-NER)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dslim/bert-base-NER/introduction_en.ipynb b/modelcenter/community/dslim/bert-base-NER/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..67ed3a6ff04d56b05272420a88884aceb15b8d32 --- /dev/null +++ b/modelcenter/community/dslim/bert-base-NER/introduction_en.ipynb @@ -0,0 +1,126 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "c5180cf2", + "metadata": {}, + "source": [ + "# bert-base-NER\n" + ] + }, + { + "cell_type": "markdown", + "id": "dbf08fd8", + "metadata": {}, + "source": [ + "## Model description\n" + ] + }, + { + "cell_type": "markdown", + "id": "11690dda", + "metadata": {}, + "source": [ + "**bert-base-NER** is a fine-tuned BERT model that is ready to use for **Named Entity Recognition** and achieves **state-of-the-art performance** for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC).\n" + ] + }, + { + "cell_type": "markdown", + "id": "738f98db", + "metadata": {}, + "source": [ + "Specifically, this model is a *bert-base-cased* model that was fine-tuned on the English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset.\n" + ] + }, + { + "cell_type": "markdown", + "id": "03c5db03", + "metadata": {}, + "source": [ + "If you'd like to use a larger BERT-large model fine-tuned on the same dataset, a **bert-large-NER** version is also available.\n" + ] + }, + { + "cell_type": "markdown", + "id": "da040b29", + "metadata": {}, + "source": [ + "## How to use\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "726ee6e9", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "73564a0c", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dslim/bert-base-NER\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "c08bc233", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{tjong-kim-sang-de-meulder-2003-introduction,\n", + "title = \"Introduction to the {C}o{NLL}-2003 Shared Task: Language-Independent Named Entity Recognition\",\n", + "author = \"Tjong Kim Sang, Erik F. and\n", + "De Meulder, Fien\",\n", + "booktitle = \"Proceedings of the Seventh Conference on Natural Language Learning at {HLT}-{NAACL} 2003\",\n", + "year = \"2003\",\n", + "url = \"https://www.aclweb.org/anthology/W03-0419\",\n", + "pages = \"142--147\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "a56e1055", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dslim/bert-base-NER](https://huggingface.co/dslim/bert-base-NER) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dslim/bert-large-NER/info.yaml b/modelcenter/community/dslim/bert-large-NER/info.yaml index 48b413ddd9bbfb929a6a262f35f423daa2ccac84..64bdd181068e46e9d330c7d169fc8d84d47b034a 100644 --- a/modelcenter/community/dslim/bert-large-NER/info.yaml +++ b/modelcenter/community/dslim/bert-large-NER/info.yaml @@ -1,24 +1,21 @@ +Datasets: conll2003 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: mit Model_Info: - name: "dslim/bert-large-NER" - description: "bert-base-NER" - description_en: "bert-base-NER" - icon: "" - from_repo: "https://huggingface.co/dslim/bert-large-NER" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Token Classification" - sub_tag: "Token分类" - -Example: - -Datasets: "conll2003" -Publisher: "dslim" -License: "mit" -Language: "English" + description: bert-base-NER + description_en: bert-base-NER + from_repo: https://huggingface.co/dslim/bert-large-NER + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dslim/bert-large-NER Paper: - - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding' - url: 'http://arxiv.org/abs/1810.04805v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding' + url: http://arxiv.org/abs/1810.04805v2 +Publisher: dslim +Task: +- sub_tag: Token分类 + sub_tag_en: Token Classification + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dslim/bert-large-NER/introduction_cn.ipynb b/modelcenter/community/dslim/bert-large-NER/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..e33561abdcbc856b31f94dc09b9d10b210d5926f --- /dev/null +++ b/modelcenter/community/dslim/bert-large-NER/introduction_cn.ipynb @@ -0,0 +1,126 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "70a24d70", + "metadata": {}, + "source": [ + "# bert-base-NER\n" + ] + }, + { + "cell_type": "markdown", + "id": "72df2cc8", + "metadata": {}, + "source": [ + "## Model description\n" + ] + }, + { + "cell_type": "markdown", + "id": "d1aabc01", + "metadata": {}, + "source": [ + "**bert-large-NER** is a fine-tuned BERT model that is ready to use for **Named Entity Recognition** and achieves **state-of-the-art performance** for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC).\n" + ] + }, + { + "cell_type": "markdown", + "id": "2d53a70b", + "metadata": {}, + "source": [ + "Specifically, this model is a *bert-large-cased* model that was fine-tuned on the English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset.\n" + ] + }, + { + "cell_type": "markdown", + "id": "60c2ceb7", + "metadata": {}, + "source": [ + "If you'd like to use a smaller BERT model fine-tuned on the same dataset, a **bert-base-NER** version is also available.\n" + ] + }, + { + "cell_type": "markdown", + "id": "42d984a9", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9ef70bc3", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1177b32e", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dslim/bert-large-NER\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "353c5156", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{tjong-kim-sang-de-meulder-2003-introduction,\n", + "title = \"Introduction to the {C}o{NLL}-2003 Shared Task: Language-Independent Named Entity Recognition\",\n", + "author = \"Tjong Kim Sang, Erik F. and\n", + "De Meulder, Fien\",\n", + "booktitle = \"Proceedings of the Seventh Conference on Natural Language Learning at {HLT}-{NAACL} 2003\",\n", + "year = \"2003\",\n", + "url = \"https://www.aclweb.org/anthology/W03-0419\",\n", + "pages = \"142--147\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "5705ae48", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/dslim/bert-large-NER](https://huggingface.co/dslim/bert-large-NER)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dslim/bert-large-NER/introduction_en.ipynb b/modelcenter/community/dslim/bert-large-NER/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..047acf0a727a07f9381bdce72626977a566694cf --- /dev/null +++ b/modelcenter/community/dslim/bert-large-NER/introduction_en.ipynb @@ -0,0 +1,126 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "574c41aa", + "metadata": {}, + "source": [ + "# bert-base-NER\n" + ] + }, + { + "cell_type": "markdown", + "id": "430be48c", + "metadata": {}, + "source": [ + "## Model description\n" + ] + }, + { + "cell_type": "markdown", + "id": "4bdfd881", + "metadata": {}, + "source": [ + "**bert-large-NER** is a fine-tuned BERT model that is ready to use for **Named Entity Recognition** and achieves **state-of-the-art performance** for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC).\n" + ] + }, + { + "cell_type": "markdown", + "id": "84ed4f8f", + "metadata": {}, + "source": [ + "Specifically, this model is a *bert-large-cased* model that was fine-tuned on the English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset.\n" + ] + }, + { + "cell_type": "markdown", + "id": "b4acb8a4", + "metadata": {}, + "source": [ + "If you'd like to use a smaller BERT model fine-tuned on the same dataset, a **bert-base-NER** version is also available.\n" + ] + }, + { + "cell_type": "markdown", + "id": "46ad4f1f", + "metadata": {}, + "source": [ + "## How to use\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b7e64545", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "380aa0dc", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dslim/bert-large-NER\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "220eb907", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{tjong-kim-sang-de-meulder-2003-introduction,\n", + "title = \"Introduction to the {C}o{NLL}-2003 Shared Task: Language-Independent Named Entity Recognition\",\n", + "author = \"Tjong Kim Sang, Erik F. and\n", + "De Meulder, Fien\",\n", + "booktitle = \"Proceedings of the Seventh Conference on Natural Language Learning at {HLT}-{NAACL} 2003\",\n", + "year = \"2003\",\n", + "url = \"https://www.aclweb.org/anthology/W03-0419\",\n", + "pages = \"142--147\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "dc0b9503", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dslim/bert-large-NER](https://huggingface.co/dslim/bert-large-NER) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/info.yaml b/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/info.yaml index 04a33129d0d954d2340dd7ca34d193b7350b88d2..74203f9330466fee14a74ad6785fb00ffff8d4b4 100644 --- a/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/info.yaml +++ b/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/info.yaml @@ -1,23 +1,19 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Romanian +License: mit Model_Info: - name: "dumitrescustefan/bert-base-romanian-cased-v1" - description: "bert-base-romanian-cased-v1" - description_en: "bert-base-romanian-cased-v1" - icon: "" - from_repo: "https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1" - + description: bert-base-romanian-cased-v1 + description_en: bert-base-romanian-cased-v1 + from_repo: https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1 + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dumitrescustefan/bert-base-romanian-cased-v1 +Paper: null +Publisher: dumitrescustefan Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "dumitrescustefan" -License: "mit" -Language: "Romanian" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/introduction_cn.ipynb b/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..cc2fc93e540a125c3b72b4b31ca2bfae6dad70d3 --- /dev/null +++ b/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/introduction_cn.ipynb @@ -0,0 +1,116 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "2a485f7a", + "metadata": {}, + "source": [ + "# bert-base-romanian-cased-v1\n" + ] + }, + { + "cell_type": "markdown", + "id": "5f911938", + "metadata": {}, + "source": [ + "The BERT **base**, **cased** model for Romanian, trained on a 15GB corpus.\n" + ] + }, + { + "cell_type": "markdown", + "id": "e2cccf2e", + "metadata": {}, + "source": [ + "### How to use\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "64b86b26", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f84498aa", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dumitrescustefan/bert-base-romanian-cased-v1\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "40176abc", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{dumitrescu-etal-2020-birth,\n", + "title = \"The birth of {R}omanian {BERT}\",\n", + "author = \"Dumitrescu, Stefan and\n", + "Avram, Andrei-Marius and\n", + "Pyysalo, Sampo\",\n", + "booktitle = \"Findings of the Association for Computational Linguistics: EMNLP 2020\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://aclanthology.org/2020.findings-emnlp.387\",\n", + "doi = \"10.18653/v1/2020.findings-emnlp.387\",\n", + "pages = \"4324--4328\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "550b09f3", + "metadata": {}, + "source": [ + "#### Acknowledgements\n" + ] + }, + { + "cell_type": "markdown", + "id": "fda88b7c", + "metadata": {}, + "source": [ + "- We'd like to thank [Sampo Pyysalo](https://github.com/spyysalo) from TurkuNLP for helping us out with the compute needed to pretrain the v1.0 BERT models. He's awesome!\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1](https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/introduction_en.ipynb b/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..be66e19cc0cd954493b98a4153e1887dfe3fdd52 --- /dev/null +++ b/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/introduction_en.ipynb @@ -0,0 +1,116 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "28a330b8", + "metadata": {}, + "source": [ + "# bert-base-romanian-cased-v1\n" + ] + }, + { + "cell_type": "markdown", + "id": "36f0d74f", + "metadata": {}, + "source": [ + "The BERT **base**, **cased** model for Romanian, trained on a 15GB corpus." + ] + }, + { + "cell_type": "markdown", + "id": "0104e14e", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b4ca4271", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9f3ca553", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dumitrescustefan/bert-base-romanian-cased-v1\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "51754d3f", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{dumitrescu-etal-2020-birth,\n", + "title = \"The birth of {R}omanian {BERT}\",\n", + "author = \"Dumitrescu, Stefan and\n", + "Avram, Andrei-Marius and\n", + "Pyysalo, Sampo\",\n", + "booktitle = \"Findings of the Association for Computational Linguistics: EMNLP 2020\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://aclanthology.org/2020.findings-emnlp.387\",\n", + "doi = \"10.18653/v1/2020.findings-emnlp.387\",\n", + "pages = \"4324--4328\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "2143146f", + "metadata": {}, + "source": [ + "#### Acknowledgements\n" + ] + }, + { + "cell_type": "markdown", + "id": "d983ac22", + "metadata": {}, + "source": [ + "- We'd like to thank [Sampo Pyysalo](https://github.com/spyysalo) from TurkuNLP for helping us out with the compute needed to pretrain the v1.0 BERT models. He's awesome!\n", + "> The model introduction and model weights originate from [https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1](https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/info.yaml b/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/info.yaml index f307e47f29bbe9143551e1d773302b3ca84159ff..05f2d42a81b597beccae2f4b070eda57fdf99613 100644 --- a/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/info.yaml +++ b/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/info.yaml @@ -1,23 +1,19 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Romanian +License: mit Model_Info: - name: "dumitrescustefan/bert-base-romanian-uncased-v1" - description: "bert-base-romanian-uncased-v1" - description_en: "bert-base-romanian-uncased-v1" - icon: "" - from_repo: "https://huggingface.co/dumitrescustefan/bert-base-romanian-uncased-v1" - + description: bert-base-romanian-uncased-v1 + description_en: bert-base-romanian-uncased-v1 + from_repo: https://huggingface.co/dumitrescustefan/bert-base-romanian-uncased-v1 + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dumitrescustefan/bert-base-romanian-uncased-v1 +Paper: null +Publisher: dumitrescustefan Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "dumitrescustefan" -License: "mit" -Language: "Romanian" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/introduction_cn.ipynb b/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..d4de60883e281cffad060f5c863604f3dd033f0a --- /dev/null +++ b/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/introduction_cn.ipynb @@ -0,0 +1,116 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "922f44e2", + "metadata": {}, + "source": [ + "# bert-base-romanian-uncased-v1\n" + ] + }, + { + "cell_type": "markdown", + "id": "2f5259bd", + "metadata": {}, + "source": [ + "The BERT **base**, **uncased** model for Romanian, trained on a 15GB corpus, version \n" + ] + }, + { + "cell_type": "markdown", + "id": "408f4468", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "acd14372", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cc5d539c", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dumitrescustefan/bert-base-romanian-uncased-v1\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "adbbab44", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{dumitrescu-etal-2020-birth,\n", + "title = \"The birth of {R}omanian {BERT}\",\n", + "author = \"Dumitrescu, Stefan and\n", + "Avram, Andrei-Marius and\n", + "Pyysalo, Sampo\",\n", + "booktitle = \"Findings of the Association for Computational Linguistics: EMNLP 2020\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://aclanthology.org/2020.findings-emnlp.387\",\n", + "doi = \"10.18653/v1/2020.findings-emnlp.387\",\n", + "pages = \"4324--4328\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "4276651e", + "metadata": {}, + "source": [ + "#### Acknowledgements\n" + ] + }, + { + "cell_type": "markdown", + "id": "84a91796", + "metadata": {}, + "source": [ + "- We'd like to thank [Sampo Pyysalo](https://github.com/spyysalo) from TurkuNLP for helping us out with the compute needed to pretrain the v1.0 BERT models. He's awesome!\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dumitrescustefan/bert-base-romanian-uncased-v1](https://huggingface.co/dumitrescustefan/bert-base-romanian-uncased-v1)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/introduction_en.ipynb b/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..ca76cb10fb0800534f1b6dcc3a3e9f8e7fb499d8 --- /dev/null +++ b/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/introduction_en.ipynb @@ -0,0 +1,116 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "cf268d0f", + "metadata": {}, + "source": [ + "# bert-base-romanian-uncased-v1\n" + ] + }, + { + "cell_type": "markdown", + "id": "453405af", + "metadata": {}, + "source": [ + "The BERT **base**, **uncased** model for Romanian, trained on a 15GB corpus, version \n" + ] + }, + { + "cell_type": "markdown", + "id": "4100824e", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cd182732", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1c16cd09", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dumitrescustefan/bert-base-romanian-uncased-v1\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "ba32e8ff", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{dumitrescu-etal-2020-birth,\n", + "title = \"The birth of {R}omanian {BERT}\",\n", + "author = \"Dumitrescu, Stefan and\n", + "Avram, Andrei-Marius and\n", + "Pyysalo, Sampo\",\n", + "booktitle = \"Findings of the Association for Computational Linguistics: EMNLP 2020\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://aclanthology.org/2020.findings-emnlp.387\",\n", + "doi = \"10.18653/v1/2020.findings-emnlp.387\",\n", + "pages = \"4324--4328\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "faa200c7", + "metadata": {}, + "source": [ + "#### Acknowledgements\n" + ] + }, + { + "cell_type": "markdown", + "id": "cb74a943", + "metadata": {}, + "source": [ + "- We'd like to thank [Sampo Pyysalo](https://github.com/spyysalo) from TurkuNLP for helping us out with the compute needed to pretrain the v1.0 BERT models. He's awesome!\n", + "> The model introduction and model weights originate from [https://huggingface.co/dumitrescustefan/bert-base-romanian-uncased-v1](https://huggingface.co/dumitrescustefan/bert-base-romanian-uncased-v1) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/info.yaml b/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/info.yaml index ea7b3848812d15c141bedd9c4b7d9110a615f91b..a0fd84b93ad01547c1aeaa438035ef6a03423e1b 100644 --- a/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/info.yaml +++ b/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/info.yaml @@ -1,26 +1,24 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: mit Model_Info: - name: "emilyalsentzer/Bio_ClinicalBERT" - description: "ClinicalBERT - Bio + Clinical BERT Model" - description_en: "ClinicalBERT - Bio + Clinical BERT Model" - icon: "" - from_repo: "https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "emilyalsentzer" -License: "mit" -Language: "English" + description: ClinicalBERT - Bio + Clinical BERT Model + description_en: ClinicalBERT - Bio + Clinical BERT Model + from_repo: https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: emilyalsentzer/Bio_ClinicalBERT Paper: - - title: 'Publicly Available Clinical BERT Embeddings' - url: 'http://arxiv.org/abs/1904.03323v3' - - title: 'BioBERT: a pre-trained biomedical language representation model for biomedical text mining' - url: 'http://arxiv.org/abs/1901.08746v4' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Publicly Available Clinical BERT Embeddings + url: http://arxiv.org/abs/1904.03323v3 +- title: 'BioBERT: a pre-trained biomedical language representation model for biomedical + text mining' + url: http://arxiv.org/abs/1901.08746v4 +Publisher: emilyalsentzer +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/introduction_cn.ipynb b/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..7064ebfbc0cf914da1739fb2647defb91a11c2d0 --- /dev/null +++ b/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/introduction_cn.ipynb @@ -0,0 +1,90 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "22b0e4db", + "metadata": {}, + "source": [ + "# ClinicalBERT - Bio + Clinical BERT Model\n" + ] + }, + { + "cell_type": "markdown", + "id": "f9d9ac37", + "metadata": {}, + "source": [ + "The [Publicly Available Clinical BERT Embeddings](https://arxiv.org/abs/1904.03323) paper contains four unique clinicalBERT models: initialized with BERT-Base (`cased_L-12_H-768_A-12`) or BioBERT (`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`) & trained on either all MIMIC notes or only discharge summaries.\n" + ] + }, + { + "cell_type": "markdown", + "id": "24aaa0b1", + "metadata": {}, + "source": [ + "This model card describes the Bio+Clinical BERT model, which was initialized from [BioBERT](https://arxiv.org/abs/1901.08746) & trained on all MIMIC notes.\n" + ] + }, + { + "cell_type": "markdown", + "id": "1449fef2", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "be5241ea", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f4c3cf6f", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"emilyalsentzer/Bio_ClinicalBERT\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "451e4ff6", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT](https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/introduction_en.ipynb b/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..fab51acf8dcc32f955310509b092a8d228657c83 --- /dev/null +++ b/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/introduction_en.ipynb @@ -0,0 +1,90 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "91c5f94f", + "metadata": {}, + "source": [ + "# ClinicalBERT - Bio + Clinical BERT Model\n" + ] + }, + { + "cell_type": "markdown", + "id": "ec471b16", + "metadata": {}, + "source": [ + "The [Publicly Available Clinical BERT Embeddings](https://arxiv.org/abs/1904.03323) paper contains four unique clinicalBERT models: initialized with BERT-Base (`cased_L-12_H-768_A-12`) or BioBERT (`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`) & trained on either all MIMIC notes or only discharge summaries.\n" + ] + }, + { + "cell_type": "markdown", + "id": "9ab166b8", + "metadata": {}, + "source": [ + "This model card describes the Bio+Clinical BERT model, which was initialized from [BioBERT](https://arxiv.org/abs/1901.08746) & trained on all MIMIC notes.\n" + ] + }, + { + "cell_type": "markdown", + "id": "69f6ed08", + "metadata": {}, + "source": [ + "## How to use the model\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "62913fa8", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7b055241", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"emilyalsentzer/Bio_ClinicalBERT\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "0716a06f", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT](https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/info.yaml b/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/info.yaml index 3f91657ab383fffd105c8fdf0d4d76f336df5689..34d508f8f54e1abbe0a2d7337354d4bb44ea7578 100644 --- a/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/info.yaml +++ b/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/info.yaml @@ -1,26 +1,24 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: mit Model_Info: - name: "emilyalsentzer/Bio_Discharge_Summary_BERT" - description: "ClinicalBERT - Bio + Discharge Summary BERT Model" - description_en: "ClinicalBERT - Bio + Discharge Summary BERT Model" - icon: "" - from_repo: "https://huggingface.co/emilyalsentzer/Bio_Discharge_Summary_BERT" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "emilyalsentzer" -License: "mit" -Language: "English" + description: ClinicalBERT - Bio + Discharge Summary BERT Model + description_en: ClinicalBERT - Bio + Discharge Summary BERT Model + from_repo: https://huggingface.co/emilyalsentzer/Bio_Discharge_Summary_BERT + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: emilyalsentzer/Bio_Discharge_Summary_BERT Paper: - - title: 'Publicly Available Clinical BERT Embeddings' - url: 'http://arxiv.org/abs/1904.03323v3' - - title: 'BioBERT: a pre-trained biomedical language representation model for biomedical text mining' - url: 'http://arxiv.org/abs/1901.08746v4' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Publicly Available Clinical BERT Embeddings + url: http://arxiv.org/abs/1904.03323v3 +- title: 'BioBERT: a pre-trained biomedical language representation model for biomedical + text mining' + url: http://arxiv.org/abs/1901.08746v4 +Publisher: emilyalsentzer +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/introduction_cn.ipynb b/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..3c3fd2a4660427e5bca19c69e92fa54c37d8704c --- /dev/null +++ b/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/introduction_cn.ipynb @@ -0,0 +1,90 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "67503ba7", + "metadata": {}, + "source": [ + "# ClinicalBERT - Bio + Discharge Summary BERT Model\n" + ] + }, + { + "cell_type": "markdown", + "id": "e2d38260", + "metadata": {}, + "source": [ + "The [Publicly Available Clinical BERT Embeddings](https://arxiv.org/abs/1904.03323) paper contains four unique clinicalBERT models: initialized with BERT-Base (`cased_L-12_H-768_A-12`) or BioBERT (`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`) & trained on either all MIMIC notes or only discharge summaries.\n" + ] + }, + { + "cell_type": "markdown", + "id": "1c92755e", + "metadata": {}, + "source": [ + "This model card describes the Bio+Discharge Summary BERT model, which was initialized from [BioBERT](https://arxiv.org/abs/1901.08746) & trained on only discharge summaries from MIMIC.\n" + ] + }, + { + "cell_type": "markdown", + "id": "068ba168", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "24e8b203", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4bcd1b84", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"emilyalsentzer/Bio_Discharge_Summary_BERT\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "0cebe09b", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/emilyalsentzer/Bio_Discharge_Summary_BERT](https://huggingface.co/emilyalsentzer/Bio_Discharge_Summary_BERT)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/introduction_en.ipynb b/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..f879e8224776c3d963d7894b2862008e7ca22841 --- /dev/null +++ b/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/introduction_en.ipynb @@ -0,0 +1,90 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a786c8f0", + "metadata": {}, + "source": [ + "# ClinicalBERT - Bio + Discharge Summary BERT Model\n" + ] + }, + { + "cell_type": "markdown", + "id": "4d8e4f1f", + "metadata": {}, + "source": [ + "The [Publicly Available Clinical BERT Embeddings](https://arxiv.org/abs/1904.03323) paper contains four unique clinicalBERT models: initialized with BERT-Base (`cased_L-12_H-768_A-12`) or BioBERT (`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`) & trained on either all MIMIC notes or only discharge summaries.\n" + ] + }, + { + "cell_type": "markdown", + "id": "83bf8287", + "metadata": {}, + "source": [ + "This model card describes the Bio+Discharge Summary BERT model, which was initialized from [BioBERT](https://arxiv.org/abs/1901.08746) & trained on only discharge summaries from MIMIC.\n" + ] + }, + { + "cell_type": "markdown", + "id": "ee7d03ef", + "metadata": {}, + "source": [ + "## How to use the model\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3ef75bb2", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c04f99b3", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"emilyalsentzer/Bio_Discharge_Summary_BERT\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "e4459a1c", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/emilyalsentzer/Bio_Discharge_Summary_BERT](https://huggingface.co/emilyalsentzer/Bio_Discharge_Summary_BERT) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-base-lm-adapt/info.yaml b/modelcenter/community/google/t5-base-lm-adapt/info.yaml index 90a398a8481eefd839f9de0375ef5ff1998e2090..7afa8d89790305b2d852485a7d9b4507bc07137e 100644 --- a/modelcenter/community/google/t5-base-lm-adapt/info.yaml +++ b/modelcenter/community/google/t5-base-lm-adapt/info.yaml @@ -1,26 +1,23 @@ +Datasets: c4 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "google/t5-base-lm-adapt" - description: "Version 1.1 - LM-Adapted" - description_en: "Version 1.1 - LM-Adapted" - icon: "" - from_repo: "https://huggingface.co/google/t5-base-lm-adapt" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4" -Publisher: "google" -License: "apache-2.0" -Language: "English" + description: Version 1.1 - LM-Adapted + description_en: Version 1.1 - LM-Adapted + from_repo: https://huggingface.co/google/t5-base-lm-adapt + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: google/t5-base-lm-adapt Paper: - - title: 'GLU Variants Improve Transformer' - url: 'http://arxiv.org/abs/2002.05202v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: GLU Variants Improve Transformer + url: http://arxiv.org/abs/2002.05202v1 +- title: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer + url: http://arxiv.org/abs/1910.10683v3 +Publisher: google +Task: +- sub_tag: 文本生成 + sub_tag_en: Text2Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/google/t5-base-lm-adapt/introduction_cn.ipynb b/modelcenter/community/google/t5-base-lm-adapt/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..8e4a107f98c9710ec2d8a01652ef5bab7e90efa5 --- /dev/null +++ b/modelcenter/community/google/t5-base-lm-adapt/introduction_cn.ipynb @@ -0,0 +1,102 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "c59ed826", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1 - LM-Adapted\n", + "\n", + "\n", + "## Version 1.1 - LM-Adapted\n", + "\n", + "[T5 Version 1.1 - LM Adapted](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k) includes the following improvements compared to the original T5 model:\n", + "\n", + "- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "and is pretrained on both the denoising and language modeling objective.\n", + "\n", + "More specifically, this checkpoint is initialized from T5 Version 1.1 - Base\n", + "and then trained for an additional 100K steps on the LM objective discussed in the [T5 paper](https://arxiv.org/pdf/1910.10683.pdf).\n", + "This adaptation improves the ability of the model to be used for prompt tuning.\n", + "\n", + "**Note**: A popular fine-tuned version of the *T5 Version 1.1 - LM Adapted* model is BigScience's T0pp.\n", + "\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6db9a194", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b70ecb24", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-base-lm-adapt\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "46aee335", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/google/t5-base-lm-adapt](https://huggingface.co/google/t5-base-lm-adapt)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-base-lm-adapt/introduction_en.ipynb b/modelcenter/community/google/t5-base-lm-adapt/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..c1a177787bace69a5b7f32d56c47dd73fcaaed08 --- /dev/null +++ b/modelcenter/community/google/t5-base-lm-adapt/introduction_en.ipynb @@ -0,0 +1,103 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "35f226d7", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1 - LM-Adapted\n", + "\n", + "\n", + "## Version 1.1 - LM-Adapted\n", + "\n", + "[T5 Version 1.1 - LM Adapted](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k) includes the following improvements compared to the original T5 model:\n", + "\n", + "- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "and is pretrained on both the denoising and language modeling objective.\n", + "\n", + "More specifically, this checkpoint is initialized from T5 Version 1.1 - Base\n", + "and then trained for an additional 100K steps on the LM objective discussed in the [T5 paper](https://arxiv.org/pdf/1910.10683.pdf).\n", + "This adaptation improves the ability of the model to be used for prompt tuning.\n", + "\n", + "**Note**: A popular fine-tuned version of the *T5 Version 1.1 - LM Adapted* model is BigScience's T0pp.\n", + "\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b471855d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f74ec3ef", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-base-lm-adapt\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "e431d080", + "metadata": {}, + "source": [ + "\n", + "> The model introduction and model weights originate from [https://huggingface.co/google/t5-base-lm-adapt](https://huggingface.co/google/t5-base-lm-adapt) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-large-lm-adapt/info.yaml b/modelcenter/community/google/t5-large-lm-adapt/info.yaml index db5c0320439bf643bb37b8cd991f703f26a824ee..417bbe7127babac160861022f0ab34fb372992e1 100644 --- a/modelcenter/community/google/t5-large-lm-adapt/info.yaml +++ b/modelcenter/community/google/t5-large-lm-adapt/info.yaml @@ -1,26 +1,23 @@ +Datasets: c4 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "google/t5-large-lm-adapt" - description: "Version 1.1 - LM-Adapted" - description_en: "Version 1.1 - LM-Adapted" - icon: "" - from_repo: "https://huggingface.co/google/t5-large-lm-adapt" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4" -Publisher: "google" -License: "apache-2.0" -Language: "English" + description: Version 1.1 - LM-Adapted + description_en: Version 1.1 - LM-Adapted + from_repo: https://huggingface.co/google/t5-large-lm-adapt + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: google/t5-large-lm-adapt Paper: - - title: 'GLU Variants Improve Transformer' - url: 'http://arxiv.org/abs/2002.05202v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: GLU Variants Improve Transformer + url: http://arxiv.org/abs/2002.05202v1 +- title: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer + url: http://arxiv.org/abs/1910.10683v3 +Publisher: google +Task: +- sub_tag: 文本生成 + sub_tag_en: Text2Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/google/t5-large-lm-adapt/introduction_cn.ipynb b/modelcenter/community/google/t5-large-lm-adapt/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..ce6a76d6967ca309d287645ba3c89fcce983e823 --- /dev/null +++ b/modelcenter/community/google/t5-large-lm-adapt/introduction_cn.ipynb @@ -0,0 +1,111 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "71352026", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1 - LM-Adapted\n", + "\n", + "\n", + "## Version 1.1 - LM-Adapted\n", + "\n", + "[T5 Version 1.1 - LM Adapted](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k) includes the following improvements compared to the original [T5 model](https://huggingface.co/t5-large):\n", + "\n", + "- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "and is pretrained on both the denoising and language modeling objective.\n", + "\n", + "More specifically, this checkpoint is initialized from T5 Version 1.1 - Large\n", + "and then trained for an additional 100K steps on the LM objective discussed in the [T5 paper](https://arxiv.org/pdf/1910.10683.pdf).\n", + "This adaptation improves the ability of the model to be used for prompt tuning.\n", + "\n", + "**Note**: A popular fine-tuned version of the *T5 Version 1.1 - LM Adapted* model is BigScience's T0pp.\n", + "\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "d41870db", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b9451fd3", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6392031c", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-large-lm-adapt\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "3b35551f", + "metadata": {}, + "source": [ + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/google/t5-large-lm-adapt](https://huggingface.co/google/t5-large-lm-adapt)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-large-lm-adapt/introduction_en.ipynb b/modelcenter/community/google/t5-large-lm-adapt/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..fdc71cd138926da6979d64bad9d23f67d5914135 --- /dev/null +++ b/modelcenter/community/google/t5-large-lm-adapt/introduction_en.ipynb @@ -0,0 +1,111 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "70cb903f", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1 - LM-Adapted\n", + "\n", + "\n", + "## Version 1.1 - LM-Adapted\n", + "\n", + "[T5 Version 1.1 - LM Adapted](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k) includes the following improvements compared to the original [T5 model](https://huggingface.co/t5-large):\n", + "\n", + "- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "and is pretrained on both the denoising and language modeling objective.\n", + "\n", + "More specifically, this checkpoint is initialized from T5 Version 1.1 - Large\n", + "and then trained for an additional 100K steps on the LM objective discussed in the [T5 paper](https://arxiv.org/pdf/1910.10683.pdf).\n", + "This adaptation improves the ability of the model to be used for prompt tuning.\n", + "\n", + "**Note**: A popular fine-tuned version of the *T5 Version 1.1 - LM Adapted* model is BigScience's T0pp.\n", + "\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "cab0c1ea", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e37976b5", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0673661a", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-large-lm-adapt\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "7b24e77f", + "metadata": {}, + "source": [ + "\n", + "> The model introduction and model weights originate from [https://huggingface.co/google/t5-large-lm-adapt](https://huggingface.co/google/t5-large-lm-adapt) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-large-ssm/download_cn.md b/modelcenter/community/google/t5-large-ssm/download_cn.md deleted file mode 100644 index 344edafe14bbc236e03312ce3492c28459055d00..0000000000000000000000000000000000000000 --- a/modelcenter/community/google/t5-large-ssm/download_cn.md +++ /dev/null @@ -1,23 +0,0 @@ -# 模型列表 - -## google/t5-large-ssm - -| 模型名称 | 模型介绍 | 模型大小 | 模型下载 | -| --- | --- | --- | --- | -|google/t5-large-ssm| | 3.12G | [model_config.json](https://bj.bcebos.com/paddlenlp/models/community/google/t5-large-ssm/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/google/t5-large-ssm/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/google/t5-large-ssm/tokenizer_config.json) | - -也可以通过`paddlenlp` cli 工具来下载对应的模型权重，使用步骤如下所示： - -* 安装paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* 下载命令行 - -```shell -paddlenlp download --cache-dir ./pretrained_models google/t5-large-ssm -``` - -有任何下载的问题都可以到[PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP)中发Issue提问。 \ No newline at end of file diff --git a/modelcenter/community/google/t5-large-ssm/download_en.md b/modelcenter/community/google/t5-large-ssm/download_en.md deleted file mode 100644 index 000055d34d5df923908f515a46d93d5435b8450f..0000000000000000000000000000000000000000 --- a/modelcenter/community/google/t5-large-ssm/download_en.md +++ /dev/null @@ -1,23 +0,0 @@ -# model list - -## - -| model | description | model_size | download | -| --- | --- | --- | --- | -|google/t5-large-ssm| | 3.12G | [model_config.json](https://bj.bcebos.com/paddlenlp/models/community/google/t5-large-ssm/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/google/t5-large-ssm/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/google/t5-large-ssm/tokenizer_config.json) | - -or you can download all of model file with the following steps: - -* install paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* download model with cli tool - -```shell -paddlenlp download --cache-dir ./pretrained_models google/t5-large-ssm -``` - -If you have any problems with it, you can post issue on [PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP) to get support. diff --git a/modelcenter/community/google/t5-large-ssm/info.yaml b/modelcenter/community/google/t5-large-ssm/info.yaml deleted file mode 100644 index 5a7cafb195e355c6db8872c1adabc6e7cca651ef..0000000000000000000000000000000000000000 --- a/modelcenter/community/google/t5-large-ssm/info.yaml +++ /dev/null @@ -1,26 +0,0 @@ -Model_Info: - name: "google/t5-large-ssm" - description: "Abstract" - description_en: "Abstract" - icon: "" - from_repo: "https://huggingface.co/google/t5-large-ssm" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4,wikipedia" -Publisher: "google" -License: "apache-2.0" -Language: "English" -Paper: - - title: 'REALM: Retrieval-Augmented Language Model Pre-Training' - url: 'http://arxiv.org/abs/2002.08909v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file diff --git a/modelcenter/community/google/t5-small-lm-adapt/info.yaml b/modelcenter/community/google/t5-small-lm-adapt/info.yaml index db6f3a7e6d3dc2d002dfff2b9aae9b503704646e..9e9b014a9865efbe406d6d3e296d30587e1779dc 100644 --- a/modelcenter/community/google/t5-small-lm-adapt/info.yaml +++ b/modelcenter/community/google/t5-small-lm-adapt/info.yaml @@ -1,26 +1,23 @@ +Datasets: c4 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "google/t5-small-lm-adapt" - description: "Version 1.1 - LM-Adapted" - description_en: "Version 1.1 - LM-Adapted" - icon: "" - from_repo: "https://huggingface.co/google/t5-small-lm-adapt" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4" -Publisher: "google" -License: "apache-2.0" -Language: "English" + description: Version 1.1 - LM-Adapted + description_en: Version 1.1 - LM-Adapted + from_repo: https://huggingface.co/google/t5-small-lm-adapt + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: google/t5-small-lm-adapt Paper: - - title: 'GLU Variants Improve Transformer' - url: 'http://arxiv.org/abs/2002.05202v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: GLU Variants Improve Transformer + url: http://arxiv.org/abs/2002.05202v1 +- title: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer + url: http://arxiv.org/abs/1910.10683v3 +Publisher: google +Task: +- sub_tag: 文本生成 + sub_tag_en: Text2Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/google/t5-small-lm-adapt/introduction_cn.ipynb b/modelcenter/community/google/t5-small-lm-adapt/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..641edd7a0924ca16d7d1b2fa9ba620ed8e1cf799 --- /dev/null +++ b/modelcenter/community/google/t5-small-lm-adapt/introduction_cn.ipynb @@ -0,0 +1,111 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "9dca1445", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1 - LM-Adapted\n", + "\n", + "\n", + "## Version 1.1 - LM-Adapted\n", + "\n", + "[T5 Version 1.1 - LM Adapted](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k) includes the following improvements compared to the original [T5 model](https://huggingface.co/t5-small):\n", + "\n", + "- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "and is pretrained on both the denoising and language modeling objective.\n", + "\n", + "More specifically, this checkpoint is initialized from T5 Version 1.1 - Small\n", + "and then trained for an additional 100K steps on the LM objective discussed in the [T5 paper](https://arxiv.org/pdf/1910.10683.pdf).\n", + "This adaptation improves the ability of the model to be used for prompt tuning.\n", + "\n", + "**Note**: A popular fine-tuned version of the *T5 Version 1.1 - LM Adapted* model is BigScience's T0pp.\n", + "\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "4c63de98", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8030fcb4", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8f6f14dd", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-small-lm-adapt\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "b8dd698b", + "metadata": {}, + "source": [ + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/google/t5-small-lm-adapt](https://huggingface.co/google/t5-small-lm-adapt)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-small-lm-adapt/introduction_en.ipynb b/modelcenter/community/google/t5-small-lm-adapt/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..21c1e824d139596eb4f84f841e1c1414eca2f1cc --- /dev/null +++ b/modelcenter/community/google/t5-small-lm-adapt/introduction_en.ipynb @@ -0,0 +1,111 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "42de6200", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1 - LM-Adapted\n", + "\n", + "\n", + "## Version 1.1 - LM-Adapted\n", + "\n", + "[T5 Version 1.1 - LM Adapted](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k) includes the following improvements compared to the original [T5 model](https://huggingface.co/t5-small):\n", + "\n", + "- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "and is pretrained on both the denoising and language modeling objective.\n", + "\n", + "More specifically, this checkpoint is initialized from [T5 Version 1.1 - Small](https://huggingface.co/google/https://huggingface.co/google/t5-v1_1-small)\n", + "and then trained for an additional 100K steps on the LM objective discussed in the [T5 paper](https://arxiv.org/pdf/1910.10683.pdf).\n", + "This adaptation improves the ability of the model to be used for prompt tuning.\n", + "\n", + "**Note**: A popular fine-tuned version of the *T5 Version 1.1 - LM Adapted* model is BigScience's T0pp.\n", + "\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "39071317", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "31a774de", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a1cd7a50", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-small-lm-adapt\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "4d283d3d", + "metadata": {}, + "source": [ + "\n", + "> The model introduction and model weights originate from [https://huggingface.co/google/t5-small-lm-adapt](https://huggingface.co/google/t5-small-lm-adapt) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-v1_1-base/info.yaml b/modelcenter/community/google/t5-v1_1-base/info.yaml index 3cedf8d0e891e3b9d8b9bb3f14aae7c4ad58efd0..971ea7d892c3a968cc05af7d109ab635076b8f00 100644 --- a/modelcenter/community/google/t5-v1_1-base/info.yaml +++ b/modelcenter/community/google/t5-v1_1-base/info.yaml @@ -1,26 +1,23 @@ +Datasets: c4 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "google/t5-v1_1-base" - description: "Version 1.1" - description_en: "Version 1.1" - icon: "" - from_repo: "https://huggingface.co/google/t5-v1_1-base" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4" -Publisher: "google" -License: "apache-2.0" -Language: "English" + description: Version 1.1 + description_en: Version 1.1 + from_repo: https://huggingface.co/google/t5-v1_1-base + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: google/t5-v1_1-base Paper: - - title: 'GLU Variants Improve Transformer' - url: 'http://arxiv.org/abs/2002.05202v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: GLU Variants Improve Transformer + url: http://arxiv.org/abs/2002.05202v1 +- title: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer + url: http://arxiv.org/abs/1910.10683v3 +Publisher: google +Task: +- sub_tag: 文本生成 + sub_tag_en: Text2Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/google/t5-v1_1-base/introduction_cn.ipynb b/modelcenter/community/google/t5-v1_1-base/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..fb4d969aa56526b40ede0b6a47b812304e514db9 --- /dev/null +++ b/modelcenter/community/google/t5-v1_1-base/introduction_cn.ipynb @@ -0,0 +1,101 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "83a0cdfd", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1\n", + "\n", + "\n", + "## Version 1.1\n", + "\n", + "[T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "**Note**: T5 Version 1.1 was only pre-trained on C4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task.\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "6196f74c", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6c13cb82", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1d62f626", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-v1_1-base\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "016545f2", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/google/t5-v1_1-base](https://huggingface.co/google/t5-v1_1-base)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-v1_1-base/introduction_en.ipynb b/modelcenter/community/google/t5-v1_1-base/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..4084d70fab28230aacbe2d3576f7c70af1c097af --- /dev/null +++ b/modelcenter/community/google/t5-v1_1-base/introduction_en.ipynb @@ -0,0 +1,101 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "2656a571", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1\n", + "\n", + "\n", + "## Version 1.1\n", + "\n", + "[T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "**Note**: T5 Version 1.1 was only pre-trained on C4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task.\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "c0cd9f02", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9323615d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0b9994b3", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-v1_1-base\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "8daa264b", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/google/t5-v1_1-base](https://huggingface.co/google/t5-v1_1-base) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-v1_1-large/info.yaml b/modelcenter/community/google/t5-v1_1-large/info.yaml index b9315915d38ce0062ea55a39835614d74733eb2c..46201b8220551c21533cc2714f7f976211e2f951 100644 --- a/modelcenter/community/google/t5-v1_1-large/info.yaml +++ b/modelcenter/community/google/t5-v1_1-large/info.yaml @@ -1,26 +1,23 @@ +Datasets: c4 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "google/t5-v1_1-large" - description: "Version 1.1" - description_en: "Version 1.1" - icon: "" - from_repo: "https://huggingface.co/google/t5-v1_1-large" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4" -Publisher: "google" -License: "apache-2.0" -Language: "English" + description: Version 1.1 + description_en: Version 1.1 + from_repo: https://huggingface.co/google/t5-v1_1-large + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: google/t5-v1_1-large Paper: - - title: 'GLU Variants Improve Transformer' - url: 'http://arxiv.org/abs/2002.05202v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: GLU Variants Improve Transformer + url: http://arxiv.org/abs/2002.05202v1 +- title: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer + url: http://arxiv.org/abs/1910.10683v3 +Publisher: google +Task: +- sub_tag: 文本生成 + sub_tag_en: Text2Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/google/t5-v1_1-large/introduction_cn.ipynb b/modelcenter/community/google/t5-v1_1-large/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..1ad787e785db45c64ad232e6d8742db04bae908f --- /dev/null +++ b/modelcenter/community/google/t5-v1_1-large/introduction_cn.ipynb @@ -0,0 +1,101 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "11d36429", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1\n", + "\n", + "\n", + "## Version 1.1\n", + "\n", + "[T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "**Note**: T5 Version 1.1 was only pre-trained on C4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task.\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "c11ad8cf", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "480104f2", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e1323ff9", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-v1_1-large\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "4348828e", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/google/t5-v1_1-large](https://huggingface.co/google/t5-v1_1-large)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-v1_1-large/introduction_en.ipynb b/modelcenter/community/google/t5-v1_1-large/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..4efcfb2f376056a749dbe870708cbe17aea5bb38 --- /dev/null +++ b/modelcenter/community/google/t5-v1_1-large/introduction_en.ipynb @@ -0,0 +1,101 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "5f0c769f", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1\n", + "\n", + "\n", + "## Version 1.1\n", + "\n", + "[T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "**Note**: T5 Version 1.1 was only pre-trained on C4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task.\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "27e206b9", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3cf23148", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "467b7ff7", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-v1_1-large\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "a5616bca", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/google/t5-v1_1-large](https://huggingface.co/google/t5-v1_1-large) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-v1_1-small/info.yaml b/modelcenter/community/google/t5-v1_1-small/info.yaml index ffaf1b744010f53450fe2b0de22d3d4010005950..8cd8aced2ddee0d900e2cac24e5294596ac9dd57 100644 --- a/modelcenter/community/google/t5-v1_1-small/info.yaml +++ b/modelcenter/community/google/t5-v1_1-small/info.yaml @@ -1,26 +1,23 @@ +Datasets: c4 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "google/t5-v1_1-small" - description: "Version 1.1" - description_en: "Version 1.1" - icon: "" - from_repo: "https://huggingface.co/google/t5-v1_1-small" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4" -Publisher: "google" -License: "apache-2.0" -Language: "English" + description: Version 1.1 + description_en: Version 1.1 + from_repo: https://huggingface.co/google/t5-v1_1-small + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: google/t5-v1_1-small Paper: - - title: 'GLU Variants Improve Transformer' - url: 'http://arxiv.org/abs/2002.05202v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: GLU Variants Improve Transformer + url: http://arxiv.org/abs/2002.05202v1 +- title: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer + url: http://arxiv.org/abs/1910.10683v3 +Publisher: google +Task: +- sub_tag: 文本生成 + sub_tag_en: Text2Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/google/t5-v1_1-small/introduction_cn.ipynb b/modelcenter/community/google/t5-v1_1-small/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..ebcc34ea3ac005d55e3a000df91bc1359847a60f --- /dev/null +++ b/modelcenter/community/google/t5-v1_1-small/introduction_cn.ipynb @@ -0,0 +1,103 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "51d7e9ca", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1\n", + "\n", + "\n", + "## Version 1.1\n", + "\n", + "[T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "**Note**: T5 Version 1.1 was only pre-trained on C4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task.\n", + "Pretraining Dataset: [C4](https://huggingface.co/datasets/c4)\n", + "\n", + "Other Community Checkpoints: [here](https://huggingface.co/models?search=t5-v1_1)\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "b4b5fc59", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ae31cbc9", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "81d25d09", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-v1_1-small\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "6e0459f7", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/google/t5-v1_1-small](https://huggingface.co/google/t5-v1_1-small)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-v1_1-small/introduction_en.ipynb b/modelcenter/community/google/t5-v1_1-small/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..e89ecb34792133edff852bb34e62a500917e512c --- /dev/null +++ b/modelcenter/community/google/t5-v1_1-small/introduction_en.ipynb @@ -0,0 +1,103 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "95b64b6f", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1\n", + "\n", + "\n", + "## Version 1.1\n", + "\n", + "[T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "**Note**: T5 Version 1.1 was only pre-trained on C4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task.\n", + "Pretraining Dataset: [C4](https://huggingface.co/datasets/c4)\n", + "\n", + "Other Community Checkpoints: [here](https://huggingface.co/models?search=t5-v1_1)\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "88ec53f3", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "082cae7f", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "05f7f4d0", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-v1_1-small\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "c7a95cdf", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/google/t5-v1_1-small](https://huggingface.co/google/t5-v1_1-small) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-bert-wwm-ext/info.yaml b/modelcenter/community/hfl/chinese-bert-wwm-ext/info.yaml index 8474c936e8a41f7ee4062a6ecd2ed6206f49a9fc..2e118e75cde3385acff469c1552770468d868837 100644 --- a/modelcenter/community/hfl/chinese-bert-wwm-ext/info.yaml +++ b/modelcenter/community/hfl/chinese-bert-wwm-ext/info.yaml @@ -1,26 +1,23 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Chinese +License: apache-2.0 Model_Info: - name: "hfl/chinese-bert-wwm-ext" - description: "Chinese BERT with Whole Word Masking" - description_en: "Chinese BERT with Whole Word Masking" - icon: "" - from_repo: "https://huggingface.co/hfl/chinese-bert-wwm-ext" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "hfl" -License: "apache-2.0" -Language: "Chinese" + description: Chinese BERT with Whole Word Masking + description_en: Chinese BERT with Whole Word Masking + from_repo: https://huggingface.co/hfl/chinese-bert-wwm-ext + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: hfl/chinese-bert-wwm-ext Paper: - - title: 'Pre-Training with Whole Word Masking for Chinese BERT' - url: 'http://arxiv.org/abs/1906.08101v3' - - title: 'Revisiting Pre-Trained Models for Chinese Natural Language Processing' - url: 'http://arxiv.org/abs/2004.13922v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Pre-Training with Whole Word Masking for Chinese BERT + url: http://arxiv.org/abs/1906.08101v3 +- title: Revisiting Pre-Trained Models for Chinese Natural Language Processing + url: http://arxiv.org/abs/2004.13922v2 +Publisher: hfl +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/hfl/chinese-bert-wwm-ext/introduction_cn.ipynb b/modelcenter/community/hfl/chinese-bert-wwm-ext/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..7d8a9de39fac2fb23e6be7d3c29dea837d1d25cc --- /dev/null +++ b/modelcenter/community/hfl/chinese-bert-wwm-ext/introduction_cn.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a5e1e8bd", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "be498a8f", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0199d11d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b71b0698", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-bert-wwm-ext\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "5d6bd99f", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "456616b7", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "9784d9b7", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "15ed9adf", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3593ecc9", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/hfl/chinese-bert-wwm-ext](https://huggingface.co/hfl/chinese-bert-wwm-ext)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-bert-wwm-ext/introduction_en.ipynb b/modelcenter/community/hfl/chinese-bert-wwm-ext/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..debab5d4a1036e653973add13c9403a5cacbf357 --- /dev/null +++ b/modelcenter/community/hfl/chinese-bert-wwm-ext/introduction_en.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "faeb5f50", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "fbf98c0e", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f6b3ac7", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f380cab7", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-bert-wwm-ext\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "a39bca7c", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "5cff4b49", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "a8781cbe", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "b7acc10f", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "86de1995", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/hfl/chinese-bert-wwm-ext](https://huggingface.co/hfl/chinese-bert-wwm-ext) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-bert-wwm/info.yaml b/modelcenter/community/hfl/chinese-bert-wwm/info.yaml index 06014fe9d0ccd158deb1f60698717081635d907b..78271b2fe7f3bd4ea353e74942af058d6fd8ed44 100644 --- a/modelcenter/community/hfl/chinese-bert-wwm/info.yaml +++ b/modelcenter/community/hfl/chinese-bert-wwm/info.yaml @@ -1,26 +1,23 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Chinese +License: apache-2.0 Model_Info: - name: "hfl/chinese-bert-wwm" - description: "Chinese BERT with Whole Word Masking" - description_en: "Chinese BERT with Whole Word Masking" - icon: "" - from_repo: "https://huggingface.co/hfl/chinese-bert-wwm" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "hfl" -License: "apache-2.0" -Language: "Chinese" + description: Chinese BERT with Whole Word Masking + description_en: Chinese BERT with Whole Word Masking + from_repo: https://huggingface.co/hfl/chinese-bert-wwm + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: hfl/chinese-bert-wwm Paper: - - title: 'Pre-Training with Whole Word Masking for Chinese BERT' - url: 'http://arxiv.org/abs/1906.08101v3' - - title: 'Revisiting Pre-Trained Models for Chinese Natural Language Processing' - url: 'http://arxiv.org/abs/2004.13922v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Pre-Training with Whole Word Masking for Chinese BERT + url: http://arxiv.org/abs/1906.08101v3 +- title: Revisiting Pre-Trained Models for Chinese Natural Language Processing + url: http://arxiv.org/abs/2004.13922v2 +Publisher: hfl +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/hfl/chinese-bert-wwm/introduction_cn.ipynb b/modelcenter/community/hfl/chinese-bert-wwm/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..6de568ac57191597c5d8b38e90e44cecc65058cc --- /dev/null +++ b/modelcenter/community/hfl/chinese-bert-wwm/introduction_cn.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a5e1e8bd", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "be498a8f", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0199d11d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b71b0698", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-bert-wwm\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "5d6bd99f", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "376186df", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "9784d9b7", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "478fe6be", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3593ecc9", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/hfl/chinese-bert-wwm](https://huggingface.co/hfl/chinese-bert-wwm)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-bert-wwm/introduction_en.ipynb b/modelcenter/community/hfl/chinese-bert-wwm/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..bba2578ced4304fc59a7ede137feed585d7d9ebe --- /dev/null +++ b/modelcenter/community/hfl/chinese-bert-wwm/introduction_en.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "faeb5f50", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "fbf98c0e", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f6b3ac7", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f380cab7", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-bert-wwm\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "a39bca7c", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "0ebe185e", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "a8781cbe", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "85d2437a", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "86de1995", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/hfl/chinese-bert-wwm-ext](https://huggingface.co/hfl/chinese-bert-wwm-ext) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/info.yaml b/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/info.yaml index 537a41236be847164e4b1c83ff6e0e6e1892418f..fc792db218ffd773a5a8919adf8a3ebaf8fb8c3c 100644 --- a/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/info.yaml +++ b/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/info.yaml @@ -1,26 +1,23 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Chinese +License: apache-2.0 Model_Info: - name: "hfl/chinese-roberta-wwm-ext-large" - description: "Please use 'Bert' related functions to load this model!" - description_en: "Please use 'Bert' related functions to load this model!" - icon: "" - from_repo: "https://huggingface.co/hfl/chinese-roberta-wwm-ext-large" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "hfl" -License: "apache-2.0" -Language: "Chinese" + description: Please use 'Bert' related functions to load this model! + description_en: Please use 'Bert' related functions to load this model! + from_repo: https://huggingface.co/hfl/chinese-roberta-wwm-ext-large + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: hfl/chinese-roberta-wwm-ext-large Paper: - - title: 'Pre-Training with Whole Word Masking for Chinese BERT' - url: 'http://arxiv.org/abs/1906.08101v3' - - title: 'Revisiting Pre-Trained Models for Chinese Natural Language Processing' - url: 'http://arxiv.org/abs/2004.13922v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Pre-Training with Whole Word Masking for Chinese BERT + url: http://arxiv.org/abs/1906.08101v3 +- title: Revisiting Pre-Trained Models for Chinese Natural Language Processing + url: http://arxiv.org/abs/2004.13922v2 +Publisher: hfl +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/introduction_cn.ipynb b/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..d0394493195d80b2ce8becbf1b2594564528ec03 --- /dev/null +++ b/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/introduction_cn.ipynb @@ -0,0 +1,156 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a5e1e8bd", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "\n", + "### Please use 'Bert' related functions to load this model!\n", + "\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "be498a8f", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0199d11d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b71b0698", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-roberta-wwm-ext-large\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "5d6bd99f", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "9429c396", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "9784d9b7", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "eb3e56a1", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3593ecc9", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/hfl/chinese-roberta-wwm-ext-large](https://huggingface.co/hfl/chinese-roberta-wwm-ext-large)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/introduction_en.ipynb b/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..43826e439f5707f46f53a4636d1cccc847c6980b --- /dev/null +++ b/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/introduction_en.ipynb @@ -0,0 +1,156 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "faeb5f50", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "\n", + "### Please use 'Bert' related functions to load this model!\n", + "\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "fbf98c0e", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f6b3ac7", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f380cab7", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-roberta-wwm-ext-large\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "a39bca7c", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "b01c1973", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "a8781cbe", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "7ad8a810", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "86de1995", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/hfl/chinese-roberta-wwm-ext-large](https://huggingface.co/hfl/chinese-roberta-wwm-ext-large) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-roberta-wwm-ext/info.yaml b/modelcenter/community/hfl/chinese-roberta-wwm-ext/info.yaml index c0e9a0a44492cea0a6770fd5fcdb5fac450f338e..79b37c73f25217f81d3e55136c84c05f2557af19 100644 --- a/modelcenter/community/hfl/chinese-roberta-wwm-ext/info.yaml +++ b/modelcenter/community/hfl/chinese-roberta-wwm-ext/info.yaml @@ -1,26 +1,23 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Chinese +License: apache-2.0 Model_Info: - name: "hfl/chinese-roberta-wwm-ext" - description: "Please use 'Bert' related functions to load this model!" - description_en: "Please use 'Bert' related functions to load this model!" - icon: "" - from_repo: "https://huggingface.co/hfl/chinese-roberta-wwm-ext" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "hfl" -License: "apache-2.0" -Language: "Chinese" + description: Please use 'Bert' related functions to load this model! + description_en: Please use 'Bert' related functions to load this model! + from_repo: https://huggingface.co/hfl/chinese-roberta-wwm-ext + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: hfl/chinese-roberta-wwm-ext Paper: - - title: 'Pre-Training with Whole Word Masking for Chinese BERT' - url: 'http://arxiv.org/abs/1906.08101v3' - - title: 'Revisiting Pre-Trained Models for Chinese Natural Language Processing' - url: 'http://arxiv.org/abs/2004.13922v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Pre-Training with Whole Word Masking for Chinese BERT + url: http://arxiv.org/abs/1906.08101v3 +- title: Revisiting Pre-Trained Models for Chinese Natural Language Processing + url: http://arxiv.org/abs/2004.13922v2 +Publisher: hfl +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/hfl/chinese-roberta-wwm-ext/introduction_cn.ipynb b/modelcenter/community/hfl/chinese-roberta-wwm-ext/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..f55e7b223b031b3fc7d81793be7b854fe771378c --- /dev/null +++ b/modelcenter/community/hfl/chinese-roberta-wwm-ext/introduction_cn.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a5e1e8bd", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "be498a8f", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0199d11d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b71b0698", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-roberta-wwm-ext\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "5d6bd99f", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "737822b2", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "9784d9b7", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "22d0c28d", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3593ecc9", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/hfl/chinese-roberta-wwm-ext](https://huggingface.co/hfl/chinese-roberta-wwm-ext)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-roberta-wwm-ext/introduction_en.ipynb b/modelcenter/community/hfl/chinese-roberta-wwm-ext/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..506685005df3e9cf1ff577d0af9da1302bd32dad --- /dev/null +++ b/modelcenter/community/hfl/chinese-roberta-wwm-ext/introduction_en.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "faeb5f50", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "fbf98c0e", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f6b3ac7", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f380cab7", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-roberta-wwm-ext\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "a39bca7c", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "f495aec9", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "a8781cbe", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "8eebfbf4", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "86de1995", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/hfl/chinese-roberta-wwm-ext](https://huggingface.co/hfl/chinese-roberta-wwm-ext) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/rbt3/info.yaml b/modelcenter/community/hfl/rbt3/info.yaml index 36c3c8d5736825c576de4b88b6bfb41c976127f8..b03acf557b36ae55cb48079e0f3a51fb57db0743 100644 --- a/modelcenter/community/hfl/rbt3/info.yaml +++ b/modelcenter/community/hfl/rbt3/info.yaml @@ -1,26 +1,23 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Chinese +License: apache-2.0 Model_Info: - name: "hfl/rbt3" - description: "This is a re-trained 3-layer RoBERTa-wwm-ext model." - description_en: "This is a re-trained 3-layer RoBERTa-wwm-ext model." - icon: "" - from_repo: "https://huggingface.co/hfl/rbt3" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "hfl" -License: "apache-2.0" -Language: "Chinese" + description: This is a re-trained 3-layer RoBERTa-wwm-ext model. + description_en: This is a re-trained 3-layer RoBERTa-wwm-ext model. + from_repo: https://huggingface.co/hfl/rbt3 + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: hfl/rbt3 Paper: - - title: 'Pre-Training with Whole Word Masking for Chinese BERT' - url: 'http://arxiv.org/abs/1906.08101v3' - - title: 'Revisiting Pre-Trained Models for Chinese Natural Language Processing' - url: 'http://arxiv.org/abs/2004.13922v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Pre-Training with Whole Word Masking for Chinese BERT + url: http://arxiv.org/abs/1906.08101v3 +- title: Revisiting Pre-Trained Models for Chinese Natural Language Processing + url: http://arxiv.org/abs/2004.13922v2 +Publisher: hfl +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/hfl/rbt3/introduction_cn.ipynb b/modelcenter/community/hfl/rbt3/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..eb219922d9678184b5c0f39395d906ffbb138934 --- /dev/null +++ b/modelcenter/community/hfl/rbt3/introduction_cn.ipynb @@ -0,0 +1,156 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a5e1e8bd", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "\n", + "### Please use 'Bert' related functions to load this model!\n", + "\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "be498a8f", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0199d11d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b71b0698", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/rbt3\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "5d6bd99f", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "73e04675", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "9784d9b7", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "068895c6", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3593ecc9", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/hfl/rbt3](https://huggingface.co/hfl/rbt3)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/rbt3/introduction_en.ipynb b/modelcenter/community/hfl/rbt3/introduction_en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..4d118772f3472bc9d9a9e87dcf03dc2244cf73d3 --- /dev/null +++ b/modelcenter/community/hfl/rbt3/introduction_en.ipynb @@ -0,0 +1,156 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "faeb5f50", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "\n", + "### Please use 'Bert' related functions to load this model!\n", + "\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "fbf98c0e", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f6b3ac7", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f380cab7", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/rbt3\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "a39bca7c", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "370bfe67", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "a8781cbe", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "4a1fe5aa", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "86de1995", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/hfl/rbt3](https://huggingface.co/hfl/rbt3) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/huggingface/bert-base-cased/info.yaml b/modelcenter/community/huggingface/bert-base-cased/info.yaml index 20c36035a2f5a18862992ec87f67c6a7619b372c..467ef27a6353e14a8458ed8c29a8080a49369d90 100644 --- a/modelcenter/community/huggingface/bert-base-cased/info.yaml +++ b/modelcenter/community/huggingface/bert-base-cased/info.yaml @@ -1,24 +1,21 @@ +Datasets: bookcorpus,wikipedia +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "bert-base-cased" - description: "BERT base model (cased)" - description_en: "BERT base model (cased)" - icon: "" - from_repo: "https://huggingface.co/bert-base-cased" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "bookcorpus,wikipedia" -Publisher: "huggingface" -License: "apache-2.0" -Language: "English" + description: BERT base model (cased) + description_en: BERT base model (cased) + from_repo: https://huggingface.co/bert-base-cased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: bert-base-cased Paper: - - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding' - url: 'http://arxiv.org/abs/1810.04805v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding' + url: http://arxiv.org/abs/1810.04805v2 +Publisher: huggingface +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/huggingface/bert-base-cased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-base-cased/introduction_cn.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..66f9747f240af7bfdf168b05f525348d03a35f7f --- /dev/null +++ b/modelcenter/community/huggingface/bert-base-cased/introduction_cn.ipynb @@ -0,0 +1,170 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "bb008e6f", + "metadata": {}, + "source": [ + "# BERT base model (cased)\n", + "\n", + "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。" + ] + }, + { + "cell_type": "markdown", + "id": "079266fb", + "metadata": {}, + "source": [ + "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n", + "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n", + "[this repository](https://github.com/google-research/bert). This model is case-sensitive: it makes a difference between\n", + "english and English.\n" + ] + }, + { + "cell_type": "markdown", + "id": "5c8220aa", + "metadata": {}, + "source": [ + "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n", + "the Hugging Face team.\n" + ] + }, + { + "cell_type": "markdown", + "id": "8564477f", + "metadata": {}, + "source": [ + "## Model description\n" + ] + }, + { + "cell_type": "markdown", + "id": "7365685d", + "metadata": {}, + "source": [ + "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n", + "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n", + "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n", + "was pretrained with two objectives:\n" + ] + }, + { + "cell_type": "markdown", + "id": "1c979d12", + "metadata": {}, + "source": [ + "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n", + "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n", + "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n", + "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n", + "sentence.\n", + "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n", + "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n", + "predict if the two sentences were following each other or not.\n" + ] + }, + { + "cell_type": "markdown", + "id": "cdc00722", + "metadata": {}, + "source": [ + "This way, the model learns an inner representation of the English language that can then be used to extract features\n", + "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n", + "classifier using the features produced by the BERT model as inputs.\n" + ] + }, + { + "cell_type": "markdown", + "id": "9253a517", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9fbfcd0a", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6185db74", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"bert-base-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "8e0ca3bd", + "metadata": {}, + "source": [ + "```\n", + "@article{DBLP:journals/corr/abs-1810-04805,\n", + "author = {Jacob Devlin and\n", + "Ming{-}Wei Chang and\n", + "Kenton Lee and\n", + "Kristina Toutanova},\n", + "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n", + "Understanding},\n", + "journal = {CoRR},\n", + "volume = {abs/1810.04805},\n", + "year = {2018},\n", + "url = {http://arxiv.org/abs/1810.04805},\n", + "archivePrefix = {arXiv},\n", + "eprint = {1810.04805},\n", + "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n", + "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n", + "bibsource = {dblp computer science bibliography, https://dblp.org}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "f14e9f06", + "metadata": {}, + "source": [ + "\n", + "
![](\"https://cdn-media.huggingface.co/exbert/button.png\")
\n",
+ "\n",
+ "\n",
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-base-cased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-cased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-base-cased/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..40c4e44fd2fb72fcbb3cc429e414bc12c8041f06
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-cased/introduction_en.ipynb
@@ -0,0 +1,171 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "58235e68",
+ "metadata": {},
+ "source": [
+ "# BERT base model (cased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "36c7d585",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is case-sensitive: it makes a difference between\n",
+ "english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d361a880",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "47b0cf99",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d1911491",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "94e45c66",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9fec6197",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5e17ee3b",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "62ae31d8",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "3c52bdd5",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "da7c4875",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "86873e48",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> The model introduction and model weights originate from https://huggingface.co/bert-base-cased and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-german-cased/info.yaml b/modelcenter/community/huggingface/bert-base-german-cased/info.yaml
index 56dceb1fbe14fe68433be6f37c209f594bc2d69d..f549130cdd7f379f86e3cb7298c9f74cad8bfef1 100644
--- a/modelcenter/community/huggingface/bert-base-german-cased/info.yaml
+++ b/modelcenter/community/huggingface/bert-base-german-cased/info.yaml
@@ -1,23 +1,19 @@
+Datasets: ''
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: German
+License: mit
Model_Info:
- name: "bert-base-german-cased"
- description: "German BERT"
- description_en: "German BERT"
- icon: ""
- from_repo: "https://huggingface.co/bert-base-german-cased"
-
+ description: German BERT
+ description_en: German BERT
+ from_repo: https://huggingface.co/bert-base-german-cased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-base-german-cased
+Paper: null
+Publisher: huggingface
Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: ""
-Publisher: "huggingface"
-License: "mit"
-Language: "German"
-Paper:
-
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-base-german-cased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-base-german-cased/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..007d6df6baa08cacfc9ddceff7bc2a3e34e23ead
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-german-cased/introduction_cn.ipynb
@@ -0,0 +1,128 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "0870a629",
+ "metadata": {},
+ "source": [
+ "# German BERT\n",
+ "\n",
+ "## Overview\n",
+ "**Language model:** bert-base-cased\n",
+ "**Language:** German\n",
+ "**Training data:** Wiki, OpenLegalData, News (~ 12GB)\n",
+ "**Eval data:** Conll03 (NER), GermEval14 (NER), GermEval18 (Classification), GNAD (Classification)\n",
+ "**Infrastructure**: 1x TPU v2\n",
+ "**Published**: Jun 14th, 2019\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b2a6c897",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "1790135e",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "99c714ac",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-german-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "54c2f398",
+ "metadata": {},
+ "source": [
+ "## Authors\n",
+ "- Branden Chan: `branden.chan [at] deepset.ai`\n",
+ "- Timo Möller: `timo.moeller [at] deepset.ai`\n",
+ "- Malte Pietsch: `malte.pietsch [at] deepset.ai`\n",
+ "- Tanay Soni: `tanay.soni [at] deepset.ai`\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "94b669bc",
+ "metadata": {},
+ "source": [
+ "## About us\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ce90710a",
+ "metadata": {},
+ "source": [
+ "We bring NLP to the industry via open source!\n",
+ "Our focus: Industry specific language models & large scale QA systems.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5dc8ba63",
+ "metadata": {},
+ "source": [
+ "Some of our work:\n",
+ "- [German BERT (aka \"bert-base-german-cased\")](https://deepset.ai/german-bert)\n",
+ "- [FARM](https://github.com/deepset-ai/FARM)\n",
+ "- [Haystack](https://github.com/deepset-ai/haystack/)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "56a1a360",
+ "metadata": {},
+ "source": [
+ "Get in touch:\n",
+ "[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai)\n",
+ "\n",
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-base-german-cased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-german-cased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-base-german-cased/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..60d78c671dc55bdd3bbaae0df7935dcdbfcd030f
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-german-cased/introduction_en.ipynb
@@ -0,0 +1,137 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "7aa268f7",
+ "metadata": {},
+ "source": [
+ "# German BERT\n",
+ "\n",
+ "## Overview\n",
+ "**Language model:** bert-base-cased\n",
+ "**Language:** German\n",
+ "**Training data:** Wiki, OpenLegalData, News (~ 12GB)\n",
+ "**Eval data:** Conll03 (NER), GermEval14 (NER), GermEval18 (Classification), GNAD (Classification)\n",
+ "**Infrastructure**: 1x TPU v2\n",
+ "**Published**: Jun 14th, 2019\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f407e80e",
+ "metadata": {},
+ "source": [
+ "**Update April 3rd, 2020**: we updated the vocabulary file on deepset's s3 to conform with the default tokenization of punctuation tokens.\n",
+ "For details see the related [FARM issue](https://github.com/deepset-ai/FARM/issues/60). If you want to use the old vocab we have also uploaded a deepset/bert-base-german-cased-oldvocab model.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "18d2ad8e",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "b80052bd",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "4ea9d4e3",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-german-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9d560e75",
+ "metadata": {},
+ "source": [
+ "## Authors\n",
+ "- Branden Chan: `branden.chan [at] deepset.ai`\n",
+ "- Timo Möller: `timo.moeller [at] deepset.ai`\n",
+ "- Malte Pietsch: `malte.pietsch [at] deepset.ai`\n",
+ "- Tanay Soni: `tanay.soni [at] deepset.ai`\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a0e43273",
+ "metadata": {},
+ "source": [
+ "## About us\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c1b05e60",
+ "metadata": {},
+ "source": [
+ "We bring NLP to the industry via open source!\n",
+ "Our focus: Industry specific language models & large scale QA systems.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5196bee9",
+ "metadata": {},
+ "source": [
+ "Some of our work:\n",
+ "- [German BERT (aka \"bert-base-german-cased\")](https://deepset.ai/german-bert)\n",
+ "- [FARM](https://github.com/deepset-ai/FARM)\n",
+ "- [Haystack](https://github.com/deepset-ai/haystack/)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "18fe01d5",
+ "metadata": {},
+ "source": [
+ "Get in touch:\n",
+ "[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai)\n",
+ "\n",
+ "> The model introduction and model weights originate from https://huggingface.co/bert-base-german-cased and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-multilingual-cased/info.yaml b/modelcenter/community/huggingface/bert-base-multilingual-cased/info.yaml
index 6f6e99962533a0b6d7ebaf20402c06fa8dc0e61a..30aa91f6e978b372f22e44d54b56b94f9d2880a9 100644
--- a/modelcenter/community/huggingface/bert-base-multilingual-cased/info.yaml
+++ b/modelcenter/community/huggingface/bert-base-multilingual-cased/info.yaml
@@ -1,24 +1,21 @@
+Datasets: wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: ''
+License: apache-2.0
Model_Info:
- name: "bert-base-multilingual-cased"
- description: "BERT multilingual base model (cased)"
- description_en: "BERT multilingual base model (cased)"
- icon: ""
- from_repo: "https://huggingface.co/bert-base-multilingual-cased"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: ""
+ description: BERT multilingual base model (cased)
+ description_en: BERT multilingual base model (cased)
+ from_repo: https://huggingface.co/bert-base-multilingual-cased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-base-multilingual-cased
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-base-multilingual-cased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-base-multilingual-cased/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..6ef00469c4a7303c535bcd4e33b397a983f98a0f
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-multilingual-cased/introduction_cn.ipynb
@@ -0,0 +1,166 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "92e18984",
+ "metadata": {},
+ "source": [
+ "# BERT multilingual base model (cased)\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cc38bad3",
+ "metadata": {},
+ "source": [
+ "Pretrained model on the top 104 languages with the largest Wikipedia using a masked language modeling (MLM) objective.\n",
+ "It was introduced in [this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is case sensitive: it makes a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a54cdf6e",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3be641ef",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "93fd337b",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of multilingual data in a self-supervised fashion. This means\n",
+ "it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2222d4b6",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2f9ea64e",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the languages in the training set that can then be used to\n",
+ "extract features useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a\n",
+ "standard classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7363abb0",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "780c0123",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "5a325830",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-multilingual-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "81ca575a",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "216555c3",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-base-multilingual-cased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-multilingual-cased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-base-multilingual-cased/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..614f449429f54cc92707d6863964ba6c8e3ad302
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-multilingual-cased/introduction_en.ipynb
@@ -0,0 +1,167 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "19d62907",
+ "metadata": {},
+ "source": [
+ "# BERT multilingual base model (cased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "09809b94",
+ "metadata": {},
+ "source": [
+ "Pretrained model on the top 104 languages with the largest Wikipedia using a masked language modeling (MLM) objective.\n",
+ "It was introduced in [this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is case sensitive: it makes a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d3a52162",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f67f02dc",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bf05022f",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of multilingual data in a self-supervised fashion. This means\n",
+ "it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "081a7a88",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "79e6eda9",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the languages in the training set that can then be used to\n",
+ "extract features useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a\n",
+ "standard classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1696fb24",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "4f7d20fd",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "3c369c9a",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-multilingual-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6338f981",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c55dc64e",
+ "metadata": {},
+ "source": [
+ "\n",
+ "> The model introduction and model weights originate from https://huggingface.co/bert-base-multilingual-cased and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-multilingual-uncased/info.yaml b/modelcenter/community/huggingface/bert-base-multilingual-uncased/info.yaml
index eea48a95605fa6eb765402f37641ccee2d2a4d73..d4008a99a41b493cec0dad2c63d12734cc5d5b3e 100644
--- a/modelcenter/community/huggingface/bert-base-multilingual-uncased/info.yaml
+++ b/modelcenter/community/huggingface/bert-base-multilingual-uncased/info.yaml
@@ -1,24 +1,21 @@
+Datasets: wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: ''
+License: apache-2.0
Model_Info:
- name: "bert-base-multilingual-uncased"
- description: "BERT multilingual base model (uncased)"
- description_en: "BERT multilingual base model (uncased)"
- icon: ""
- from_repo: "https://huggingface.co/bert-base-multilingual-uncased"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: ""
+ description: BERT multilingual base model (uncased)
+ description_en: BERT multilingual base model (uncased)
+ from_repo: https://huggingface.co/bert-base-multilingual-uncased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-base-multilingual-uncased
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-base-multilingual-uncased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-base-multilingual-uncased/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..81274e373acf6527025f96beec50f056a15909fa
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-multilingual-uncased/introduction_cn.ipynb
@@ -0,0 +1,166 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "867cb6e6",
+ "metadata": {},
+ "source": [
+ "# BERT multilingual base model (uncased)\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "207ffc57",
+ "metadata": {},
+ "source": [
+ "Pretrained model on the top 102 languages with the largest Wikipedia using a masked language modeling (MLM) objective.\n",
+ "It was introduced in [this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8b2e2c13",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "40d071c9",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "af4a1260",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of multilingual data in a self-supervised fashion. This means\n",
+ "it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "81abfbcb",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "64988b6b",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the languages in the training set that can then be used to\n",
+ "extract features useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a\n",
+ "standard classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "79c3e104",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "d7b2d0ec",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "52f8d16d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-multilingual-uncased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f11b298a",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "548a9d6c",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-base-multilingual-uncased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-multilingual-uncased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-base-multilingual-uncased/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..3c6411361bcc904efe6cf2bb1336becf6a355ff3
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-multilingual-uncased/introduction_en.ipynb
@@ -0,0 +1,166 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "ff0c69a5",
+ "metadata": {},
+ "source": [
+ "# BERT multilingual base model (uncased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1ad499a9",
+ "metadata": {},
+ "source": [
+ "Pretrained model on the top 102 languages with the largest Wikipedia using a masked language modeling (MLM) objective.\n",
+ "It was introduced in [this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a8878d0c",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4581e670",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c8d5f59f",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of multilingual data in a self-supervised fashion. This means\n",
+ "it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "836834df",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bafe70e4",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the languages in the training set that can then be used to\n",
+ "extract features useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a\n",
+ "standard classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cf2a29e2",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "dc792d6e",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "7a6faf50",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-multilingual-uncased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5b616f23",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "67e01093",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/bert-base-multilingual-uncased and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-uncased/info.yaml b/modelcenter/community/huggingface/bert-base-uncased/info.yaml
index c5b7c80fb6bbf17376f459101d03e2f866776592..e60141f57ef148f3be1b32c99d8dcb1b6aac6a18 100644
--- a/modelcenter/community/huggingface/bert-base-uncased/info.yaml
+++ b/modelcenter/community/huggingface/bert-base-uncased/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-base-uncased"
- description: "BERT base model (uncased)"
- description_en: "BERT base model (uncased)"
- icon: ""
- from_repo: "https://huggingface.co/bert-base-uncased"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT base model (uncased)
+ description_en: BERT base model (uncased)
+ from_repo: https://huggingface.co/bert-base-uncased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-base-uncased
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-base-uncased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-base-uncased/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..c848895ed2df847be3a375f60f607a5b4a7a9b3e
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-uncased/introduction_cn.ipynb
@@ -0,0 +1,214 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "a14866e7",
+ "metadata": {},
+ "source": [
+ "# BERT base model (uncased)\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d348c680",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9a790b40",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "985d2894",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "985bd7ee",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labeling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2e1ee5f4",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally masks the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ae584a51",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences, for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a4d48848",
+ "metadata": {},
+ "source": [
+ "## Model variations\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dcb46068",
+ "metadata": {},
+ "source": [
+ "BERT has originally been released in base and large variations, for cased and uncased input text. The uncased models also strips out an accent markers.\n",
+ "Chinese and multilingual uncased and cased versions followed shortly after.\n",
+ "Modified preprocessing with whole word masking has replaced subpiece masking in a following work, with the release of two models.\n",
+ "Other 24 smaller models are released afterward.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bdf3ec7e",
+ "metadata": {},
+ "source": [
+ "The detailed release history can be found on the [google-research/bert readme](https://github.com/google-research/bert/blob/master/README.md) on github.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d66e6fc4",
+ "metadata": {},
+ "source": [
+ "| Model | #params | Language |\n",
+ "|------------------------|--------------------------------|-------|\n",
+ "| [`bert-base-uncased`](https://huggingface.co/bert-base-uncased) | 110M | English |\n",
+ "| [`bert-large-uncased`](https://huggingface.co/bert-large-uncased) | 340M | English | sub\n",
+ "| [`bert-base-cased`](https://huggingface.co/bert-base-cased) | 110M | English |\n",
+ "| [`bert-large-cased`](https://huggingface.co/bert-large-cased) | 340M | English |\n",
+ "| [`bert-base-chinese`](https://huggingface.co/bert-base-chinese) | 110M | Chinese |\n",
+ "| [`bert-base-multilingual-cased`](https://huggingface.co/bert-base-multilingual-cased) | 110M | Multiple |\n",
+ "| [`bert-large-uncased-whole-word-masking`](https://huggingface.co/bert-large-uncased-whole-word-masking) | 340M | English |\n",
+ "| [`bert-large-cased-whole-word-masking`](https://huggingface.co/bert-large-cased-whole-word-masking) | 340M | English |\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "93c97712",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "e4daab88",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "09dec4f3",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-uncased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "85541d34",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "82898490",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-base-uncased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-uncased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-base-uncased/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..ea2c73ba4463b1d6f383901a77169bf315293a39
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-uncased/introduction_en.ipynb
@@ -0,0 +1,214 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "86c2dd31",
+ "metadata": {},
+ "source": [
+ "# BERT base model (uncased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e25590e2",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "632646c9",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6d37733d",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "20eb0099",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labeling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a43bc44c",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally masks the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3ea31760",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences, for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c44e01b0",
+ "metadata": {},
+ "source": [
+ "## Model variations\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6cb3e530",
+ "metadata": {},
+ "source": [
+ "BERT has originally been released in base and large variations, for cased and uncased input text. The uncased models also strips out an accent markers.\n",
+ "Chinese and multilingual uncased and cased versions followed shortly after.\n",
+ "Modified preprocessing with whole word masking has replaced subpiece masking in a following work, with the release of two models.\n",
+ "Other 24 smaller models are released afterward.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "557a417a",
+ "metadata": {},
+ "source": [
+ "The detailed release history can be found on the [google-research/bert readme](https://github.com/google-research/bert/blob/master/README.md) on github.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0f4bf9e0",
+ "metadata": {},
+ "source": [
+ "| Model | #params | Language |\n",
+ "|------------------------|--------------------------------|-------|\n",
+ "| [`bert-base-uncased`](https://huggingface.co/bert-base-uncased) | 110M | English |\n",
+ "| [`bert-large-uncased`](https://huggingface.co/bert-large-uncased) | 340M | English | sub\n",
+ "| [`bert-base-cased`](https://huggingface.co/bert-base-cased) | 110M | English |\n",
+ "| [`bert-large-cased`](https://huggingface.co/bert-large-cased) | 340M | English |\n",
+ "| [`bert-base-chinese`](https://huggingface.co/bert-base-chinese) | 110M | Chinese |\n",
+ "| [`bert-base-multilingual-cased`](https://huggingface.co/bert-base-multilingual-cased) | 110M | Multiple |\n",
+ "| [`bert-large-uncased-whole-word-masking`](https://huggingface.co/bert-large-uncased-whole-word-masking) | 340M | English |\n",
+ "| [`bert-large-cased-whole-word-masking`](https://huggingface.co/bert-large-cased-whole-word-masking) | 340M | English |\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "909c1c8d",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "68db3da7",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "04d6a56d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-uncased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "76d1a4dc",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1bcee897",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> The model introduction and model weights originate from https://huggingface.co/bert-base-uncased and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/info.yaml b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/info.yaml
index f1718d55596e23bc7e7b87be75b667604b185db4..a7ac22baa8fb7ac13192b928f292b5fb01fbf982 100644
--- a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/info.yaml
+++ b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-large-cased-whole-word-masking-finetuned-squad"
- description: "BERT large model (cased) whole word masking finetuned on SQuAD"
- description_en: "BERT large model (cased) whole word masking finetuned on SQuAD"
- icon: ""
- from_repo: "https://huggingface.co/bert-large-cased-whole-word-masking-finetuned-squad"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Question Answering"
- sub_tag: "回答问题"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT large model (cased) whole word masking finetuned on SQuAD
+ description_en: BERT large model (cased) whole word masking finetuned on SQuAD
+ from_repo: https://huggingface.co/bert-large-cased-whole-word-masking-finetuned-squad
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-large-cased-whole-word-masking-finetuned-squad
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 回答问题
+ sub_tag_en: Question Answering
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..d291733e170b825570899526281988dce6ded372
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/introduction_cn.ipynb
@@ -0,0 +1,208 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "7b02f8e4",
+ "metadata": {},
+ "source": [
+ "# BERT large model (cased) whole word masking finetuned on SQuAD\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7804aeec",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is cased: it makes a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9ee7c4ee",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2198ff25",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "159c04c3",
+ "metadata": {},
+ "source": [
+ "After pre-training, this model was fine-tuned on the SQuAD dataset with one of our fine-tuning scripts. See below for more information regarding this fine-tuning.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cec53443",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0a6d113e",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3776a729",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8ef5e147",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f494c97f",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "eeffccad",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7754e7ed",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dc30e3d4",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "6c0a8e7e",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "3eb39f84",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-cased-whole-word-masking-finetuned-squad\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "82b3ff37",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cb789a5a",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-large-cased-whole-word-masking-finetuned-squad ，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/introduction_en.ipynb b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..022a59dcefc4633a33da9bff319e7858318ed87b
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/introduction_en.ipynb
@@ -0,0 +1,208 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "1dfd11c1",
+ "metadata": {},
+ "source": [
+ "# BERT large model (cased) whole word masking finetuned on SQuAD\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7105fb8c",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is cased: it makes a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e3d8b394",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "be078628",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "278aee7f",
+ "metadata": {},
+ "source": [
+ "After pre-training, this model was fine-tuned on the SQuAD dataset with one of our fine-tuning scripts. See below for more information regarding this fine-tuning.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ce69aca2",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "89b52c17",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2ebe9e94",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7131c024",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4a8e4aea",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "717dd1f6",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6778930f",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1ffc0609",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "678acd58",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "c5318a0c",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-cased-whole-word-masking-finetuned-squad\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f930fd97",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b3240bd3",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/bert-large-cased-whole-word-masking-finetuned-squad and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/info.yaml b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/info.yaml
index b7bf0bdeeb448aec0d8c1b6829fdd1ce6b6e7ca7..cbf482fb81d7929dd18b217e335d006196073fa5 100644
--- a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/info.yaml
+++ b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-large-cased-whole-word-masking"
- description: "BERT large model (cased) whole word masking"
- description_en: "BERT large model (cased) whole word masking"
- icon: ""
- from_repo: "https://huggingface.co/bert-large-cased-whole-word-masking"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT large model (cased) whole word masking
+ description_en: BERT large model (cased) whole word masking
+ from_repo: https://huggingface.co/bert-large-cased-whole-word-masking
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-large-cased-whole-word-masking
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..e9d6ca523003a99022d01673cea34b06bccd1cf0
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/introduction_cn.ipynb
@@ -0,0 +1,201 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "1d5ffd6a",
+ "metadata": {},
+ "source": [
+ "# BERT large model (cased) whole word masking\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9e7590bd",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is cased: it makes a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2456751c",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "204d6ee6",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "743ff269",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bce1ffcc",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d5d83b7c",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "38a98598",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "89b5e554",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3f205174",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6b9cf751",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "74a0400e",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "d8952fcc",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "365e04c2",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-cased-whole-word-masking\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1cef8f18",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0d54ff2d",
+ "metadata": {},
+ "source": [
+ "\n",
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-large-cased-whole-word-masking ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/introduction_en.ipynb b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..9617a15c8909ceb2ec07e84ec681c2385a354e5d
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/introduction_en.ipynb
@@ -0,0 +1,237 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "58f64e54",
+ "metadata": {},
+ "source": [
+ "# BERT large model (cased) whole word masking\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6814fe73",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is cased: it makes a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7a6b1b28",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e6c8ddc5",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dfcd9c6b",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d758dbd9",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c4e44287",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d07abc2a",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5fcb83d6",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1be2f6a5",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cd047a65",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "93925c79",
+ "metadata": {},
+ "source": [
+ "## Intended uses & limitations\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f6c1f9b9",
+ "metadata": {},
+ "source": [
+ "You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to\n",
+ "be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=bert) to look for\n",
+ "fine-tuned versions on a task that interests you.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a682ee5c",
+ "metadata": {},
+ "source": [
+ "Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked)\n",
+ "to make decisions, such as sequence classification, token classification or question answering. For tasks such as text\n",
+ "generation you should look at model like GPT2.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "394e6456",
+ "metadata": {},
+ "source": [
+ "### How to use\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9e5fdb9a",
+ "metadata": {},
+ "source": [
+ "You can use this model directly with a pipeline for masked language modeling:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "77af91fe",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "ae5caf8d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-cased-whole-word-masking\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0f43705d",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "54ae4165",
+ "metadata": {},
+ "source": [
+ "\n",
+ "> The model introduction and model weights originate from https://huggingface.co/bert-large-cased-whole-word-masking and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-cased/info.yaml b/modelcenter/community/huggingface/bert-large-cased/info.yaml
index ae878bc6aa580fc51a714ae995d84b97652fd796..89d74497831dbfc03222e47c8bc86603bdf4a155 100644
--- a/modelcenter/community/huggingface/bert-large-cased/info.yaml
+++ b/modelcenter/community/huggingface/bert-large-cased/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-large-cased"
- description: "BERT large model (cased)"
- description_en: "BERT large model (cased)"
- icon: ""
- from_repo: "https://huggingface.co/bert-large-cased"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT large model (cased)
+ description_en: BERT large model (cased)
+ from_repo: https://huggingface.co/bert-large-cased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-large-cased
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-large-cased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-large-cased/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..6a92402ed97386bfea322335b29ad11a05bfcf84
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-cased/introduction_cn.ipynb
@@ -0,0 +1,213 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "360e146a",
+ "metadata": {},
+ "source": [
+ "# BERT large model (cased)\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bb3eb868",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is cased: it makes a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0f512012",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8dfae0e4",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "29d97a32",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "84dd3c36",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dbb66981",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4a3d9a5c",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "85a286cd",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e9f5c5f1",
+ "metadata": {},
+ "source": [
+ "## Intended uses & limitations\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d3ae1617",
+ "metadata": {},
+ "source": [
+ "You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to\n",
+ "be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=bert) to look for\n",
+ "fine-tuned versions on a task that interests you.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1d814aa3",
+ "metadata": {},
+ "source": [
+ "Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked)\n",
+ "to make decisions, such as sequence classification, token classification or question answering. For tasks such as text\n",
+ "generation you should look at model like GPT2.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7c9cb698",
+ "metadata": {},
+ "source": [
+ "### How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "266349de",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "e0d0fb84",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d58fffcd",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8591ee7f",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-large-cased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-cased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-large-cased/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..e4795deb48d985e3b7bcf1bf10b3ff9d7813cab5
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-cased/introduction_en.ipynb
@@ -0,0 +1,185 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "2460ffb6",
+ "metadata": {},
+ "source": [
+ "# BERT large model (cased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "07c2aecf",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is cased: it makes a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "fb6201f0",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ffd4c0b9",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0b465123",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7a5eb557",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d40678bb",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8fc24335",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "355e9553",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "47e2e497",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "c4d80b50",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f73f3925",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2873617b",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bc4aea4d",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/bert-large-cased and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/info.yaml b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/info.yaml
index 0a4f922ce0c6eaf7c85e482b4e24052453782806..a7510ff89948ad0e522ca44fc6ab4881605c3e34 100644
--- a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/info.yaml
+++ b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-large-uncased-whole-word-masking-finetuned-squad"
- description: "BERT large model (uncased) whole word masking finetuned on SQuAD"
- description_en: "BERT large model (uncased) whole word masking finetuned on SQuAD"
- icon: ""
- from_repo: "https://huggingface.co/bert-large-uncased-whole-word-masking-finetuned-squad"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Question Answering"
- sub_tag: "回答问题"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT large model (uncased) whole word masking finetuned on SQuAD
+ description_en: BERT large model (uncased) whole word masking finetuned on SQuAD
+ from_repo: https://huggingface.co/bert-large-uncased-whole-word-masking-finetuned-squad
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-large-uncased-whole-word-masking-finetuned-squad
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 回答问题
+ sub_tag_en: Question Answering
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..93f59ac901661ad2b3b257d91132f020bfd51694
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/introduction_cn.ipynb
@@ -0,0 +1,207 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "aad9532a",
+ "metadata": {},
+ "source": [
+ "# BERT large model (uncased) whole word masking finetuned on SQuAD\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "724df271",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f2b9e3bf",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6566eb12",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7b45422b",
+ "metadata": {},
+ "source": [
+ "After pre-training, this model was fine-tuned on the SQuAD dataset with one of our fine-tuning scripts. See below for more information regarding this fine-tuning.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c9957f91",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "43cba468",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "457bfeee",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "77c83270",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0ba87de6",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f363132f",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "83a4e49f",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "68565c6d",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "457a1c54",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "a9369c0d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-uncased-whole-word-masking-finetuned-squad\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c5fefb8f",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "654c0920",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-large-uncased-whole-word-masking-finetuned-squad ，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/introduction_en.ipynb b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..f940e0507bd429a93b177b3db9bbf91575fb540f
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/introduction_en.ipynb
@@ -0,0 +1,209 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "2d4f4368",
+ "metadata": {},
+ "source": [
+ "# BERT large model (uncased) whole word masking finetuned on SQuAD\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "afef45e0",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c94536b9",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "50254dea",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4b482be9",
+ "metadata": {},
+ "source": [
+ "After pre-training, this model was fine-tuned on the SQuAD dataset with one of our fine-tuning scripts. See below for more information regarding this fine-tuning.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "adfc36af",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "22f554a7",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "eccd3048",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3d4098e8",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1047d1ad",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7046db0c",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "09659088",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "65769919",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "4449cfac",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "1e8dcf70",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-uncased-whole-word-masking-finetuned-squad\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "49471f4b",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d783c8fc",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/bert-large-uncased-whole-word-masking-finetuned-squad and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/info.yaml b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/info.yaml
index 82a56eb146ec48c7e16e2e11395c6e47c3153c1e..8629aba00fdf3defb1d617d8d1c0fb01ad2fb277 100644
--- a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/info.yaml
+++ b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-large-uncased-whole-word-masking"
- description: "BERT large model (uncased) whole word masking"
- description_en: "BERT large model (uncased) whole word masking"
- icon: ""
- from_repo: "https://huggingface.co/bert-large-uncased-whole-word-masking"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT large model (uncased) whole word masking
+ description_en: BERT large model (uncased) whole word masking
+ from_repo: https://huggingface.co/bert-large-uncased-whole-word-masking
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-large-uncased-whole-word-masking
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..a686b3e044f116a0e97110f531216d95db0a8ade
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/introduction_cn.ipynb
@@ -0,0 +1,201 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "cf43e770",
+ "metadata": {},
+ "source": [
+ "# BERT large model (uncased) whole word masking\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "af8c3816",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c103e84b",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "19a76368",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "67f11a2c",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "778cf97d",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dddbb307",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "40becad1",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3fc265b6",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "65e4a308",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6d0b86c1",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dd94b8be",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "bc669f99",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "4580650d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-uncased-whole-word-masking\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "475fd35d",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f09b9b09",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-large-uncased-whole-word-masking ，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/introduction_en.ipynb b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..2557af1b7380a8c89ec582d5db454cf9f85308c8
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/introduction_en.ipynb
@@ -0,0 +1,237 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "ceefe52d",
+ "metadata": {},
+ "source": [
+ "# BERT large model (uncased) whole word masking\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "14552c09",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "78d1e4a0",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cdbe484a",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "fdbba80d",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "aba33624",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "459ca6e6",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "65f2ae1a",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "86e8d7eb",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b81821d8",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3a576172",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0038f06c",
+ "metadata": {},
+ "source": [
+ "## Intended uses & limitations\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ba8c18de",
+ "metadata": {},
+ "source": [
+ "You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to\n",
+ "be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=bert) to look for\n",
+ "fine-tuned versions on a task that interests you.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bb72ad39",
+ "metadata": {},
+ "source": [
+ "Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked)\n",
+ "to make decisions, such as sequence classification, token classification or question answering. For tasks such as text\n",
+ "generation you should look at model like GPT2.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "54b59ca8",
+ "metadata": {},
+ "source": [
+ "### How to use\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a0ff2a80",
+ "metadata": {},
+ "source": [
+ "You can use this model directly with a pipeline for masked language modeling:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "990ce14a",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "7d468ffb",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-uncased-whole-word-masking\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "93d6e9e4",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c9d05272",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/bert-large-uncased-whole-word-masking and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-uncased/info.yaml b/modelcenter/community/huggingface/bert-large-uncased/info.yaml
index 41677ffce758b0633b47856575eb021bd5c447c9..cf05a3fbf7b8dcd1bdfefbbc7153f62634384a76 100644
--- a/modelcenter/community/huggingface/bert-large-uncased/info.yaml
+++ b/modelcenter/community/huggingface/bert-large-uncased/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-large-uncased"
- description: "BERT large model (uncased)"
- description_en: "BERT large model (uncased)"
- icon: ""
- from_repo: "https://huggingface.co/bert-large-uncased"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT large model (uncased)
+ description_en: BERT large model (uncased)
+ from_repo: https://huggingface.co/bert-large-uncased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-large-uncased
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-large-uncased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-large-uncased/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..dabec910400bdd371124930fa38dc05f9b0aef9e
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-uncased/introduction_cn.ipynb
@@ -0,0 +1,185 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "c000df74",
+ "metadata": {},
+ "source": [
+ "# BERT large model (uncased)\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bd7436a9",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "87c430c2",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e2004f07",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ad86c301",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8f12ab3c",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3fc80525",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c31d15b4",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "822f7f40",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7fcdeb04",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "db4ceaa3",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "dc6a0473",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-uncased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1156d387",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9d07ca08",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-large-uncased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-uncased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-large-uncased/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..e73afd86ee3beb204455b3bea22d66b68ea2c529
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-uncased/introduction_en.ipynb
@@ -0,0 +1,185 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "a4fae520",
+ "metadata": {},
+ "source": [
+ "# BERT large model (uncased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c410d1ae",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "40166ab8",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dacb968e",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c519206d",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2dd87a78",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "507ce60a",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7fb7a8a0",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ebe2c593",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "547e3cc8",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "669cb05f",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "09a4bc02",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-uncased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3ae36313",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bed31ba3",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/bert-large-uncased and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/distilbert-base-multilingual-cased/info.yaml b/modelcenter/community/huggingface/distilbert-base-multilingual-cased/info.yaml
index 24ef1d56d6959a6bcf5c07844534bf8e8523863d..3e9ab3a60afed047db68b8ea0db89ae309fa1ef9 100644
--- a/modelcenter/community/huggingface/distilbert-base-multilingual-cased/info.yaml
+++ b/modelcenter/community/huggingface/distilbert-base-multilingual-cased/info.yaml
@@ -1,26 +1,23 @@
+Datasets: wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: ''
+License: apache-2.0
Model_Info:
- name: "distilbert-base-multilingual-cased"
- description: "Model Card for DistilBERT base multilingual (cased)"
- description_en: "Model Card for DistilBERT base multilingual (cased)"
- icon: ""
- from_repo: "https://huggingface.co/distilbert-base-multilingual-cased"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: ""
+ description: Model Card for DistilBERT base multilingual (cased)
+ description_en: Model Card for DistilBERT base multilingual (cased)
+ from_repo: https://huggingface.co/distilbert-base-multilingual-cased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: distilbert-base-multilingual-cased
Paper:
- - title: 'DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter'
- url: 'http://arxiv.org/abs/1910.01108v4'
- - title: 'Quantifying the Carbon Emissions of Machine Learning'
- url: 'http://arxiv.org/abs/1910.09700v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter'
+ url: http://arxiv.org/abs/1910.01108v4
+- title: Quantifying the Carbon Emissions of Machine Learning
+ url: http://arxiv.org/abs/1910.09700v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/distilbert-base-multilingual-cased/introduction_cn.ipynb b/modelcenter/community/huggingface/distilbert-base-multilingual-cased/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..32ecf21ba60b43142a2d4e7b0daf019ce91d489b
--- /dev/null
+++ b/modelcenter/community/huggingface/distilbert-base-multilingual-cased/introduction_cn.ipynb
@@ -0,0 +1,143 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "922fd8e5",
+ "metadata": {},
+ "source": [
+ "# Model Card for DistilBERT base multilingual (cased)\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a1024bec",
+ "metadata": {},
+ "source": [
+ "## Model Description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bcdfe024",
+ "metadata": {},
+ "source": [
+ "This model is a distilled version of the [BERT base multilingual model](https://huggingface.co/bert-base-multilingual-cased/). The code for the distillation process can be found [here](https://github.com/huggingface/transformers/tree/main/examples/research_projects/distillation). This model is cased: it does make a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5051aaa6",
+ "metadata": {},
+ "source": [
+ "The model is trained on the concatenation of Wikipedia in 104 different languages listed [here](https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages).\n",
+ "The model has 6 layers, 768 dimension and 12 heads, totalizing 134M parameters (compared to 177M parameters for mBERT-base).\n",
+ "On average, this model, referred to as DistilmBERT, is twice as fast as mBERT-base.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cdddc273",
+ "metadata": {},
+ "source": [
+ "We encourage potential users of this model to check out the [BERT base multilingual model card](https://huggingface.co/bert-base-multilingual-cased) to learn more about usage, limitations and potential biases.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8eebedbf",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** Victor Sanh, Lysandre Debut, Julien Chaumond, Thomas Wolf (Hugging Face)\n",
+ "- **Model type:** Transformer-based language model\n",
+ "- **Language(s) (NLP):** 104 languages; see full list [here](https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages)\n",
+ "- **License:** Apache 2.0\n",
+ "- **Related Models:** [BERT base multilingual model](https://huggingface.co/bert-base-multilingual-cased)\n",
+ "- **Resources for more information:**\n",
+ "- [GitHub Repository](https://github.com/huggingface/transformers/blob/main/examples/research_projects/distillation/README.md)\n",
+ "- [Associated Paper](https://arxiv.org/abs/1910.01108)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e9f48c0b",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f4dde273",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "b940cddf",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"distilbert-base-multilingual-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7ab62874",
+ "metadata": {},
+ "source": [
+ "# Citation\n",
+ "\n",
+ "```\n",
+ "@article{Sanh2019DistilBERTAD,\n",
+ " title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},\n",
+ " author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf},\n",
+ " journal={ArXiv},\n",
+ " year={2019},\n",
+ " volume={abs/1910.01108}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8bdb4ee1",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/distilbert-base-multilingual-cased ，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/distilbert-base-multilingual-cased/introduction_en.ipynb b/modelcenter/community/huggingface/distilbert-base-multilingual-cased/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..5fbfcda3645cdd95892818bf2ae7ec5f6c06ba74
--- /dev/null
+++ b/modelcenter/community/huggingface/distilbert-base-multilingual-cased/introduction_en.ipynb
@@ -0,0 +1,121 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "4260a150",
+ "metadata": {},
+ "source": [
+ "# Model Card for DistilBERT base multilingual (cased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "53f1b1c2",
+ "metadata": {},
+ "source": [
+ "This model is a distilled version of the [BERT base multilingual model](https://huggingface.co/bert-base-multilingual-cased/). The code for the distillation process can be found [here](https://github.com/huggingface/transformers/tree/main/examples/research_projects/distillation). This model is cased: it does make a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f417583b",
+ "metadata": {},
+ "source": [
+ "The model is trained on the concatenation of Wikipedia in 104 different languages listed [here](https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages).\n",
+ "The model has 6 layers, 768 dimension and 12 heads, totalizing 134M parameters (compared to 177M parameters for mBERT-base).\n",
+ "On average, this model, referred to as DistilmBERT, is twice as fast as mBERT-base.\n",
+ "\n",
+ "- **Developed by:** Victor Sanh, Lysandre Debut, Julien Chaumond, Thomas Wolf (Hugging Face)\n",
+ "- **Model type:** Transformer-based language model\n",
+ "- **Language(s) (NLP):** 104 languages; see full list [here](https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages)\n",
+ "- **License:** Apache 2.0\n",
+ "- **Related Models:** [BERT base multilingual model](https://huggingface.co/bert-base-multilingual-cased)\n",
+ "- **Resources for more information:**\n",
+ "- [GitHub Repository](https://github.com/huggingface/transformers/blob/main/examples/research_projects/distillation/README.md)\n",
+ "- [Associated Paper](https://arxiv.org/abs/1910.01108)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f47ce9b7",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "a1353b5f",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "e23a860f",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"distilbert-base-multilingual-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "38c30ea4",
+ "metadata": {},
+ "source": [
+ "# Citation\n",
+ "\n",
+ "```\n",
+ "@article{Sanh2019DistilBERTAD,\n",
+ " title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},\n",
+ " author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf},\n",
+ " journal={ArXiv},\n",
+ " year={2019},\n",
+ " volume={abs/1910.01108}\n",
+ "}\n",
+ "```\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0ee03d6a",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/distilbert-base-multilingual-cased and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/distilgpt2/info.yaml b/modelcenter/community/huggingface/distilgpt2/info.yaml
index dcaa018efe05d4f7727c71e81d3c0d7a33194b2d..070a0794d7e468856d5a6e14854e38576f2e5128 100644
--- a/modelcenter/community/huggingface/distilgpt2/info.yaml
+++ b/modelcenter/community/huggingface/distilgpt2/info.yaml
@@ -1,32 +1,30 @@
+Datasets: openwebtext
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "distilgpt2"
- description: "DistilGPT2"
- description_en: "DistilGPT2"
- icon: ""
- from_repo: "https://huggingface.co/distilgpt2"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Text Generation"
- sub_tag: "文本生成"
-
-Example:
-
-Datasets: "openwebtext"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: DistilGPT2
+ description_en: DistilGPT2
+ from_repo: https://huggingface.co/distilgpt2
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: distilgpt2
Paper:
- - title: 'DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter'
- url: 'http://arxiv.org/abs/1910.01108v4'
- - title: 'Can Model Compression Improve NLP Fairness'
- url: 'http://arxiv.org/abs/2201.08542v1'
- - title: 'Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal'
- url: 'http://arxiv.org/abs/2203.12574v1'
- - title: 'Quantifying the Carbon Emissions of Machine Learning'
- url: 'http://arxiv.org/abs/1910.09700v2'
- - title: 'Distilling the Knowledge in a Neural Network'
- url: 'http://arxiv.org/abs/1503.02531v1'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter'
+ url: http://arxiv.org/abs/1910.01108v4
+- title: Can Model Compression Improve NLP Fairness
+ url: http://arxiv.org/abs/2201.08542v1
+- title: Mitigating Gender Bias in Distilled Language Models via Counterfactual Role
+ Reversal
+ url: http://arxiv.org/abs/2203.12574v1
+- title: Quantifying the Carbon Emissions of Machine Learning
+ url: http://arxiv.org/abs/1910.09700v2
+- title: Distilling the Knowledge in a Neural Network
+ url: http://arxiv.org/abs/1503.02531v1
+Publisher: huggingface
+Task:
+- sub_tag: 文本生成
+ sub_tag_en: Text Generation
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/distilgpt2/introduction_cn.ipynb b/modelcenter/community/huggingface/distilgpt2/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..082df8e1455cbf27c99ec3367e9d3da99eecd325
--- /dev/null
+++ b/modelcenter/community/huggingface/distilgpt2/introduction_cn.ipynb
@@ -0,0 +1,142 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "72047643",
+ "metadata": {},
+ "source": [
+ "# DistilGPT2\n",
+ "\n",
+ "详细内容请看[GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "20c299c9",
+ "metadata": {},
+ "source": [
+ "DistilGPT2 (short for Distilled-GPT2) is an English-language model pre-trained with the supervision of the smallest version of Generative Pre-trained Transformer 2 (GPT-2). Like GPT-2, DistilGPT2 can be used to generate text. Users of this model card should also consider information about the design, training, and limitations of GPT-2.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c624b3d1",
+ "metadata": {},
+ "source": [
+ "## Model Details\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "92002396",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** Hugging Face\n",
+ "- **Model type:** Transformer-based Language Model\n",
+ "- **Language:** English\n",
+ "- **License:** Apache 2.0\n",
+ "- **Model Description:** DistilGPT2 is an English-language model pre-trained with the supervision of the 124 million parameter version of GPT-2. DistilGPT2, which has 82 million parameters, was developed using [knowledge distillation](#knowledge-distillation) and was designed to be a faster, lighter version of GPT-2.\n",
+ "- **Resources for more information:** See this repository for more about Distil\\* (a class of compressed models including Distilled-GPT2), [Sanh et al. (2019)](https://arxiv.org/abs/1910.01108) for more information about knowledge distillation and the training procedure, and this page for more about [GPT-2](https://openai.com/blog/better-language-models/).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a1a84778",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f9c6043d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "a9f0754d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"distilgpt2\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "03d3d465",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@inproceedings{sanh2019distilbert,\n",
+ "title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},\n",
+ "author={Sanh, Victor and Debut, Lysandre and Chaumond, Julien and Wolf, Thomas},\n",
+ "booktitle={NeurIPS EMC^2 Workshop},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7966636a",
+ "metadata": {},
+ "source": [
+ "## Glossary\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "533038ef",
+ "metadata": {},
+ "source": [
+ "-\t**Knowledge Distillation**: As described in [Sanh et al. (2019)](https://arxiv.org/pdf/1910.01108.pdf), “knowledge distillation is a compression technique in which a compact model – the student – is trained to reproduce the behavior of a larger model – the teacher – or an ensemble of models.” Also see [Bucila et al. (2006)](https://www.cs.cornell.edu/~caruana/compression.kdd06.pdf) and [Hinton et al. (2015)](https://arxiv.org/abs/1503.02531).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a7ff7cc1",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> 此模型介绍及权重来源于[https://huggingface.co/distilgpt2](https://huggingface.co/distilgpt2)，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/distilgpt2/introduction_en.ipynb b/modelcenter/community/huggingface/distilgpt2/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..d30a33f98bfba59963c9c0f6064a47604310f761
--- /dev/null
+++ b/modelcenter/community/huggingface/distilgpt2/introduction_en.ipynb
@@ -0,0 +1,134 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "1b34fb8a",
+ "metadata": {},
+ "source": [
+ "# DistilGPT2\n",
+ "\n",
+ "You can get more details from [GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f3ab8949",
+ "metadata": {},
+ "source": [
+ "DistilGPT2 (short for Distilled-GPT2) is an English-language model pre-trained with the supervision of the smallest version of Generative Pre-trained Transformer 2 (GPT-2). Like GPT-2, DistilGPT2 can be used to generate text. Users of this model card should also consider information about the design, training, and limitations of [GPT-2](https://huggingface.co/gpt2).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c6fbc1da",
+ "metadata": {},
+ "source": [
+ "## Model Details\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e2929e2f",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** Hugging Face\n",
+ "- **Model type:** Transformer-based Language Model\n",
+ "- **Language:** English\n",
+ "- **License:** Apache 2.0\n",
+ "- **Model Description:** DistilGPT2 is an English-language model pre-trained with the supervision of the 124 million parameter version of GPT-2. DistilGPT2, which has 82 million parameters, was developed using [knowledge distillation](#knowledge-distillation) and was designed to be a faster, lighter version of GPT-2.\n",
+ "- **Resources for more information:** See [this repository](https://github.com/huggingface/transformers/tree/main/examples/research_projects/distillation) for more about Distil\\* (a class of compressed models including Distilled-GPT2), [Sanh et al. (2019)](https://arxiv.org/abs/1910.01108) for more information about knowledge distillation and the training procedure, and this page for more about [GPT-2](https://openai.com/blog/better-language-models/).\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "5e226406",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "51f32d75",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"distilgpt2\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "adb84dc8",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@inproceedings{sanh2019distilbert,\n",
+ "title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},\n",
+ "author={Sanh, Victor and Debut, Lysandre and Chaumond, Julien and Wolf, Thomas},\n",
+ "booktitle={NeurIPS EMC^2 Workshop},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7d2aaec2",
+ "metadata": {},
+ "source": [
+ "## Glossary\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "004026dd",
+ "metadata": {},
+ "source": [
+ "-\t**Knowledge Distillation**: As described in [Sanh et al. (2019)](https://arxiv.org/pdf/1910.01108.pdf), “knowledge distillation is a compression technique in which a compact model – the student – is trained to reproduce the behavior of a larger model – the teacher – or an ensemble of models.” Also see [Bucila et al. (2006)](https://www.cs.cornell.edu/~caruana/compression.kdd06.pdf) and [Hinton et al. (2015)](https://arxiv.org/abs/1503.02531).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f8d12799",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> The model introduction and model weights originate from [https://huggingface.co/distilgpt2](https://huggingface.co/distilgpt2) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/distilroberta-base/info.yaml b/modelcenter/community/huggingface/distilroberta-base/info.yaml
index eceb32f20bb90e25e20224dfec78d49ec9788d95..dd53f5e49ce132520508d485856cf001db3e821b 100644
--- a/modelcenter/community/huggingface/distilroberta-base/info.yaml
+++ b/modelcenter/community/huggingface/distilroberta-base/info.yaml
@@ -1,26 +1,23 @@
+Datasets: openwebtext
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "distilroberta-base"
- description: "Model Card for DistilRoBERTa base"
- description_en: "Model Card for DistilRoBERTa base"
- icon: ""
- from_repo: "https://huggingface.co/distilroberta-base"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "openwebtext"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: Model Card for DistilRoBERTa base
+ description_en: Model Card for DistilRoBERTa base
+ from_repo: https://huggingface.co/distilroberta-base
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: distilroberta-base
Paper:
- - title: 'DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter'
- url: 'http://arxiv.org/abs/1910.01108v4'
- - title: 'Quantifying the Carbon Emissions of Machine Learning'
- url: 'http://arxiv.org/abs/1910.09700v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter'
+ url: http://arxiv.org/abs/1910.01108v4
+- title: Quantifying the Carbon Emissions of Machine Learning
+ url: http://arxiv.org/abs/1910.09700v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/distilroberta-base/introduction_cn.ipynb b/modelcenter/community/huggingface/distilroberta-base/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..db6d2867f872fd561a2cf9a9214b670ffcb95907
--- /dev/null
+++ b/modelcenter/community/huggingface/distilroberta-base/introduction_cn.ipynb
@@ -0,0 +1,126 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "7f49bb4b",
+ "metadata": {},
+ "source": [
+ "# Model Card for DistilRoBERTa base\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "88c832ab",
+ "metadata": {},
+ "source": [
+ "## Model Description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3a2333a1",
+ "metadata": {},
+ "source": [
+ "This model is a distilled version of the RoBERTa-base model. It follows the same training procedure as DistilBERT.\n",
+ "The code for the distillation process can be found [here](https://github.com/huggingface/transformers/tree/master/examples/distillation).\n",
+ "This model is case-sensitive: it makes a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9ac70255",
+ "metadata": {},
+ "source": [
+ "The model has 6 layers, 768 dimension and 12 heads, totalizing 82M parameters (compared to 125M parameters for RoBERTa-base).\n",
+ "On average DistilRoBERTa is twice as fast as Roberta-base.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a0757c23",
+ "metadata": {},
+ "source": [
+ "We encourage users of this model card to check out the RoBERTa-base model card to learn more about usage, limitations and potential biases.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2865466d",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** Victor Sanh, Lysandre Debut, Julien Chaumond, Thomas Wolf (Hugging Face)\n",
+ "- **Model type:** Transformer-based language model\n",
+ "- **Language(s) (NLP):** English\n",
+ "- **License:** Apache 2.0\n",
+ "- **Related Models:** RoBERTa-base model card\n",
+ "- [Associated Paper](https://arxiv.org/abs/1910.01108)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a204fad3",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "b2e488ed",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "43d7726b",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"distilroberta-base\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e30fb0eb",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> 此模型介绍及权重来源于[https://huggingface.co/distilroberta-base](https://huggingface.co/distilroberta-base)，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/distilroberta-base/introduction_en.ipynb b/modelcenter/community/huggingface/distilroberta-base/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..588b1d0b4d57a2f675df3597c925e7ec7e3c22af
--- /dev/null
+++ b/modelcenter/community/huggingface/distilroberta-base/introduction_en.ipynb
@@ -0,0 +1,128 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "4bd898ca",
+ "metadata": {},
+ "source": [
+ "# Model Card for DistilRoBERTa base\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7d39a086",
+ "metadata": {},
+ "source": [
+ "## Model Description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e2043d14",
+ "metadata": {},
+ "source": [
+ "This model is a distilled version of the [RoBERTa-base model](https://huggingface.co/roberta-base). It follows the same training procedure as [DistilBERT](https://huggingface.co/distilbert-base-uncased).\n",
+ "The code for the distillation process can be found [here](https://github.com/huggingface/transformers/tree/master/examples/distillation).\n",
+ "This model is case-sensitive: it makes a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "10aefe84",
+ "metadata": {},
+ "source": [
+ "The model has 6 layers, 768 dimension and 12 heads, totalizing 82M parameters (compared to 125M parameters for RoBERTa-base).\n",
+ "On average DistilRoBERTa is twice as fast as Roberta-base.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d7ebd775",
+ "metadata": {},
+ "source": [
+ "We encourage users of this model card to check out the [RoBERTa-base model card](https://huggingface.co/roberta-base) to learn more about usage, limitations and potential biases.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "423d28b1",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** Victor Sanh, Lysandre Debut, Julien Chaumond, Thomas Wolf (Hugging Face)\n",
+ "- **Model type:** Transformer-based language model\n",
+ "- **Language(s) (NLP):** English\n",
+ "- **License:** Apache 2.0\n",
+ "- **Related Models:** [RoBERTa-base model card](https://huggingface.co/roberta-base)\n",
+ "- **Resources for more information:**\n",
+ "- [GitHub Repository](https://github.com/huggingface/transformers/blob/main/examples/research_projects/distillation/README.md)\n",
+ "- [Associated Paper](https://arxiv.org/abs/1910.01108)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "715b4360",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "9ad9b1a9",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "94e4d093",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"distilroberta-base\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e258a20c",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> The model introduction and model weights originate from [https://huggingface.co/distilroberta-base](https://huggingface.co/distilroberta-base) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/gpt2-large/info.yaml b/modelcenter/community/huggingface/gpt2-large/info.yaml
index 5877d17dbe18cec42babfc4e04f8d7301117c5fa..b158fe17633b323d19c7632851709f40a6e2df8c 100644
--- a/modelcenter/community/huggingface/gpt2-large/info.yaml
+++ b/modelcenter/community/huggingface/gpt2-large/info.yaml
@@ -1,24 +1,21 @@
+Datasets: ''
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: mit
Model_Info:
- name: "gpt2-large"
- description: "GPT-2 Large"
- description_en: "GPT-2 Large"
- icon: ""
- from_repo: "https://huggingface.co/gpt2-large"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Text Generation"
- sub_tag: "文本生成"
-
-Example:
-
-Datasets: ""
-Publisher: "huggingface"
-License: "mit"
-Language: "English"
+ description: GPT-2 Large
+ description_en: GPT-2 Large
+ from_repo: https://huggingface.co/gpt2-large
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: gpt2-large
Paper:
- - title: 'Quantifying the Carbon Emissions of Machine Learning'
- url: 'http://arxiv.org/abs/1910.09700v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: Quantifying the Carbon Emissions of Machine Learning
+ url: http://arxiv.org/abs/1910.09700v2
+Publisher: huggingface
+Task:
+- sub_tag: 文本生成
+ sub_tag_en: Text Generation
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/gpt2-large/introduction_cn.ipynb b/modelcenter/community/huggingface/gpt2-large/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..0a394483916fd6e82621ac7bde9c979077d93bae
--- /dev/null
+++ b/modelcenter/community/huggingface/gpt2-large/introduction_cn.ipynb
@@ -0,0 +1,158 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "32b8730a",
+ "metadata": {},
+ "source": [
+ "# GPT-2 Large\n",
+ "\n",
+ "详细内容请看[GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "de66cac3",
+ "metadata": {},
+ "source": [
+ "## Table of Contents\n",
+ "- [Model Details](#model-details)\n",
+ "- [How To Get Started With the Model](#how-to-get-started-with-the-model)\n",
+ "- [Uses](#uses)\n",
+ "- [Risks, Limitations and Biases](#risks-limitations-and-biases)\n",
+ "- [Training](#training)\n",
+ "- [Evaluation](#evaluation)\n",
+ "- [Environmental Impact](#environmental-impact)\n",
+ "- [Technical Specifications](#technical-specifications)\n",
+ "- [Citation Information](#citation-information)\n",
+ "- [Model Card Authors](#model-card-author)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8afa58ef",
+ "metadata": {},
+ "source": [
+ "## Model Details\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e4e46496",
+ "metadata": {},
+ "source": [
+ "**Model Description:** GPT-2 Large is the **774M parameter** version of GPT-2, a transformer-based language model created and released by OpenAI. The model is a pretrained model on English language using a causal language modeling (CLM) objective.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "15b8f634",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** OpenAI, see [associated research paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) and [GitHub repo](https://github.com/openai/gpt-2) for model developers.\n",
+ "- **Model Type:** Transformer-based language model\n",
+ "- **Language(s):** English\n",
+ "- **License:** [Modified MIT License](https://github.com/openai/gpt-2/blob/master/LICENSE)\n",
+ "- **Related Models:** [GPT-2](https://huggingface.co/gpt2), [GPT-Medium](https://huggingface.co/gpt2-medium) and [GPT-XL](https://huggingface.co/gpt2-xl)\n",
+ "- **Resources for more information:**\n",
+ "- [Research Paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\n",
+ "- [OpenAI Blog Post](https://openai.com/blog/better-language-models/)\n",
+ "- [GitHub Repo](https://github.com/openai/gpt-2)\n",
+ "- [OpenAI Model Card for GPT-2](https://github.com/openai/gpt-2/blob/master/model_card.md)\n",
+ "- Test the full generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6c2023d9",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "b17e6efb",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "33c1f565",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"gpt2-large\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8060d283",
+ "metadata": {},
+ "source": [
+ "## Citatioin\n",
+ "\n",
+ "```\n",
+ "@article{radford2019language,\n",
+ "title={Language models are unsupervised multitask learners},\n",
+ "author={Radford, Alec and Wu, Jeffrey and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya and others},\n",
+ "journal={OpenAI blog},\n",
+ "volume={1},\n",
+ "number={8},\n",
+ "pages={9},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "083f0d9c",
+ "metadata": {},
+ "source": [
+ "## Model Card Authors\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f9e4bb43",
+ "metadata": {},
+ "source": [
+ "This model card was written by the Hugging Face team.\n",
+ "\n",
+ "> 此模型介绍及权重来源于[https://huggingface.co/gpt2-large](https://huggingface.co/gpt2-large)，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/gpt2-large/introduction_en.ipynb b/modelcenter/community/huggingface/gpt2-large/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..e3dce42d674f8a22441374995f0b1829b557c8d2
--- /dev/null
+++ b/modelcenter/community/huggingface/gpt2-large/introduction_en.ipynb
@@ -0,0 +1,157 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "dc26013b",
+ "metadata": {},
+ "source": [
+ "# GPT-2 Large\n",
+ "\n",
+ "You can get more details from [GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "38e29e37",
+ "metadata": {},
+ "source": [
+ "## Table of Contents\n",
+ "- [Model Details](#model-details)\n",
+ "- [How To Get Started With the Model](#how-to-get-started-with-the-model)\n",
+ "- [Uses](#uses)\n",
+ "- [Risks, Limitations and Biases](#risks-limitations-and-biases)\n",
+ "- [Training](#training)\n",
+ "- [Evaluation](#evaluation)\n",
+ "- [Environmental Impact](#environmental-impact)\n",
+ "- [Technical Specifications](#technical-specifications)\n",
+ "- [Citation Information](#citation-information)\n",
+ "- [Model Card Authors](#model-card-author)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "590c3fbd",
+ "metadata": {},
+ "source": [
+ "## Model Details\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1a2cd621",
+ "metadata": {},
+ "source": [
+ "**Model Description:** GPT-2 Large is the **774M parameter** version of GPT-2, a transformer-based language model created and released by OpenAI. The model is a pretrained model on English language using a causal language modeling (CLM) objective.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0155f43f",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** OpenAI, see [associated research paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) and [GitHub repo](https://github.com/openai/gpt-2) for model developers.\n",
+ "- **Model Type:** Transformer-based language model\n",
+ "- **Language(s):** English\n",
+ "- **License:** [Modified MIT License](https://github.com/openai/gpt-2/blob/master/LICENSE)\n",
+ "- **Related Models:** https://huggingface.co/gpt2, GPT-Medium and GPT-XL\n",
+ "- **Resources for more information:**\n",
+ "- [Research Paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\n",
+ "- [OpenAI Blog Post](https://openai.com/blog/better-language-models/)\n",
+ "- [GitHub Repo](https://github.com/openai/gpt-2)\n",
+ "- [OpenAI Model Card for GPT-2](https://github.com/openai/gpt-2/blob/master/model_card.md)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "18e2772d",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "30207821",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "2ae65fe6",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"gpt2-large\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e8b7c92b",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@article{radford2019language,\n",
+ "title={Language models are unsupervised multitask learners},\n",
+ "author={Radford, Alec and Wu, Jeffrey and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya and others},\n",
+ "journal={OpenAI blog},\n",
+ "volume={1},\n",
+ "number={8},\n",
+ "pages={9},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7cded70d",
+ "metadata": {},
+ "source": [
+ "## Model Card Authors\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ff9ab2d4",
+ "metadata": {},
+ "source": [
+ "This model card was written by the Hugging Face team.\n",
+ "\n",
+ "> The model introduction and model weights originate from [https://huggingface.co/gpt2-large](https://huggingface.co/gpt2-large) and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/gpt2-medium/info.yaml b/modelcenter/community/huggingface/gpt2-medium/info.yaml
index 2b8d53c8f1c561033b151f1db7952714b87c2a3b..3e03ccdfaa3d853f3a1a105b4c87a75364a90d32 100644
--- a/modelcenter/community/huggingface/gpt2-medium/info.yaml
+++ b/modelcenter/community/huggingface/gpt2-medium/info.yaml
@@ -1,24 +1,21 @@
+Datasets: ''
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: mit
Model_Info:
- name: "gpt2-medium"
- description: "GPT-2 Medium"
- description_en: "GPT-2 Medium"
- icon: ""
- from_repo: "https://huggingface.co/gpt2-medium"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Text Generation"
- sub_tag: "文本生成"
-
-Example:
-
-Datasets: ""
-Publisher: "huggingface"
-License: "mit"
-Language: "English"
+ description: GPT-2 Medium
+ description_en: GPT-2 Medium
+ from_repo: https://huggingface.co/gpt2-medium
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: gpt2-medium
Paper:
- - title: 'Quantifying the Carbon Emissions of Machine Learning'
- url: 'http://arxiv.org/abs/1910.09700v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: Quantifying the Carbon Emissions of Machine Learning
+ url: http://arxiv.org/abs/1910.09700v2
+Publisher: huggingface
+Task:
+- sub_tag: 文本生成
+ sub_tag_en: Text Generation
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/gpt2-medium/introduction_cn.ipynb b/modelcenter/community/huggingface/gpt2-medium/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..cfd546c3fd22c765bb9859f1558b6dc36244c14a
--- /dev/null
+++ b/modelcenter/community/huggingface/gpt2-medium/introduction_cn.ipynb
@@ -0,0 +1,140 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "25324e9c",
+ "metadata": {},
+ "source": [
+ "# GPT-2 Medium\n",
+ "\n",
+ "详细内容请看[GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "806177e3",
+ "metadata": {},
+ "source": [
+ "## Model Details\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dbcaecb0",
+ "metadata": {},
+ "source": [
+ "**Model Description:** GPT-2 Medium is the **355M parameter** version of GPT-2, a transformer-based language model created and released by OpenAI. The model is a pretrained model on English language using a causal language modeling (CLM) objective.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ab73e9f0",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** OpenAI, see [associated research paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) and [GitHub repo](https://github.com/openai/gpt-2) for model developers.\n",
+ "- **Model Type:** Transformer-based language model\n",
+ "- **Language(s):** English\n",
+ "- **License:** [Modified MIT License](https://github.com/openai/gpt-2/blob/master/LICENSE)\n",
+ "- **Related Models:** [GPT2](https://huggingface.co/gpt2), [GPT2-Large](https://huggingface.co/gpt2-large) and [GPT2-XL](https://huggingface.co/gpt2-xl)\n",
+ "- **Resources for more information:**\n",
+ "- [Research Paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\n",
+ "- [OpenAI Blog Post](https://openai.com/blog/better-language-models/)\n",
+ "- [GitHub Repo](https://github.com/openai/gpt-2)\n",
+ "- [OpenAI Model Card for GPT-2](https://github.com/openai/gpt-2/blob/master/model_card.md)\n",
+ "- Test the full generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "70c3fd36",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "1bae5ee0",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "11b32577",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"gpt2-medium\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "08f90ea0",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@article{radford2019language,\n",
+ "title={Language models are unsupervised multitask learners},\n",
+ "author={Radford, Alec and Wu, Jeffrey and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya and others},\n",
+ "journal={OpenAI blog},\n",
+ "volume={1},\n",
+ "number={8},\n",
+ "pages={9},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "64d79312",
+ "metadata": {},
+ "source": [
+ "## Model Card Authors\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d14dd2ac",
+ "metadata": {},
+ "source": [
+ "This model card was written by the Hugging Face team.\n",
+ "\n",
+ "> 此模型介绍及权重来源于 https://huggingface.co/gpt2-medium ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/gpt2-medium/introduction_en.ipynb b/modelcenter/community/huggingface/gpt2-medium/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..a324c5b99fbd68795118c6bb0a1ef47bd710cfad
--- /dev/null
+++ b/modelcenter/community/huggingface/gpt2-medium/introduction_en.ipynb
@@ -0,0 +1,146 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "46995787",
+ "metadata": {},
+ "source": [
+ "# GPT-2 Medium\n",
+ "\n",
+ "You can get more details from [GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f695ad73",
+ "metadata": {},
+ "source": [
+ "## Model Details\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5a8170d9",
+ "metadata": {},
+ "source": [
+ "**Model Description:** GPT-2 Medium is the **355M parameter** version of GPT-2, a transformer-based language model created and released by OpenAI. The model is a pretrained model on English language using a causal language modeling (CLM) objective.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1d0dc244",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** OpenAI, see [associated research paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) and [GitHub repo](https://github.com/openai/gpt-2) for model developers.\n",
+ "- **Model Type:** Transformer-based language model\n",
+ "- **Language(s):** English\n",
+ "- **License:** [Modified MIT License](https://github.com/openai/gpt-2/blob/master/LICENSE)\n",
+ "- **Related Models:** [GPT2](https://huggingface.co/gpt2), [GPT2-Large](https://huggingface.co/gpt2-large) and [GPT2-XL](https://huggingface.co/gpt2-xl)\n",
+ "- **Resources for more information:**\n",
+ "- [Research Paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\n",
+ "- [OpenAI Blog Post](https://openai.com/blog/better-language-models/)\n",
+ "- [GitHub Repo](https://github.com/openai/gpt-2)\n",
+ "- [OpenAI Model Card for GPT-2](https://github.com/openai/gpt-2/blob/master/model_card.md)\n",
+ "- Test the full generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "adc5a3f9",
+ "metadata": {},
+ "source": [
+ "## How to Get Started with the Model\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7566eafd",
+ "metadata": {},
+ "source": [
+ "Use the code below to get started with the model. \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "ab4c71ee",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "b0167528",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"gpt2-medium\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "52cdcf9e",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@article{radford2019language,\n",
+ "title={Language models are unsupervised multitask learners},\n",
+ "author={Radford, Alec and Wu, Jeffrey and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya and others},\n",
+ "journal={OpenAI blog},\n",
+ "volume={1},\n",
+ "number={8},\n",
+ "pages={9},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "eb327c10",
+ "metadata": {},
+ "source": [
+ "## Model Card Authors\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "50fb7de8",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/gpt2-medium and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/gpt2/info.yaml b/modelcenter/community/huggingface/gpt2/info.yaml
index 6663ffdf613fdbd2b2ff142fd02bdc37040926a4..ce027f72352227f8f10cae7382c95a2f8c46b66e 100644
--- a/modelcenter/community/huggingface/gpt2/info.yaml
+++ b/modelcenter/community/huggingface/gpt2/info.yaml
@@ -1,23 +1,19 @@
+Datasets: ''
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: mit
Model_Info:
- name: "gpt2"
- description: "GPT-2"
- description_en: "GPT-2"
- icon: ""
- from_repo: "https://huggingface.co/gpt2"
-
+ description: GPT-2
+ description_en: GPT-2
+ from_repo: https://huggingface.co/gpt2
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: gpt2
+Paper: null
+Publisher: huggingface
Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Text Generation"
- sub_tag: "文本生成"
-
-Example:
-
-Datasets: ""
-Publisher: "huggingface"
-License: "mit"
-Language: "English"
-Paper:
-
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- sub_tag: 文本生成
+ sub_tag_en: Text Generation
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/gpt2/introduction_cn.ipynb b/modelcenter/community/huggingface/gpt2/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..a7849400e0f849fd784d2c35b76d5018269a3745
--- /dev/null
+++ b/modelcenter/community/huggingface/gpt2/introduction_cn.ipynb
@@ -0,0 +1,160 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "a4cd103f",
+ "metadata": {},
+ "source": [
+ "# GPT-2\n",
+ "\n",
+ "详细内容请看[GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e10dfe6d",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in\n",
+ "[this paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\n",
+ "and first released at [this page](https://openai.com/blog/better-language-models/).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d1b13043",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing GPT-2 also wrote a\n",
+ "[model card](https://github.com/openai/gpt-2/blob/master/model_card.md) for their model. Content from this model card\n",
+ "has been written by the Hugging Face team to complete the information they provided and give specific examples of bias.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "016271a5",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e3a53155",
+ "metadata": {},
+ "source": [
+ "GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This\n",
+ "means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots\n",
+ "of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely,\n",
+ "it was trained to guess the next word in sentences.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6836ad17",
+ "metadata": {},
+ "source": [
+ "More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence,\n",
+ "shifted one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the\n",
+ "predictions for the token `i` only uses the inputs from `1` to `i` but not the future tokens.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "26946ce6",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks. The model is best at what it was pretrained for however, which is generating texts from a\n",
+ "prompt.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "571b41cf",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "a6233e8e",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "2e906136",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"gpt2\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "78f26b7f",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@article{radford2019language,\n",
+ "title={Language Models are Unsupervised Multitask Learners},\n",
+ "author={Radford, Alec and Wu, Jeff and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2f646c57",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> 此模型介绍及权重来源于[https://huggingface.co/gpt2](https://huggingface.co/gpt2)，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ },
+ "vscode": {
+ "interpreter": {
+ "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5"
+ }
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/gpt2/introduction_en.ipynb b/modelcenter/community/huggingface/gpt2/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..fe5a7baa774ebf7410f52919235da2b61a6d78b8
--- /dev/null
+++ b/modelcenter/community/huggingface/gpt2/introduction_en.ipynb
@@ -0,0 +1,164 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "2d373572",
+ "metadata": {},
+ "source": [
+ "# GPT-2\n",
+ "\n",
+ "You can get more details from [GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "00be5831",
+ "metadata": {},
+ "source": [
+ "Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b5857cc2",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in\n",
+ "[this paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\n",
+ "and first released at [this page](https://openai.com/blog/better-language-models/).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b0abac76",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing GPT-2 also wrote a\n",
+ "[model card](https://github.com/openai/gpt-2/blob/master/model_card.md) for their model. Content from this model card\n",
+ "has been written by the Hugging Face team to complete the information they provided and give specific examples of bias.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "fa2c7f4b",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "294521bd",
+ "metadata": {},
+ "source": [
+ "GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This\n",
+ "means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots\n",
+ "of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely,\n",
+ "it was trained to guess the next word in sentences.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b1204c32",
+ "metadata": {},
+ "source": [
+ "More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence,\n",
+ "shifted one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the\n",
+ "predictions for the token `i` only uses the inputs from `1` to `i` but not the future tokens.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a019cc9e",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks. The model is best at what it was pretrained for however, which is generating texts from a\n",
+ "prompt.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "54ae8500",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "d33fddda",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "d0e160c6",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"gpt2\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "fcb8a843",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@article{radford2019language,\n",
+ "title={Language Models are Unsupervised Multitask Learners},\n",
+ "author={Radford, Alec and Wu, Jeff and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "513848f8",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> The model introduction and model weights originate from [https://huggingface.co/gpt2](https://huggingface.co/gpt2) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/indobenchmark/indobert-base-p1/info.yaml b/modelcenter/community/indobenchmark/indobert-base-p1/info.yaml
index a27d7a18a17f9f8b9dc3279e374080c3e975199d..4bf70a48f44f1e9b8cc532f0d7a220c0df026af3 100644
--- a/modelcenter/community/indobenchmark/indobert-base-p1/info.yaml
+++ b/modelcenter/community/indobenchmark/indobert-base-p1/info.yaml
@@ -1,24 +1,22 @@
+Datasets: indonlu
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: Indonesian
+License: mit
Model_Info:
- name: "indobenchmark/indobert-base-p1"
- description: "IndoBERT Base Model (phase1 - uncased)"
- description_en: "IndoBERT Base Model (phase1 - uncased)"
- icon: ""
- from_repo: "https://huggingface.co/indobenchmark/indobert-base-p1"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Feature Extraction"
- sub_tag: "特征抽取"
-
-Example:
-
-Datasets: "indonlu"
-Publisher: "indobenchmark"
-License: "mit"
-Language: "Indonesian"
+ description: IndoBERT Base Model (phase1 - uncased)
+ description_en: IndoBERT Base Model (phase1 - uncased)
+ from_repo: https://huggingface.co/indobenchmark/indobert-base-p1
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: indobenchmark/indobert-base-p1
Paper:
- - title: 'IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding'
- url: 'http://arxiv.org/abs/2009.05387v3'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language
+ Understanding'
+ url: http://arxiv.org/abs/2009.05387v3
+Publisher: indobenchmark
+Task:
+- sub_tag: 特征抽取
+ sub_tag_en: Feature Extraction
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/indobenchmark/indobert-base-p1/introduction_cn.ipynb b/modelcenter/community/indobenchmark/indobert-base-p1/introduction_cn.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..cc12f5618c3f8699ca5ffb00a9ddef1109896661
--- /dev/null
+++ b/modelcenter/community/indobenchmark/indobert-base-p1/introduction_cn.ipynb
@@ -0,0 +1,124 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "3f5a12e4",
+ "metadata": {},
+ "source": [
+ "# IndoBERT Base Model (phase1 - uncased)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e2fcac01",
+ "metadata": {},
+ "source": [
+ "[IndoBERT](https://arxiv.org/abs/2009.05387) is a state-of-the-art language model for Indonesian based on the BERT model. The pretrained model is trained using a masked language modeling (MLM) objective and next sentence prediction (NSP) objective.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6a9d6a02",
+ "metadata": {},
+ "source": [
+ "## All Pre-trained Models\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3020975b",
+ "metadata": {},
+ "source": [
+ "| Model | #params | Arch. | Training data |\n",
+ "|--------------------------------|--------------------------------|-------|-----------------------------------|\n",
+ "| `indobenchmark/indobert-base-p1` | 124.5M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-base-p2` | 124.5M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-large-p1` | 335.2M | Large | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-large-p2` | 335.2M | Large | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-base-p1` | 11.7M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-base-p2` | 11.7M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-large-p1` | 17.7M | Large | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-large-p2` | 17.7M | Large | Indo4B (23.43 GB of text) |\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d0e3771a",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f1f38760",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "a11bc38f",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"indobenchmark/indobert-base-p1\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d1fe4366",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@inproceedings{wilie2020indonlu,\n",
+ "title={IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding},\n",
+ "author={Bryan Wilie and Karissa Vincentio and Genta Indra Winata and Samuel Cahyawijaya and X. Li and Zhi Yuan Lim and S. Soleman and R. Mahendra and Pascale Fung and Syafri Bahar and A. Purwarianti},\n",
+ "booktitle={Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing},\n",
+ "year={2020}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "95f83dc9",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于[https://huggingface.co/indobenchmark/indobert-base-p1](https://huggingface.co/indobenchmark/indobert-base-p1)，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/indobenchmark/indobert-base-p1/introduction_en.ipynb b/modelcenter/community/indobenchmark/indobert-base-p1/introduction_en.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..ec38f774dc314e6a43a5ec7b72f4419c4ffed65b
--- /dev/null
+++ b/modelcenter/community/indobenchmark/indobert-base-p1/introduction_en.ipynb
@@ -0,0 +1,124 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "d6793868",
+ "metadata": {},
+ "source": [
+ "# IndoBERT Base Model (phase1 - uncased)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "48b35590",
+ "metadata": {},
+ "source": [
+ "[IndoBERT](https://arxiv.org/abs/2009.05387) is a state-of-the-art language model for Indonesian based on the BERT model. The pretrained model is trained using a masked language modeling (MLM) objective and next sentence prediction (NSP) objective.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e5dc323c",
+ "metadata": {},
+ "source": [
+ "## All Pre-trained Models\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7db5d6e5",
+ "metadata": {},
+ "source": [
+ "| Model | #params | Arch. | Training data |\n",
+ "|--------------------------------|--------------------------------|-------|-----------------------------------|\n",
+ "| `indobenchmark/indobert-base-p1` | 124.5M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-base-p2` | 124.5M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-large-p1` | 335.2M | Large | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-large-p2` | 335.2M | Large | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-base-p1` | 11.7M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-base-p2` | 11.7M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-large-p1` | 17.7M | Large | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-large-p2` | 17.7M | Large | Indo4B (23.43 GB of text) |\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "fc8827fd",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "e5b6e205",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "6701163d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"indobenchmark/indobert-base-p1\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "fb28cf5b",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@inproceedings{wilie2020indonlu,\n",
+ "title={IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding},\n",
+ "author={Bryan Wilie and Karissa Vincentio and Genta Indra Winata and Samuel Cahyawijaya and X. Li and Zhi Yuan Lim and S. Soleman and R. Mahendra and Pascale Fung and Syafri Bahar and A. Purwarianti},\n",
+ "booktitle={Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing},\n",
+ "year={2020}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e155d1ce",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from [https://huggingface.co/indobenchmark/indobert-base-p1](https://huggingface.co/indobenchmark/indobert-base-p1) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/johngiorgi/declutr-base/download_cn.md b/modelcenter/community/johngiorgi/declutr-base/download_cn.md
deleted file mode 100644
index 90761ce377dcfc6b0c3cf3a63576eaa42ed0dc14..0000000000000000000000000000000000000000
--- a/modelcenter/community/johngiorgi/declutr-base/download_cn.md
+++ /dev/null
@@ -1,23 +0,0 @@
-# 模型列表
-
-## johngiorgi/declutr-base
-
-| 模型名称 | 模型介绍 | 模型大小 | 模型下载 |
-| --- | --- | --- | --- |
-|johngiorgi/declutr-base| | 625.22MB | [merges.txt](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/merges.txt)[model_config.json](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/tokenizer_config.json)
[vocab.json](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/vocab.json)
[vocab.txt](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/vocab.txt) | - -也可以通过`paddlenlp` cli 工具来下载对应的模型权重，使用步骤如下所示： - -* 安装paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* 下载命令行 - -```shell -paddlenlp download --cache-dir ./pretrained_models johngiorgi/declutr-base -``` - -有任何下载的问题都可以到[PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP)中发Issue提问。 \ No newline at end of file diff --git a/modelcenter/community/johngiorgi/declutr-base/download_en.md b/modelcenter/community/johngiorgi/declutr-base/download_en.md deleted file mode 100644 index f25c8efd028cdb5c0d2386f0d6f173fa31eebd39..0000000000000000000000000000000000000000 --- a/modelcenter/community/johngiorgi/declutr-base/download_en.md +++ /dev/null @@ -1,23 +0,0 @@ -# model list - -## - -| model | description | model_size | download | -| --- | --- | --- | --- | -|johngiorgi/declutr-base| | 625.22MB | [merges.txt](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/merges.txt)
[model_config.json](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/tokenizer_config.json)
[vocab.json](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/vocab.json)
[vocab.txt](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/vocab.txt) | - -or you can download all of model file with the following steps: - -* install paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* download model with cli tool - -```shell -paddlenlp download --cache-dir ./pretrained_models johngiorgi/declutr-base -``` - -If you have any problems with it, you can post issue on [PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP) to get support. diff --git a/modelcenter/community/johngiorgi/declutr-base/info.yaml b/modelcenter/community/johngiorgi/declutr-base/info.yaml deleted file mode 100644 index 86e42194738a2c421d9180847dbbbb371887cb91..0000000000000000000000000000000000000000 --- a/modelcenter/community/johngiorgi/declutr-base/info.yaml +++ /dev/null @@ -1,28 +0,0 @@ -Model_Info: - name: "johngiorgi/declutr-base" - description: "DeCLUTR-base" - description_en: "DeCLUTR-base" - icon: "" - from_repo: "https://huggingface.co/johngiorgi/declutr-base" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Sentence Similarity" - sub_tag: "句子相似度" -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Feature Extraction" - sub_tag: "特征抽取" - -Example: - -Datasets: "openwebtext" -Publisher: "johngiorgi" -License: "apache-2.0" -Language: "English" -Paper: - - title: 'DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations' - url: 'http://arxiv.org/abs/2006.03659v4' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file