[model_config.json](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/tokenizer_config.json)
[vocab.json](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/vocab.json)
[vocab.txt](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/vocab.txt) | - -也可以通过`paddlenlp` cli 工具来下载对应的模型权重，使用步骤如下所示： - -* 安装paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* 下载命令行 - -```shell -paddlenlp download --cache-dir ./pretrained_models csarron/roberta-base-squad-v1 -``` - -有任何下载的问题都可以到[PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP)中发Issue提问。 \ No newline at end of file diff --git a/modelcenter/community/csarron/roberta-base-squad-v1/download_en.md b/modelcenter/community/csarron/roberta-base-squad-v1/download_en.md deleted file mode 100644 index bbdc2aeb..00000000 --- a/modelcenter/community/csarron/roberta-base-squad-v1/download_en.md +++ /dev/null @@ -1,23 +0,0 @@ -# model list - -## - -| model | description | model_size | download | -| --- | --- | --- | --- | -|csarron/roberta-base-squad-v1| | 475.51MB | [merges.txt](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/merges.txt)
[model_config.json](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/tokenizer_config.json)
[vocab.json](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/vocab.json)
[vocab.txt](https://bj.bcebos.com/paddlenlp/models/community/csarron/roberta-base-squad-v1/vocab.txt) | - -or you can download all of model file with the following steps: - -* install paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* download model with cli tool - -```shell -paddlenlp download --cache-dir ./pretrained_models csarron/roberta-base-squad-v1 -``` - -If you have any problems with it, you can post issue on [PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP) to get support. diff --git a/modelcenter/community/csarron/roberta-base-squad-v1/info.yaml b/modelcenter/community/csarron/roberta-base-squad-v1/info.yaml deleted file mode 100644 index 1095b2a2..00000000 --- a/modelcenter/community/csarron/roberta-base-squad-v1/info.yaml +++ /dev/null @@ -1,24 +0,0 @@ -Model_Info: - name: "csarron/roberta-base-squad-v1" - description: "RoBERTa-base fine-tuned on SQuAD v1" - description_en: "RoBERTa-base fine-tuned on SQuAD v1" - icon: "" - from_repo: "https://huggingface.co/csarron/roberta-base-squad-v1" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Question Answering" - sub_tag: "回答问题" - -Example: - -Datasets: "squad" -Publisher: "csarron" -License: "mit" -Language: "English" -Paper: - - title: 'RoBERTa: A Robustly Optimized BERT Pretraining Approach' - url: 'http://arxiv.org/abs/1907.11692v1' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file diff --git a/modelcenter/community/dbmdz/bert-base-german-cased/info.yaml b/modelcenter/community/dbmdz/bert-base-german-cased/info.yaml index 2eb9d8d9..ba61f657 100644 --- a/modelcenter/community/dbmdz/bert-base-german-cased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-german-cased/info.yaml @@ -1,23 +1,19 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: German +License: mit Model_Info: - name: "dbmdz/bert-base-german-cased" - description: "🤗 + 📚 dbmdz German BERT models" - description_en: "🤗 + 📚 dbmdz German BERT models" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-german-cased" - + description: 🤗 + 📚 dbmdz German BERT models + description_en: 🤗 + 📚 dbmdz German BERT models + from_repo: https://huggingface.co/dbmdz/bert-base-german-cased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-german-cased +Paper: null +Publisher: dbmdz Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "dbmdz" -License: "mit" -Language: "German" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dbmdz/bert-base-german-cased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-german-cased/introduction_cn.ipynb new file mode 100644 index 00000000..2a4843b0 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-german-cased/introduction_cn.ipynb @@ -0,0 +1,95 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "9fb28341", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz German BERT models\n", + "\n", + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources another German BERT models 🎉\n", + "\n", + "# German BERT\n", + "\n", + "## Stats\n", + "\n", + "In addition to the recently released [German BERT](https://deepset.ai/german-bert)\n", + "model by [deepset](https://deepset.ai/) we provide another German-language model.\n", + "\n", + "The source data for the model consists of a recent Wikipedia dump, EU Bookshop corpus,\n", + "Open Subtitles, CommonCrawl, ParaCrawl and News Crawl. This results in a dataset with\n", + "a size of 16GB and 2,350,234,427 tokens.\n", + "\n", + "For sentence splitting, we use [spacy](https://spacy.io/). Our preprocessing steps\n", + "(sentence piece model for vocab generation) follow those used for training\n", + "[SciBERT](https://github.com/allenai/scibert). The model is trained with an initial\n", + "sequence length of 512 subwords and was performed for 1.5M steps." + ] + }, + { + "cell_type": "markdown", + "id": "589fadf4", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "646e12d4", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5935d3e0", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-german-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "b05add24", + "metadata": {}, + "source": [ + "# Reference\n", + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-german-cased](https://huggingface.co/dbmdz/bert-base-german-cased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-german-cased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-german-cased/introduction_en.ipynb new file mode 100644 index 00000000..8f9e678c --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-german-cased/introduction_en.ipynb @@ -0,0 +1,93 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "e875e0cc", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz German BERT models\n", + "\n", + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources another German BERT models 🎉\n", + "\n", + "# German BERT\n", + "\n", + "## Stats\n", + "\n", + "In addition to the recently released [German BERT](https://deepset.ai/german-bert)\n", + "model by [deepset](https://deepset.ai/) we provide another German-language model.\n", + "\n", + "The source data for the model consists of a recent Wikipedia dump, EU Bookshop corpus,\n", + "Open Subtitles, CommonCrawl, ParaCrawl and News Crawl. This results in a dataset with\n", + "a size of 16GB and 2,350,234,427 tokens.\n", + "\n", + "For sentence splitting, we use [spacy](https://spacy.io/). Our preprocessing steps\n", + "(sentence piece model for vocab generation) follow those used for training\n", + "[SciBERT](https://github.com/allenai/scibert). The model is trained with an initial\n", + "sequence length of 512 subwords and was performed for 1.5M steps." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8dcad967", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c7c65281", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-german-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "1b52feb8", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "bc00304a", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-german-cased](https://huggingface.co/dbmdz/bert-base-german-cased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-german-uncased/info.yaml b/modelcenter/community/dbmdz/bert-base-german-uncased/info.yaml index 519dcaa0..da3e1629 100644 --- a/modelcenter/community/dbmdz/bert-base-german-uncased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-german-uncased/info.yaml @@ -1,23 +1,19 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: German +License: mit Model_Info: - name: "dbmdz/bert-base-german-uncased" - description: "🤗 + 📚 dbmdz German BERT models" - description_en: "🤗 + 📚 dbmdz German BERT models" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-german-uncased" - + description: 🤗 + 📚 dbmdz German BERT models + description_en: 🤗 + 📚 dbmdz German BERT models + from_repo: https://huggingface.co/dbmdz/bert-base-german-uncased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-german-uncased +Paper: null +Publisher: dbmdz Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "dbmdz" -License: "mit" -Language: "German" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dbmdz/bert-base-german-uncased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-german-uncased/introduction_cn.ipynb new file mode 100644 index 00000000..9490ad20 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-german-uncased/introduction_cn.ipynb @@ -0,0 +1,102 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "46b7bbb6", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz German BERT models\n", + "\n", + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources another German BERT models 🎉\n", + "\n", + "# German BERT\n", + "\n", + "## Stats\n", + "\n", + "In addition to the recently released [German BERT](https://deepset.ai/german-bert)\n", + "model by [deepset](https://deepset.ai/) we provide another German-language model.\n", + "\n", + "The source data for the model consists of a recent Wikipedia dump, EU Bookshop corpus,\n", + "Open Subtitles, CommonCrawl, ParaCrawl and News Crawl. This results in a dataset with\n", + "a size of 16GB and 2,350,234,427 tokens.\n", + "\n", + "For sentence splitting, we use [spacy](https://spacy.io/). Our preprocessing steps\n", + "(sentence piece model for vocab generation) follow those used for training\n", + "[SciBERT](https://github.com/allenai/scibert). The model is trained with an initial\n", + "sequence length of 512 subwords and was performed for 1.5M steps." + ] + }, + { + "cell_type": "markdown", + "id": "bc37d3e3", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2afff18c", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "967f058e", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-german-uncased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "483dbced", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "04e50d8c", + "metadata": {}, + "source": [ + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-german-uncased](https://huggingface.co/dbmdz/bert-base-german-uncased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-german-uncased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-german-uncased/introduction_en.ipynb new file mode 100644 index 00000000..3c19f2ff --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-german-uncased/introduction_en.ipynb @@ -0,0 +1,101 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "5e0d446c", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz German BERT models\n", + "\n", + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources another German BERT models 🎉\n", + "\n", + "# German BERT\n", + "\n", + "## Stats\n", + "\n", + "In addition to the recently released [German BERT](https://deepset.ai/german-bert)\n", + "model by [deepset](https://deepset.ai/) we provide another German-language model.\n", + "\n", + "The source data for the model consists of a recent Wikipedia dump, EU Bookshop corpus,\n", + "Open Subtitles, CommonCrawl, ParaCrawl and News Crawl. This results in a dataset with\n", + "a size of 16GB and 2,350,234,427 tokens.\n", + "\n", + "For sentence splitting, we use [spacy](https://spacy.io/). Our preprocessing steps\n", + "(sentence piece model for vocab generation) follow those used for training\n", + "[SciBERT](https://github.com/allenai/scibert). The model is trained with an initial\n", + "sequence length of 512 subwords and was performed for 1.5M steps." + ] + }, + { + "cell_type": "markdown", + "id": "524680d5", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "39332440", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "19cf118e", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-german-uncased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "fb81d709", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "747fd5d3", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-german-uncased](https://huggingface.co/dbmdz/bert-base-german-uncased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-italian-uncased/info.yaml b/modelcenter/community/dbmdz/bert-base-italian-uncased/info.yaml index 9aea3699..1072ac17 100644 --- a/modelcenter/community/dbmdz/bert-base-italian-uncased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-italian-uncased/info.yaml @@ -1,23 +1,19 @@ +Datasets: wikipedia +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Italian +License: mit Model_Info: - name: "dbmdz/bert-base-italian-uncased" - description: "🤗 + 📚 dbmdz BERT and ELECTRA models" - description_en: "🤗 + 📚 dbmdz BERT and ELECTRA models" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-italian-uncased" - + description: 🤗 + 📚 dbmdz BERT and ELECTRA models + description_en: 🤗 + 📚 dbmdz BERT and ELECTRA models + from_repo: https://huggingface.co/dbmdz/bert-base-italian-uncased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-italian-uncased +Paper: null +Publisher: dbmdz Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "wikipedia" -Publisher: "dbmdz" -License: "mit" -Language: "Italian" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dbmdz/bert-base-italian-uncased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-italian-uncased/introduction_cn.ipynb new file mode 100644 index 00000000..eebce580 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-italian-uncased/introduction_cn.ipynb @@ -0,0 +1,152 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "dea2fc9e", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz BERT and ELECTRA models\n" + ] + }, + { + "cell_type": "markdown", + "id": "00744cbd", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources Italian BERT and ELECTRA models 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "d7106b74", + "metadata": {}, + "source": [ + "# Italian BERT\n" + ] + }, + { + "cell_type": "markdown", + "id": "7ee0fd67", + "metadata": {}, + "source": [ + "The source data for the Italian BERT model consists of a recent Wikipedia dump and\n", + "various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final\n", + "training corpus has a size of 13GB and 2,050,057,573 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "a3961910", + "metadata": {}, + "source": [ + "For sentence splitting, we use NLTK (faster compared to spacy).\n", + "Our cased and uncased models are training with an initial sequence length of 512\n", + "subwords for ~2-3M steps.\n" + ] + }, + { + "cell_type": "markdown", + "id": "480e4fea", + "metadata": {}, + "source": [ + "For the XXL Italian models, we use the same training data from OPUS and extend\n", + "it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/).\n", + "Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "d710804e", + "metadata": {}, + "source": [ + "Note: Unfortunately, a wrong vocab size was used when training the XXL models.\n", + "This explains the mismatch of the \"real\" vocab size of 31102, compared to the\n", + "vocab size specified in `config.json`. However, the model is working and all\n", + "evaluations were done under those circumstances.\n", + "See [this issue](https://github.com/dbmdz/berts/issues/7) for more information.\n" + ] + }, + { + "cell_type": "markdown", + "id": "2d9c79e5", + "metadata": {}, + "source": [ + "The Italian ELECTRA model was trained on the \"XXL\" corpus for 1M steps in total using a batch\n", + "size of 128. We pretty much following the ELECTRA training procedure as used for\n", + "[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra).\n" + ] + }, + { + "cell_type": "markdown", + "id": "3ee71cee", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9ffe9a93", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "82d327d4", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-italian-uncased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "56d92161", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "ad146f63", + "metadata": {}, + "source": [ + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-italian-uncased](https://huggingface.co/dbmdz/bert-base-italian-uncased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-italian-uncased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-italian-uncased/introduction_en.ipynb new file mode 100644 index 00000000..843ea2d2 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-italian-uncased/introduction_en.ipynb @@ -0,0 +1,151 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "8601b7e0", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz BERT and ELECTRA models\n" + ] + }, + { + "cell_type": "markdown", + "id": "2e2ee06f", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources Italian BERT and ELECTRA models 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "a7b6e470", + "metadata": {}, + "source": [ + "# Italian BERT\n" + ] + }, + { + "cell_type": "markdown", + "id": "d1afb03c", + "metadata": {}, + "source": [ + "The source data for the Italian BERT model consists of a recent Wikipedia dump and\n", + "various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final\n", + "training corpus has a size of 13GB and 2,050,057,573 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "a900d41a", + "metadata": {}, + "source": [ + "For sentence splitting, we use NLTK (faster compared to spacy).\n", + "Our cased and uncased models are training with an initial sequence length of 512\n", + "subwords for ~2-3M steps.\n" + ] + }, + { + "cell_type": "markdown", + "id": "d4ea3425", + "metadata": {}, + "source": [ + "For the XXL Italian models, we use the same training data from OPUS and extend\n", + "it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/).\n", + "Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "f1d5804d", + "metadata": {}, + "source": [ + "Note: Unfortunately, a wrong vocab size was used when training the XXL models.\n", + "This explains the mismatch of the \"real\" vocab size of 31102, compared to the\n", + "vocab size specified in `config.json`. However, the model is working and all\n", + "evaluations were done under those circumstances.\n", + "See [this issue](https://github.com/dbmdz/berts/issues/7) for more information.\n" + ] + }, + { + "cell_type": "markdown", + "id": "cc4f3d3d", + "metadata": {}, + "source": [ + "The Italian ELECTRA model was trained on the \"XXL\" corpus for 1M steps in total using a batch\n", + "size of 128. We pretty much following the ELECTRA training procedure as used for\n", + "[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra).\n" + ] + }, + { + "cell_type": "markdown", + "id": "76e431e8", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1b014af1", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ca7904c6", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-italian-uncased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "261390e6", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "f5c0c815", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-italian-uncased](https://huggingface.co/dbmdz/bert-base-italian-uncased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/info.yaml b/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/info.yaml index 5a0420fc..4ed571e8 100644 --- a/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/info.yaml @@ -1,23 +1,19 @@ +Datasets: wikipedia +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Italian +License: mit Model_Info: - name: "dbmdz/bert-base-italian-xxl-cased" - description: "🤗 + 📚 dbmdz BERT and ELECTRA models" - description_en: "🤗 + 📚 dbmdz BERT and ELECTRA models" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-italian-xxl-cased" - + description: 🤗 + 📚 dbmdz BERT and ELECTRA models + description_en: 🤗 + 📚 dbmdz BERT and ELECTRA models + from_repo: https://huggingface.co/dbmdz/bert-base-italian-xxl-cased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-italian-xxl-cased +Paper: null +Publisher: dbmdz Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "wikipedia" -Publisher: "dbmdz" -License: "mit" -Language: "Italian" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/introduction_cn.ipynb new file mode 100644 index 00000000..f96fe1ec --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/introduction_cn.ipynb @@ -0,0 +1,151 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "4e448d86", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz BERT and ELECTRA models\n" + ] + }, + { + "cell_type": "markdown", + "id": "9bcf089b", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources Italian BERT and ELECTRA models 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "fb9adbdd", + "metadata": {}, + "source": [ + "# Italian BERT\n" + ] + }, + { + "cell_type": "markdown", + "id": "e5a80c49", + "metadata": {}, + "source": [ + "The source data for the Italian BERT model consists of a recent Wikipedia dump and\n", + "various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final\n", + "training corpus has a size of 13GB and 2,050,057,573 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "3513aa96", + "metadata": {}, + "source": [ + "For sentence splitting, we use NLTK (faster compared to spacy).\n", + "Our cased and uncased models are training with an initial sequence length of 512\n", + "subwords for ~2-3M steps.\n" + ] + }, + { + "cell_type": "markdown", + "id": "ca0e58ee", + "metadata": {}, + "source": [ + "For the XXL Italian models, we use the same training data from OPUS and extend\n", + "it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/).\n", + "Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "744e0851", + "metadata": {}, + "source": [ + "Note: Unfortunately, a wrong vocab size was used when training the XXL models.\n", + "This explains the mismatch of the \"real\" vocab size of 31102, compared to the\n", + "vocab size specified in `config.json`. However, the model is working and all\n", + "evaluations were done under those circumstances.\n", + "See [this issue](https://github.com/dbmdz/berts/issues/7) for more information.\n" + ] + }, + { + "cell_type": "markdown", + "id": "3bb28396", + "metadata": {}, + "source": [ + "The Italian ELECTRA model was trained on the \"XXL\" corpus for 1M steps in total using a batch\n", + "size of 128. We pretty much following the ELECTRA training procedure as used for\n", + "[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra).\n" + ] + }, + { + "cell_type": "markdown", + "id": "4c0c2ecb", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e059cf91", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "95a883f8", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-italian-xxl-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "6b2c856e", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "ffdf7223", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-italian-xxl-cased](https://huggingface.co/dbmdz/bert-base-italian-xxl-cased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/introduction_en.ipynb new file mode 100644 index 00000000..cdd5de4e --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-italian-xxl-cased/introduction_en.ipynb @@ -0,0 +1,151 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "41ca2df0", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz BERT and ELECTRA models\n" + ] + }, + { + "cell_type": "markdown", + "id": "58e60a32", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources Italian BERT and ELECTRA models 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "c7b5f379", + "metadata": {}, + "source": [ + "# Italian BERT\n" + ] + }, + { + "cell_type": "markdown", + "id": "5bf65013", + "metadata": {}, + "source": [ + "The source data for the Italian BERT model consists of a recent Wikipedia dump and\n", + "various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final\n", + "training corpus has a size of 13GB and 2,050,057,573 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "a3aadc8d", + "metadata": {}, + "source": [ + "For sentence splitting, we use NLTK (faster compared to spacy).\n", + "Our cased and uncased models are training with an initial sequence length of 512\n", + "subwords for ~2-3M steps.\n" + ] + }, + { + "cell_type": "markdown", + "id": "4d670485", + "metadata": {}, + "source": [ + "For the XXL Italian models, we use the same training data from OPUS and extend\n", + "it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/).\n", + "Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "a366dc7d", + "metadata": {}, + "source": [ + "Note: Unfortunately, a wrong vocab size was used when training the XXL models.\n", + "This explains the mismatch of the \"real\" vocab size of 31102, compared to the\n", + "vocab size specified in `config.json`. However, the model is working and all\n", + "evaluations were done under those circumstances.\n", + "See [this issue](https://github.com/dbmdz/berts/issues/7) for more information.\n" + ] + }, + { + "cell_type": "markdown", + "id": "eaee3adf", + "metadata": {}, + "source": [ + "The Italian ELECTRA model was trained on the \"XXL\" corpus for 1M steps in total using a batch\n", + "size of 128. We pretty much following the ELECTRA training procedure as used for\n", + "[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra).\n" + ] + }, + { + "cell_type": "markdown", + "id": "c5151f8a", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f9e48e19", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c02e6f47", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-italian-xxl-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "13e03e4b", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "66705724", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-italian-xxl-cased](https://huggingface.co/dbmdz/bert-base-italian-xxl-cased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/info.yaml b/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/info.yaml index 3b0c895b..2715cc51 100644 --- a/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/info.yaml @@ -1,19 +1,15 @@ -Model_Info: - name: "dbmdz/bert-base-turkish-128k-cased" - description: "🤗 + 📚 dbmdz Turkish BERT model" - description_en: "🤗 + 📚 dbmdz Turkish BERT model" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-turkish-128k-cased" - -Task: - -Example: - -Datasets: "" -Publisher: "dbmdz" -License: "mit" -Language: "Turkish" -Paper: - +Datasets: '' +Example: null +IfOnlineDemo: 0 IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +Language: Turkish +License: mit +Model_Info: + description: 🤗 + 📚 dbmdz Turkish BERT model + description_en: 🤗 + 📚 dbmdz Turkish BERT model + from_repo: https://huggingface.co/dbmdz/bert-base-turkish-128k-cased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-turkish-128k-cased +Paper: null +Publisher: dbmdz +Task: null diff --git a/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/introduction_cn.ipynb new file mode 100644 index 00000000..a5568c66 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/introduction_cn.ipynb @@ -0,0 +1,151 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "bda1db47", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz Turkish BERT model\n" + ] + }, + { + "cell_type": "markdown", + "id": "ba254a15", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources a cased model for Turkish 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "bf2818ba", + "metadata": {}, + "source": [ + "# 🇹🇷 BERTurk\n" + ] + }, + { + "cell_type": "markdown", + "id": "788f7baa", + "metadata": {}, + "source": [ + "BERTurk is a community-driven cased BERT model for Turkish.\n" + ] + }, + { + "cell_type": "markdown", + "id": "5e051a7d", + "metadata": {}, + "source": [ + "Some datasets used for pretraining and evaluation are contributed from the\n", + "awesome Turkish NLP community, as well as the decision for the model name: BERTurk.\n" + ] + }, + { + "cell_type": "markdown", + "id": "1edbcf52", + "metadata": {}, + "source": [ + "## Stats\n" + ] + }, + { + "cell_type": "markdown", + "id": "5b7c3ff4", + "metadata": {}, + "source": [ + "The current version of the model is trained on a filtered and sentence\n", + "segmented version of the Turkish [OSCAR corpus](https://traces1.inria.fr/oscar/),\n", + "a recent Wikipedia dump, various [OPUS corpora](http://opus.nlpl.eu/) and a\n", + "special corpus provided by [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/).\n" + ] + }, + { + "cell_type": "markdown", + "id": "9413f21a", + "metadata": {}, + "source": [ + "The final training corpus has a size of 35GB and 44,04,976,662 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "25593952", + "metadata": {}, + "source": [ + "For this model we use a vocab size of 128k.\n" + ] + }, + { + "cell_type": "markdown", + "id": "962cf00d", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d4a4e8e3", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7157d7da", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-turkish-128k-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "e47155ee", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "081638b2", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-turkish-128k-cased](https://huggingface.co/dbmdz/bert-base-turkish-128k-cased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/introduction_en.ipynb new file mode 100644 index 00000000..4979c965 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-turkish-128k-cased/introduction_en.ipynb @@ -0,0 +1,151 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "911a1be9", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz Turkish BERT model\n" + ] + }, + { + "cell_type": "markdown", + "id": "4f09b0f1", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources a cased model for Turkish 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "fa2a78a0", + "metadata": {}, + "source": [ + "# 🇹🇷 BERTurk\n" + ] + }, + { + "cell_type": "markdown", + "id": "8b2f8c68", + "metadata": {}, + "source": [ + "BERTurk is a community-driven cased BERT model for Turkish.\n" + ] + }, + { + "cell_type": "markdown", + "id": "fe23e365", + "metadata": {}, + "source": [ + "Some datasets used for pretraining and evaluation are contributed from the\n", + "awesome Turkish NLP community, as well as the decision for the model name: BERTurk.\n" + ] + }, + { + "cell_type": "markdown", + "id": "2da0ce24", + "metadata": {}, + "source": [ + "## Stats\n" + ] + }, + { + "cell_type": "markdown", + "id": "d3f6af43", + "metadata": {}, + "source": [ + "The current version of the model is trained on a filtered and sentence\n", + "segmented version of the Turkish [OSCAR corpus](https://traces1.inria.fr/oscar/),\n", + "a recent Wikipedia dump, various [OPUS corpora](http://opus.nlpl.eu/) and a\n", + "special corpus provided by [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/).\n" + ] + }, + { + "cell_type": "markdown", + "id": "0d8d60c1", + "metadata": {}, + "source": [ + "The final training corpus has a size of 35GB and 44,04,976,662 tokens.\n" + ] + }, + { + "cell_type": "markdown", + "id": "ce42504f", + "metadata": {}, + "source": [ + "For this model we use a vocab size of 128k.\n" + ] + }, + { + "cell_type": "markdown", + "id": "4815bfdb", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7a084604", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8d041c78", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-turkish-128k-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "da82079c", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "b6632d46", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-turkish-128k-cased](https://huggingface.co/dbmdz/bert-base-turkish-128k-cased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-turkish-cased/info.yaml b/modelcenter/community/dbmdz/bert-base-turkish-cased/info.yaml index 394d341a..61997086 100644 --- a/modelcenter/community/dbmdz/bert-base-turkish-cased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-turkish-cased/info.yaml @@ -1,19 +1,15 @@ -Model_Info: - name: "dbmdz/bert-base-turkish-cased" - description: "🤗 + 📚 dbmdz Turkish BERT model" - description_en: "🤗 + 📚 dbmdz Turkish BERT model" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-turkish-cased" - -Task: - -Example: - -Datasets: "" -Publisher: "dbmdz" -License: "mit" -Language: "Turkish" -Paper: - +Datasets: '' +Example: null +IfOnlineDemo: 0 IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +Language: Turkish +License: mit +Model_Info: + description: 🤗 + 📚 dbmdz Turkish BERT model + description_en: 🤗 + 📚 dbmdz Turkish BERT model + from_repo: https://huggingface.co/dbmdz/bert-base-turkish-cased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-turkish-cased +Paper: null +Publisher: dbmdz +Task: null diff --git a/modelcenter/community/dbmdz/bert-base-turkish-cased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-turkish-cased/introduction_cn.ipynb new file mode 100644 index 00000000..0e32f052 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-turkish-cased/introduction_cn.ipynb @@ -0,0 +1,118 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "e9075248", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz Turkish BERT model\n" + ] + }, + { + "cell_type": "markdown", + "id": "0f68224a", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources a cased model for Turkish 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "a800751f", + "metadata": {}, + "source": [ + "# 🇹🇷 BERTurk\n" + ] + }, + { + "cell_type": "markdown", + "id": "0f418bcc", + "metadata": {}, + "source": [ + "BERTurk is a community-driven cased BERT model for Turkish.\n" + ] + }, + { + "cell_type": "markdown", + "id": "059528cb", + "metadata": {}, + "source": [ + "Some datasets used for pretraining and evaluation are contributed from the\n", + "awesome Turkish NLP community, as well as the decision for the model name: BERTurk.\n" + ] + }, + { + "cell_type": "markdown", + "id": "ec8d00db", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1cb273ef", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "45fd943c", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-turkish-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "0653e10b", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "14b8d869", + "metadata": {}, + "source": [ + "\n", + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-turkish-cased](https://huggingface.co/dbmdz/bert-base-turkish-cased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-turkish-cased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-turkish-cased/introduction_en.ipynb new file mode 100644 index 00000000..f2d90a6d --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-turkish-cased/introduction_en.ipynb @@ -0,0 +1,117 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "22b9df2e", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz Turkish BERT model\n" + ] + }, + { + "cell_type": "markdown", + "id": "509b461d", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources a cased model for Turkish 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "84ab205a", + "metadata": {}, + "source": [ + "# 🇹🇷 BERTurk\n" + ] + }, + { + "cell_type": "markdown", + "id": "aafa4b5d", + "metadata": {}, + "source": [ + "BERTurk is a community-driven cased BERT model for Turkish.\n" + ] + }, + { + "cell_type": "markdown", + "id": "16251feb", + "metadata": {}, + "source": [ + "Some datasets used for pretraining and evaluation are contributed from the\n", + "awesome Turkish NLP community, as well as the decision for the model name: BERTurk.\n" + ] + }, + { + "cell_type": "markdown", + "id": "1bdf0158", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "aa2b4d91", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0f55d31d", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-turkish-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "9aae3e54", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "839b89b9", + "metadata": {}, + "source": [ + "\n", + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-turkish-cased](https://huggingface.co/dbmdz/bert-base-turkish-cased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-turkish-uncased/info.yaml b/modelcenter/community/dbmdz/bert-base-turkish-uncased/info.yaml index f771d3f6..ffabf371 100644 --- a/modelcenter/community/dbmdz/bert-base-turkish-uncased/info.yaml +++ b/modelcenter/community/dbmdz/bert-base-turkish-uncased/info.yaml @@ -1,19 +1,15 @@ -Model_Info: - name: "dbmdz/bert-base-turkish-uncased" - description: "🤗 + 📚 dbmdz Turkish BERT model" - description_en: "🤗 + 📚 dbmdz Turkish BERT model" - icon: "" - from_repo: "https://huggingface.co/dbmdz/bert-base-turkish-uncased" - -Task: - -Example: - -Datasets: "" -Publisher: "dbmdz" -License: "mit" -Language: "Turkish" -Paper: - +Datasets: '' +Example: null +IfOnlineDemo: 0 IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +Language: Turkish +License: mit +Model_Info: + description: 🤗 + 📚 dbmdz Turkish BERT model + description_en: 🤗 + 📚 dbmdz Turkish BERT model + from_repo: https://huggingface.co/dbmdz/bert-base-turkish-uncased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dbmdz/bert-base-turkish-uncased +Paper: null +Publisher: dbmdz +Task: null diff --git a/modelcenter/community/dbmdz/bert-base-turkish-uncased/introduction_cn.ipynb b/modelcenter/community/dbmdz/bert-base-turkish-uncased/introduction_cn.ipynb new file mode 100644 index 00000000..c9332fb9 --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-turkish-uncased/introduction_cn.ipynb @@ -0,0 +1,118 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f3dbf349", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz Turkish BERT model\n" + ] + }, + { + "cell_type": "markdown", + "id": "479d8e10", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources an uncased model for Turkish 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "fc31d938", + "metadata": {}, + "source": [ + "# 🇹🇷 BERTurk\n" + ] + }, + { + "cell_type": "markdown", + "id": "c05c98f4", + "metadata": {}, + "source": [ + "BERTurk is a community-driven uncased BERT model for Turkish.\n" + ] + }, + { + "cell_type": "markdown", + "id": "116bbd89", + "metadata": {}, + "source": [ + "Some datasets used for pretraining and evaluation are contributed from the\n", + "awesome Turkish NLP community, as well as the decision for the model name: BERTurk.\n" + ] + }, + { + "cell_type": "markdown", + "id": "eab29ae1", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "531e7c2f", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c23a11a9", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-turkish-uncased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "1a4d2556", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "4e10e25f", + "metadata": {}, + "source": [ + "\n", + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dbmdz/bert-base-turkish-uncased](https://huggingface.co/dbmdz/bert-base-turkish-uncased)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dbmdz/bert-base-turkish-uncased/introduction_en.ipynb b/modelcenter/community/dbmdz/bert-base-turkish-uncased/introduction_en.ipynb new file mode 100644 index 00000000..18d5794e --- /dev/null +++ b/modelcenter/community/dbmdz/bert-base-turkish-uncased/introduction_en.ipynb @@ -0,0 +1,118 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f1968bb1", + "metadata": {}, + "source": [ + "# 🤗 + 📚 dbmdz Turkish BERT model\n" + ] + }, + { + "cell_type": "markdown", + "id": "37119e6e", + "metadata": {}, + "source": [ + "In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State\n", + "Library open sources an uncased model for Turkish 🎉\n" + ] + }, + { + "cell_type": "markdown", + "id": "e2428d3f", + "metadata": {}, + "source": [ + "# 🇹🇷 BERTurk\n" + ] + }, + { + "cell_type": "markdown", + "id": "455a98e2", + "metadata": {}, + "source": [ + "BERTurk is a community-driven uncased BERT model for Turkish.\n" + ] + }, + { + "cell_type": "markdown", + "id": "3c7b1272", + "metadata": {}, + "source": [ + "Some datasets used for pretraining and evaluation are contributed from the\n", + "awesome Turkish NLP community, as well as the decision for the model name: BERTurk.\n" + ] + }, + { + "cell_type": "markdown", + "id": "cdd8f852", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "81436ade", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bd223538", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dbmdz/bert-base-turkish-uncased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "7edb6fa7", + "metadata": {}, + "source": [ + "# Reference" + ] + }, + { + "cell_type": "markdown", + "id": "95b108cb", + "metadata": {}, + "source": [ + "\n", + "\n", + "> The model introduction and model weights originate from [https://huggingface.co/dbmdz/bert-base-turkish-uncased](https://huggingface.co/dbmdz/bert-base-turkish-uncased) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/deepparag/Aeona/info.yaml b/modelcenter/community/deepparag/Aeona/info.yaml index 65ec5dc9..e9e858f7 100644 --- a/modelcenter/community/deepparag/Aeona/info.yaml +++ b/modelcenter/community/deepparag/Aeona/info.yaml @@ -1,23 +1,19 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: '' +License: mit Model_Info: - name: "deepparag/Aeona" - description: "Aeona | Chatbot" - description_en: "Aeona | Chatbot" - icon: "" - from_repo: "https://huggingface.co/deepparag/Aeona" - + description: Aeona | Chatbot + description_en: Aeona | Chatbot + from_repo: https://huggingface.co/deepparag/Aeona + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: deepparag/Aeona +Paper: null +Publisher: deepparag Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "" -Publisher: "deepparag" -License: "mit" -Language: "" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 文本生成 + sub_tag_en: Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/deepparag/Aeona/introduction_cn.ipynb b/modelcenter/community/deepparag/Aeona/introduction_cn.ipynb new file mode 100644 index 00000000..7ccce23b --- /dev/null +++ b/modelcenter/community/deepparag/Aeona/introduction_cn.ipynb @@ -0,0 +1,116 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "9d9bb2aa", + "metadata": {}, + "source": [ + "# Aeona | Chatbot\n" + ] + }, + { + "cell_type": "markdown", + "id": "7361d804", + "metadata": {}, + "source": [ + "An generative AI made using [microsoft/DialoGPT-small](https://huggingface.co/microsoft/DialoGPT-small).\n" + ] + }, + { + "cell_type": "markdown", + "id": "008bcb8d", + "metadata": {}, + "source": [ + "Recommended to use along with an [AIML Chatbot](https://github.com/deepsarda/Aeona-Aiml) to reduce load, get better replies, add name and personality to your bot.\n", + "Using an AIML Chatbot will allow you to hardcode some replies also.\n" + ] + }, + { + "cell_type": "markdown", + "id": "b4bfb9cd", + "metadata": {}, + "source": [ + "## Evaluation\n", + "Below is a comparison of Aeona vs. other baselines on the mixed dataset given above using automatic evaluation metrics.\n" + ] + }, + { + "cell_type": "markdown", + "id": "4d478ffa", + "metadata": {}, + "source": [ + "| Model | Perplexity |\n", + "|---|---|\n", + "| Seq2seq Baseline [3] | 29.8 |\n", + "| Wolf et al. [5] | 16.3 |\n", + "| GPT-2 baseline | 99.5 |\n", + "| DialoGPT baseline | 56.6 |\n", + "| DialoGPT finetuned | 11.4 |\n", + "| PersonaGPT | 10.2 |\n", + "| **Aeona** | **7.9** |\n" + ] + }, + { + "cell_type": "markdown", + "id": "ebb927ce", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ea2a9d8e", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dc15795c", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"deepparag/Aeona\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "074bd20d", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/deepparag/Aeona](https://huggingface.co/deepparag/Aeona)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/deepparag/Aeona/introduction_en.ipynb b/modelcenter/community/deepparag/Aeona/introduction_en.ipynb new file mode 100644 index 00000000..e55b926e --- /dev/null +++ b/modelcenter/community/deepparag/Aeona/introduction_en.ipynb @@ -0,0 +1,116 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f8079990", + "metadata": {}, + "source": [ + "# Aeona | Chatbot\n" + ] + }, + { + "cell_type": "markdown", + "id": "6a69f81a", + "metadata": {}, + "source": [ + "An generative AI made using [microsoft/DialoGPT-small](https://huggingface.co/microsoft/DialoGPT-small).\n" + ] + }, + { + "cell_type": "markdown", + "id": "a65479b8", + "metadata": {}, + "source": [ + "Recommended to use along with an [AIML Chatbot](https://github.com/deepsarda/Aeona-Aiml) to reduce load, get better replies, add name and personality to your bot.\n", + "Using an AIML Chatbot will allow you to hardcode some replies also.\n" + ] + }, + { + "cell_type": "markdown", + "id": "ea390590", + "metadata": {}, + "source": [ + "## Evaluation\n", + "Below is a comparison of Aeona vs. other baselines on the mixed dataset given above using automatic evaluation metrics.\n" + ] + }, + { + "cell_type": "markdown", + "id": "5b64325a", + "metadata": {}, + "source": [ + "| Model | Perplexity |\n", + "|---|---|\n", + "| Seq2seq Baseline [3] | 29.8 |\n", + "| Wolf et al. [5] | 16.3 |\n", + "| GPT-2 baseline | 99.5 |\n", + "| DialoGPT baseline | 56.6 |\n", + "| DialoGPT finetuned | 11.4 |\n", + "| PersonaGPT | 10.2 |\n", + "| **Aeona** | **7.9** |\n" + ] + }, + { + "cell_type": "markdown", + "id": "bf7f0d0e", + "metadata": {}, + "source": [ + "## Usage\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "16b58290", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cf18c96e", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"deepparag/Aeona\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "fae16f6e", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/deepparag/Aeona](https://huggingface.co/deepparag/Aeona) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/deepparag/DumBot/download_cn.md b/modelcenter/community/deepparag/DumBot/download_cn.md deleted file mode 100644 index 4a06958c..00000000 --- a/modelcenter/community/deepparag/DumBot/download_cn.md +++ /dev/null @@ -1,23 +0,0 @@ -# 模型列表 - -## deepparag/DumBot - -| 模型名称 | 模型介绍 | 模型大小 | 模型下载 | -| --- | --- | --- | --- | -|deepparag/DumBot| | 621.95MB | [merges.txt](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/merges.txt)
[model_config.json](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/tokenizer_config.json)
[vocab.json](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/vocab.json) | - -也可以通过`paddlenlp` cli 工具来下载对应的模型权重，使用步骤如下所示： - -* 安装paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* 下载命令行 - -```shell -paddlenlp download --cache-dir ./pretrained_models deepparag/DumBot -``` - -有任何下载的问题都可以到[PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP)中发Issue提问。 \ No newline at end of file diff --git a/modelcenter/community/deepparag/DumBot/download_en.md b/modelcenter/community/deepparag/DumBot/download_en.md deleted file mode 100644 index f2bd3c74..00000000 --- a/modelcenter/community/deepparag/DumBot/download_en.md +++ /dev/null @@ -1,23 +0,0 @@ -# model list - -## - -| model | description | model_size | download | -| --- | --- | --- | --- | -|deepparag/DumBot| | 621.95MB | [merges.txt](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/merges.txt)
[model_config.json](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/tokenizer_config.json)
[vocab.json](https://bj.bcebos.com/paddlenlp/models/community/deepparag/DumBot/vocab.json) | - -or you can download all of model file with the following steps: - -* install paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* download model with cli tool - -```shell -paddlenlp download --cache-dir ./pretrained_models deepparag/DumBot -``` - -If you have any problems with it, you can post issue on [PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP) to get support. diff --git a/modelcenter/community/deepparag/DumBot/info.yaml b/modelcenter/community/deepparag/DumBot/info.yaml deleted file mode 100644 index 8f99ea06..00000000 --- a/modelcenter/community/deepparag/DumBot/info.yaml +++ /dev/null @@ -1,23 +0,0 @@ -Model_Info: - name: "deepparag/DumBot" - description: "THIS AI IS OUTDATED. See [Aeona](https://huggingface.co/deepparag/Aeona)" - description_en: "THIS AI IS OUTDATED. See [Aeona](https://huggingface.co/deepparag/Aeona)" - icon: "" - from_repo: "https://huggingface.co/deepparag/DumBot" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "" -Publisher: "deepparag" -License: "mit" -Language: "" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file diff --git a/modelcenter/community/deepset/roberta-base-squad2-distilled/info.yaml b/modelcenter/community/deepset/roberta-base-squad2-distilled/info.yaml index f351a6e3..2db8b542 100644 --- a/modelcenter/community/deepset/roberta-base-squad2-distilled/info.yaml +++ b/modelcenter/community/deepset/roberta-base-squad2-distilled/info.yaml @@ -1,23 +1,19 @@ +Datasets: squad_v2 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: mit Model_Info: - name: "deepset/roberta-base-squad2-distilled" - description: "Overview" - description_en: "Overview" - icon: "" - from_repo: "https://huggingface.co/deepset/roberta-base-squad2-distilled" - + description: Overview + description_en: Overview + from_repo: https://huggingface.co/deepset/roberta-base-squad2-distilled + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: deepset/roberta-base-squad2-distilled +Paper: null +Publisher: deepset Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Question Answering" - sub_tag: "回答问题" - -Example: - -Datasets: "squad_v2" -Publisher: "deepset" -License: "mit" -Language: "English" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 回答问题 + sub_tag_en: Question Answering + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/deepset/roberta-base-squad2-distilled/introduction_cn.ipynb b/modelcenter/community/deepset/roberta-base-squad2-distilled/introduction_cn.ipynb new file mode 100644 index 00000000..c456482d --- /dev/null +++ b/modelcenter/community/deepset/roberta-base-squad2-distilled/introduction_cn.ipynb @@ -0,0 +1,98 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "85b7cc2e", + "metadata": {}, + "source": [ + "## Overview\n", + "\n", + "Language model: deepset/roberta-base-squad2-distilled\n", + "\n", + "Language: English\n", + "\n", + "Training data: SQuAD 2.0 training set Eval data: SQuAD 2.0 dev set Infrastructure: 4x V100 GPU\n", + "\n", + "Published: Dec 8th, 2021" + ] + }, + { + "cell_type": "markdown", + "id": "a455ff64", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d51fa907", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4590c7eb", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"deepset/roberta-base-squad2-distilled\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "ac6e34fd", + "metadata": {}, + "source": [ + "## Authors\n", + "- Timo Möller: `timo.moeller [at] deepset.ai`\n", + "- Julian Risch: `julian.risch [at] deepset.ai`\n", + "- Malte Pietsch: `malte.pietsch [at] deepset.ai`\n", + "- Michel Bartels: `michel.bartels [at] deepset.ai`\n", + "## About us\n", + "\n", + "We bring NLP to the industry via open source!\n", + "Our focus: Industry specific language models & large scale QA systems.\n" + ] + }, + { + "cell_type": "markdown", + "id": "3d22bf87", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/deepset/roberta-base-squad2-distilled](https://huggingface.co/deepset/roberta-base-squad2-distilled)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/deepset/roberta-base-squad2-distilled/introduction_en.ipynb b/modelcenter/community/deepset/roberta-base-squad2-distilled/introduction_en.ipynb new file mode 100644 index 00000000..fe7b0fa8 --- /dev/null +++ b/modelcenter/community/deepset/roberta-base-squad2-distilled/introduction_en.ipynb @@ -0,0 +1,98 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "b917c220", + "metadata": {}, + "source": [ + "## Overview\n", + "\n", + "Language model: deepset/roberta-base-squad2-distilled\n", + "\n", + "Language: English\n", + "\n", + "Training data: SQuAD 2.0 training set Eval data: SQuAD 2.0 dev set Infrastructure: 4x V100 GPU\n", + "\n", + "Published: Dec 8th, 2021" + ] + }, + { + "cell_type": "markdown", + "id": "94e41c66", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8b2c9009", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b9472a8e", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"deepset/roberta-base-squad2-distilled\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "942ce61d", + "metadata": {}, + "source": [ + "## Authors\n", + "- Timo Möller: `timo.moeller [at] deepset.ai`\n", + "- Julian Risch: `julian.risch [at] deepset.ai`\n", + "- Malte Pietsch: `malte.pietsch [at] deepset.ai`\n", + "- Michel Bartels: `michel.bartels [at] deepset.ai`\n", + "## About us\n", + "\n", + "We bring NLP to the industry via open source!\n", + "Our focus: Industry specific language models & large scale QA systems.\n" + ] + }, + { + "cell_type": "markdown", + "id": "d65be46f", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/deepset/roberta-base-squad2-distilled](https://huggingface.co/deepset/roberta-base-squad2-distilled) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dslim/bert-base-NER/info.yaml b/modelcenter/community/dslim/bert-base-NER/info.yaml index 763642eb..29aea2ab 100644 --- a/modelcenter/community/dslim/bert-base-NER/info.yaml +++ b/modelcenter/community/dslim/bert-base-NER/info.yaml @@ -1,24 +1,21 @@ +Datasets: conll2003 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: mit Model_Info: - name: "dslim/bert-base-NER" - description: "bert-base-NER" - description_en: "bert-base-NER" - icon: "" - from_repo: "https://huggingface.co/dslim/bert-base-NER" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Token Classification" - sub_tag: "Token分类" - -Example: - -Datasets: "conll2003" -Publisher: "dslim" -License: "mit" -Language: "English" + description: bert-base-NER + description_en: bert-base-NER + from_repo: https://huggingface.co/dslim/bert-base-NER + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dslim/bert-base-NER Paper: - - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding' - url: 'http://arxiv.org/abs/1810.04805v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding' + url: http://arxiv.org/abs/1810.04805v2 +Publisher: dslim +Task: +- sub_tag: Token分类 + sub_tag_en: Token Classification + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dslim/bert-base-NER/introduction_cn.ipynb b/modelcenter/community/dslim/bert-base-NER/introduction_cn.ipynb new file mode 100644 index 00000000..19163312 --- /dev/null +++ b/modelcenter/community/dslim/bert-base-NER/introduction_cn.ipynb @@ -0,0 +1,125 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "4dd2d9a8", + "metadata": {}, + "source": [ + "# bert-base-NER\n" + ] + }, + { + "cell_type": "markdown", + "id": "39c0b4be", + "metadata": {}, + "source": [ + "## Model description\n" + ] + }, + { + "cell_type": "markdown", + "id": "0961b24f", + "metadata": {}, + "source": [ + "**bert-base-NER** is a fine-tuned BERT model that is ready to use for **Named Entity Recognition** and achieves **state-of-the-art performance** for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC).\n" + ] + }, + { + "cell_type": "markdown", + "id": "58641459", + "metadata": {}, + "source": [ + "Specifically, this model is a *bert-base-cased* model that was fine-tuned on the English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset.\n" + ] + }, + { + "cell_type": "markdown", + "id": "9da0ddda", + "metadata": {}, + "source": [ + "If you'd like to use a larger BERT-large model fine-tuned on the same dataset, a **bert-large-NER** version is also available.\n" + ] + }, + { + "cell_type": "markdown", + "id": "4d5adc68", + "metadata": {}, + "source": [ + "## How to use\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "88f3ea49", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "76ef1f0e", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dslim/bert-base-NER\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "137e381c", + "metadata": {}, + "source": [ + "## Citation\n", + "```\n", + "@inproceedings{tjong-kim-sang-de-meulder-2003-introduction,\n", + "title = \"Introduction to the {C}o{NLL}-2003 Shared Task: Language-Independent Named Entity Recognition\",\n", + "author = \"Tjong Kim Sang, Erik F. and\n", + "De Meulder, Fien\",\n", + "booktitle = \"Proceedings of the Seventh Conference on Natural Language Learning at {HLT}-{NAACL} 2003\",\n", + "year = \"2003\",\n", + "url = \"https://www.aclweb.org/anthology/W03-0419\",\n", + "pages = \"142--147\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3a632df2", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/dslim/bert-base-NER](https://huggingface.co/dslim/bert-base-NER)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dslim/bert-base-NER/introduction_en.ipynb b/modelcenter/community/dslim/bert-base-NER/introduction_en.ipynb new file mode 100644 index 00000000..67ed3a6f --- /dev/null +++ b/modelcenter/community/dslim/bert-base-NER/introduction_en.ipynb @@ -0,0 +1,126 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "c5180cf2", + "metadata": {}, + "source": [ + "# bert-base-NER\n" + ] + }, + { + "cell_type": "markdown", + "id": "dbf08fd8", + "metadata": {}, + "source": [ + "## Model description\n" + ] + }, + { + "cell_type": "markdown", + "id": "11690dda", + "metadata": {}, + "source": [ + "**bert-base-NER** is a fine-tuned BERT model that is ready to use for **Named Entity Recognition** and achieves **state-of-the-art performance** for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC).\n" + ] + }, + { + "cell_type": "markdown", + "id": "738f98db", + "metadata": {}, + "source": [ + "Specifically, this model is a *bert-base-cased* model that was fine-tuned on the English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset.\n" + ] + }, + { + "cell_type": "markdown", + "id": "03c5db03", + "metadata": {}, + "source": [ + "If you'd like to use a larger BERT-large model fine-tuned on the same dataset, a **bert-large-NER** version is also available.\n" + ] + }, + { + "cell_type": "markdown", + "id": "da040b29", + "metadata": {}, + "source": [ + "## How to use\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "726ee6e9", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "73564a0c", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dslim/bert-base-NER\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "c08bc233", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{tjong-kim-sang-de-meulder-2003-introduction,\n", + "title = \"Introduction to the {C}o{NLL}-2003 Shared Task: Language-Independent Named Entity Recognition\",\n", + "author = \"Tjong Kim Sang, Erik F. and\n", + "De Meulder, Fien\",\n", + "booktitle = \"Proceedings of the Seventh Conference on Natural Language Learning at {HLT}-{NAACL} 2003\",\n", + "year = \"2003\",\n", + "url = \"https://www.aclweb.org/anthology/W03-0419\",\n", + "pages = \"142--147\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "a56e1055", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dslim/bert-base-NER](https://huggingface.co/dslim/bert-base-NER) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dslim/bert-large-NER/info.yaml b/modelcenter/community/dslim/bert-large-NER/info.yaml index 48b413dd..64bdd181 100644 --- a/modelcenter/community/dslim/bert-large-NER/info.yaml +++ b/modelcenter/community/dslim/bert-large-NER/info.yaml @@ -1,24 +1,21 @@ +Datasets: conll2003 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: mit Model_Info: - name: "dslim/bert-large-NER" - description: "bert-base-NER" - description_en: "bert-base-NER" - icon: "" - from_repo: "https://huggingface.co/dslim/bert-large-NER" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Token Classification" - sub_tag: "Token分类" - -Example: - -Datasets: "conll2003" -Publisher: "dslim" -License: "mit" -Language: "English" + description: bert-base-NER + description_en: bert-base-NER + from_repo: https://huggingface.co/dslim/bert-large-NER + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dslim/bert-large-NER Paper: - - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding' - url: 'http://arxiv.org/abs/1810.04805v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding' + url: http://arxiv.org/abs/1810.04805v2 +Publisher: dslim +Task: +- sub_tag: Token分类 + sub_tag_en: Token Classification + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dslim/bert-large-NER/introduction_cn.ipynb b/modelcenter/community/dslim/bert-large-NER/introduction_cn.ipynb new file mode 100644 index 00000000..e33561ab --- /dev/null +++ b/modelcenter/community/dslim/bert-large-NER/introduction_cn.ipynb @@ -0,0 +1,126 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "70a24d70", + "metadata": {}, + "source": [ + "# bert-base-NER\n" + ] + }, + { + "cell_type": "markdown", + "id": "72df2cc8", + "metadata": {}, + "source": [ + "## Model description\n" + ] + }, + { + "cell_type": "markdown", + "id": "d1aabc01", + "metadata": {}, + "source": [ + "**bert-large-NER** is a fine-tuned BERT model that is ready to use for **Named Entity Recognition** and achieves **state-of-the-art performance** for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC).\n" + ] + }, + { + "cell_type": "markdown", + "id": "2d53a70b", + "metadata": {}, + "source": [ + "Specifically, this model is a *bert-large-cased* model that was fine-tuned on the English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset.\n" + ] + }, + { + "cell_type": "markdown", + "id": "60c2ceb7", + "metadata": {}, + "source": [ + "If you'd like to use a smaller BERT model fine-tuned on the same dataset, a **bert-base-NER** version is also available.\n" + ] + }, + { + "cell_type": "markdown", + "id": "42d984a9", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9ef70bc3", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1177b32e", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dslim/bert-large-NER\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "353c5156", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{tjong-kim-sang-de-meulder-2003-introduction,\n", + "title = \"Introduction to the {C}o{NLL}-2003 Shared Task: Language-Independent Named Entity Recognition\",\n", + "author = \"Tjong Kim Sang, Erik F. and\n", + "De Meulder, Fien\",\n", + "booktitle = \"Proceedings of the Seventh Conference on Natural Language Learning at {HLT}-{NAACL} 2003\",\n", + "year = \"2003\",\n", + "url = \"https://www.aclweb.org/anthology/W03-0419\",\n", + "pages = \"142--147\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "5705ae48", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/dslim/bert-large-NER](https://huggingface.co/dslim/bert-large-NER)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dslim/bert-large-NER/introduction_en.ipynb b/modelcenter/community/dslim/bert-large-NER/introduction_en.ipynb new file mode 100644 index 00000000..047acf0a --- /dev/null +++ b/modelcenter/community/dslim/bert-large-NER/introduction_en.ipynb @@ -0,0 +1,126 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "574c41aa", + "metadata": {}, + "source": [ + "# bert-base-NER\n" + ] + }, + { + "cell_type": "markdown", + "id": "430be48c", + "metadata": {}, + "source": [ + "## Model description\n" + ] + }, + { + "cell_type": "markdown", + "id": "4bdfd881", + "metadata": {}, + "source": [ + "**bert-large-NER** is a fine-tuned BERT model that is ready to use for **Named Entity Recognition** and achieves **state-of-the-art performance** for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC).\n" + ] + }, + { + "cell_type": "markdown", + "id": "84ed4f8f", + "metadata": {}, + "source": [ + "Specifically, this model is a *bert-large-cased* model that was fine-tuned on the English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset.\n" + ] + }, + { + "cell_type": "markdown", + "id": "b4acb8a4", + "metadata": {}, + "source": [ + "If you'd like to use a smaller BERT model fine-tuned on the same dataset, a **bert-base-NER** version is also available.\n" + ] + }, + { + "cell_type": "markdown", + "id": "46ad4f1f", + "metadata": {}, + "source": [ + "## How to use\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b7e64545", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "380aa0dc", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dslim/bert-large-NER\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "220eb907", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{tjong-kim-sang-de-meulder-2003-introduction,\n", + "title = \"Introduction to the {C}o{NLL}-2003 Shared Task: Language-Independent Named Entity Recognition\",\n", + "author = \"Tjong Kim Sang, Erik F. and\n", + "De Meulder, Fien\",\n", + "booktitle = \"Proceedings of the Seventh Conference on Natural Language Learning at {HLT}-{NAACL} 2003\",\n", + "year = \"2003\",\n", + "url = \"https://www.aclweb.org/anthology/W03-0419\",\n", + "pages = \"142--147\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "dc0b9503", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/dslim/bert-large-NER](https://huggingface.co/dslim/bert-large-NER) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/info.yaml b/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/info.yaml index 04a33129..74203f93 100644 --- a/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/info.yaml +++ b/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/info.yaml @@ -1,23 +1,19 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Romanian +License: mit Model_Info: - name: "dumitrescustefan/bert-base-romanian-cased-v1" - description: "bert-base-romanian-cased-v1" - description_en: "bert-base-romanian-cased-v1" - icon: "" - from_repo: "https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1" - + description: bert-base-romanian-cased-v1 + description_en: bert-base-romanian-cased-v1 + from_repo: https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1 + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dumitrescustefan/bert-base-romanian-cased-v1 +Paper: null +Publisher: dumitrescustefan Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "dumitrescustefan" -License: "mit" -Language: "Romanian" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/introduction_cn.ipynb b/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/introduction_cn.ipynb new file mode 100644 index 00000000..cc2fc93e --- /dev/null +++ b/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/introduction_cn.ipynb @@ -0,0 +1,116 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "2a485f7a", + "metadata": {}, + "source": [ + "# bert-base-romanian-cased-v1\n" + ] + }, + { + "cell_type": "markdown", + "id": "5f911938", + "metadata": {}, + "source": [ + "The BERT **base**, **cased** model for Romanian, trained on a 15GB corpus.\n" + ] + }, + { + "cell_type": "markdown", + "id": "e2cccf2e", + "metadata": {}, + "source": [ + "### How to use\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "64b86b26", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f84498aa", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dumitrescustefan/bert-base-romanian-cased-v1\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "40176abc", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{dumitrescu-etal-2020-birth,\n", + "title = \"The birth of {R}omanian {BERT}\",\n", + "author = \"Dumitrescu, Stefan and\n", + "Avram, Andrei-Marius and\n", + "Pyysalo, Sampo\",\n", + "booktitle = \"Findings of the Association for Computational Linguistics: EMNLP 2020\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://aclanthology.org/2020.findings-emnlp.387\",\n", + "doi = \"10.18653/v1/2020.findings-emnlp.387\",\n", + "pages = \"4324--4328\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "550b09f3", + "metadata": {}, + "source": [ + "#### Acknowledgements\n" + ] + }, + { + "cell_type": "markdown", + "id": "fda88b7c", + "metadata": {}, + "source": [ + "- We'd like to thank [Sampo Pyysalo](https://github.com/spyysalo) from TurkuNLP for helping us out with the compute needed to pretrain the v1.0 BERT models. He's awesome!\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1](https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/introduction_en.ipynb b/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/introduction_en.ipynb new file mode 100644 index 00000000..be66e19c --- /dev/null +++ b/modelcenter/community/dumitrescustefan/bert-base-romanian-cased-v1/introduction_en.ipynb @@ -0,0 +1,116 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "28a330b8", + "metadata": {}, + "source": [ + "# bert-base-romanian-cased-v1\n" + ] + }, + { + "cell_type": "markdown", + "id": "36f0d74f", + "metadata": {}, + "source": [ + "The BERT **base**, **cased** model for Romanian, trained on a 15GB corpus." + ] + }, + { + "cell_type": "markdown", + "id": "0104e14e", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b4ca4271", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9f3ca553", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dumitrescustefan/bert-base-romanian-cased-v1\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "51754d3f", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{dumitrescu-etal-2020-birth,\n", + "title = \"The birth of {R}omanian {BERT}\",\n", + "author = \"Dumitrescu, Stefan and\n", + "Avram, Andrei-Marius and\n", + "Pyysalo, Sampo\",\n", + "booktitle = \"Findings of the Association for Computational Linguistics: EMNLP 2020\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://aclanthology.org/2020.findings-emnlp.387\",\n", + "doi = \"10.18653/v1/2020.findings-emnlp.387\",\n", + "pages = \"4324--4328\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "2143146f", + "metadata": {}, + "source": [ + "#### Acknowledgements\n" + ] + }, + { + "cell_type": "markdown", + "id": "d983ac22", + "metadata": {}, + "source": [ + "- We'd like to thank [Sampo Pyysalo](https://github.com/spyysalo) from TurkuNLP for helping us out with the compute needed to pretrain the v1.0 BERT models. He's awesome!\n", + "> The model introduction and model weights originate from [https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1](https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/info.yaml b/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/info.yaml index f307e47f..05f2d42a 100644 --- a/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/info.yaml +++ b/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/info.yaml @@ -1,23 +1,19 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Romanian +License: mit Model_Info: - name: "dumitrescustefan/bert-base-romanian-uncased-v1" - description: "bert-base-romanian-uncased-v1" - description_en: "bert-base-romanian-uncased-v1" - icon: "" - from_repo: "https://huggingface.co/dumitrescustefan/bert-base-romanian-uncased-v1" - + description: bert-base-romanian-uncased-v1 + description_en: bert-base-romanian-uncased-v1 + from_repo: https://huggingface.co/dumitrescustefan/bert-base-romanian-uncased-v1 + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: dumitrescustefan/bert-base-romanian-uncased-v1 +Paper: null +Publisher: dumitrescustefan Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "dumitrescustefan" -License: "mit" -Language: "Romanian" -Paper: - -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/introduction_cn.ipynb b/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/introduction_cn.ipynb new file mode 100644 index 00000000..d4de6088 --- /dev/null +++ b/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/introduction_cn.ipynb @@ -0,0 +1,116 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "922f44e2", + "metadata": {}, + "source": [ + "# bert-base-romanian-uncased-v1\n" + ] + }, + { + "cell_type": "markdown", + "id": "2f5259bd", + "metadata": {}, + "source": [ + "The BERT **base**, **uncased** model for Romanian, trained on a 15GB corpus, version \n" + ] + }, + { + "cell_type": "markdown", + "id": "408f4468", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "acd14372", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cc5d539c", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dumitrescustefan/bert-base-romanian-uncased-v1\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "adbbab44", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{dumitrescu-etal-2020-birth,\n", + "title = \"The birth of {R}omanian {BERT}\",\n", + "author = \"Dumitrescu, Stefan and\n", + "Avram, Andrei-Marius and\n", + "Pyysalo, Sampo\",\n", + "booktitle = \"Findings of the Association for Computational Linguistics: EMNLP 2020\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://aclanthology.org/2020.findings-emnlp.387\",\n", + "doi = \"10.18653/v1/2020.findings-emnlp.387\",\n", + "pages = \"4324--4328\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "4276651e", + "metadata": {}, + "source": [ + "#### Acknowledgements\n" + ] + }, + { + "cell_type": "markdown", + "id": "84a91796", + "metadata": {}, + "source": [ + "- We'd like to thank [Sampo Pyysalo](https://github.com/spyysalo) from TurkuNLP for helping us out with the compute needed to pretrain the v1.0 BERT models. He's awesome!\n", + "> 此模型介绍及权重来源于[https://huggingface.co/dumitrescustefan/bert-base-romanian-uncased-v1](https://huggingface.co/dumitrescustefan/bert-base-romanian-uncased-v1)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/introduction_en.ipynb b/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/introduction_en.ipynb new file mode 100644 index 00000000..ca76cb10 --- /dev/null +++ b/modelcenter/community/dumitrescustefan/bert-base-romanian-uncased-v1/introduction_en.ipynb @@ -0,0 +1,116 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "cf268d0f", + "metadata": {}, + "source": [ + "# bert-base-romanian-uncased-v1\n" + ] + }, + { + "cell_type": "markdown", + "id": "453405af", + "metadata": {}, + "source": [ + "The BERT **base**, **uncased** model for Romanian, trained on a 15GB corpus, version \n" + ] + }, + { + "cell_type": "markdown", + "id": "4100824e", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cd182732", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1c16cd09", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"dumitrescustefan/bert-base-romanian-uncased-v1\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "ba32e8ff", + "metadata": {}, + "source": [ + "## Citation\n", + "\n", + "```\n", + "@inproceedings{dumitrescu-etal-2020-birth,\n", + "title = \"The birth of {R}omanian {BERT}\",\n", + "author = \"Dumitrescu, Stefan and\n", + "Avram, Andrei-Marius and\n", + "Pyysalo, Sampo\",\n", + "booktitle = \"Findings of the Association for Computational Linguistics: EMNLP 2020\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://aclanthology.org/2020.findings-emnlp.387\",\n", + "doi = \"10.18653/v1/2020.findings-emnlp.387\",\n", + "pages = \"4324--4328\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "faa200c7", + "metadata": {}, + "source": [ + "#### Acknowledgements\n" + ] + }, + { + "cell_type": "markdown", + "id": "cb74a943", + "metadata": {}, + "source": [ + "- We'd like to thank [Sampo Pyysalo](https://github.com/spyysalo) from TurkuNLP for helping us out with the compute needed to pretrain the v1.0 BERT models. He's awesome!\n", + "> The model introduction and model weights originate from [https://huggingface.co/dumitrescustefan/bert-base-romanian-uncased-v1](https://huggingface.co/dumitrescustefan/bert-base-romanian-uncased-v1) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/info.yaml b/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/info.yaml index ea7b3848..a0fd84b9 100644 --- a/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/info.yaml +++ b/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/info.yaml @@ -1,26 +1,24 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: mit Model_Info: - name: "emilyalsentzer/Bio_ClinicalBERT" - description: "ClinicalBERT - Bio + Clinical BERT Model" - description_en: "ClinicalBERT - Bio + Clinical BERT Model" - icon: "" - from_repo: "https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "emilyalsentzer" -License: "mit" -Language: "English" + description: ClinicalBERT - Bio + Clinical BERT Model + description_en: ClinicalBERT - Bio + Clinical BERT Model + from_repo: https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: emilyalsentzer/Bio_ClinicalBERT Paper: - - title: 'Publicly Available Clinical BERT Embeddings' - url: 'http://arxiv.org/abs/1904.03323v3' - - title: 'BioBERT: a pre-trained biomedical language representation model for biomedical text mining' - url: 'http://arxiv.org/abs/1901.08746v4' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Publicly Available Clinical BERT Embeddings + url: http://arxiv.org/abs/1904.03323v3 +- title: 'BioBERT: a pre-trained biomedical language representation model for biomedical + text mining' + url: http://arxiv.org/abs/1901.08746v4 +Publisher: emilyalsentzer +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/introduction_cn.ipynb b/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/introduction_cn.ipynb new file mode 100644 index 00000000..7064ebfb --- /dev/null +++ b/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/introduction_cn.ipynb @@ -0,0 +1,90 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "22b0e4db", + "metadata": {}, + "source": [ + "# ClinicalBERT - Bio + Clinical BERT Model\n" + ] + }, + { + "cell_type": "markdown", + "id": "f9d9ac37", + "metadata": {}, + "source": [ + "The [Publicly Available Clinical BERT Embeddings](https://arxiv.org/abs/1904.03323) paper contains four unique clinicalBERT models: initialized with BERT-Base (`cased_L-12_H-768_A-12`) or BioBERT (`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`) & trained on either all MIMIC notes or only discharge summaries.\n" + ] + }, + { + "cell_type": "markdown", + "id": "24aaa0b1", + "metadata": {}, + "source": [ + "This model card describes the Bio+Clinical BERT model, which was initialized from [BioBERT](https://arxiv.org/abs/1901.08746) & trained on all MIMIC notes.\n" + ] + }, + { + "cell_type": "markdown", + "id": "1449fef2", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "be5241ea", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f4c3cf6f", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"emilyalsentzer/Bio_ClinicalBERT\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "451e4ff6", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT](https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/introduction_en.ipynb b/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/introduction_en.ipynb new file mode 100644 index 00000000..fab51acf --- /dev/null +++ b/modelcenter/community/emilyalsentzer/Bio_ClinicalBERT/introduction_en.ipynb @@ -0,0 +1,90 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "91c5f94f", + "metadata": {}, + "source": [ + "# ClinicalBERT - Bio + Clinical BERT Model\n" + ] + }, + { + "cell_type": "markdown", + "id": "ec471b16", + "metadata": {}, + "source": [ + "The [Publicly Available Clinical BERT Embeddings](https://arxiv.org/abs/1904.03323) paper contains four unique clinicalBERT models: initialized with BERT-Base (`cased_L-12_H-768_A-12`) or BioBERT (`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`) & trained on either all MIMIC notes or only discharge summaries.\n" + ] + }, + { + "cell_type": "markdown", + "id": "9ab166b8", + "metadata": {}, + "source": [ + "This model card describes the Bio+Clinical BERT model, which was initialized from [BioBERT](https://arxiv.org/abs/1901.08746) & trained on all MIMIC notes.\n" + ] + }, + { + "cell_type": "markdown", + "id": "69f6ed08", + "metadata": {}, + "source": [ + "## How to use the model\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "62913fa8", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7b055241", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"emilyalsentzer/Bio_ClinicalBERT\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "0716a06f", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT](https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/info.yaml b/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/info.yaml index 3f91657a..34d508f8 100644 --- a/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/info.yaml +++ b/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/info.yaml @@ -1,26 +1,24 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: mit Model_Info: - name: "emilyalsentzer/Bio_Discharge_Summary_BERT" - description: "ClinicalBERT - Bio + Discharge Summary BERT Model" - description_en: "ClinicalBERT - Bio + Discharge Summary BERT Model" - icon: "" - from_repo: "https://huggingface.co/emilyalsentzer/Bio_Discharge_Summary_BERT" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "emilyalsentzer" -License: "mit" -Language: "English" + description: ClinicalBERT - Bio + Discharge Summary BERT Model + description_en: ClinicalBERT - Bio + Discharge Summary BERT Model + from_repo: https://huggingface.co/emilyalsentzer/Bio_Discharge_Summary_BERT + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: emilyalsentzer/Bio_Discharge_Summary_BERT Paper: - - title: 'Publicly Available Clinical BERT Embeddings' - url: 'http://arxiv.org/abs/1904.03323v3' - - title: 'BioBERT: a pre-trained biomedical language representation model for biomedical text mining' - url: 'http://arxiv.org/abs/1901.08746v4' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Publicly Available Clinical BERT Embeddings + url: http://arxiv.org/abs/1904.03323v3 +- title: 'BioBERT: a pre-trained biomedical language representation model for biomedical + text mining' + url: http://arxiv.org/abs/1901.08746v4 +Publisher: emilyalsentzer +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/introduction_cn.ipynb b/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/introduction_cn.ipynb new file mode 100644 index 00000000..3c3fd2a4 --- /dev/null +++ b/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/introduction_cn.ipynb @@ -0,0 +1,90 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "67503ba7", + "metadata": {}, + "source": [ + "# ClinicalBERT - Bio + Discharge Summary BERT Model\n" + ] + }, + { + "cell_type": "markdown", + "id": "e2d38260", + "metadata": {}, + "source": [ + "The [Publicly Available Clinical BERT Embeddings](https://arxiv.org/abs/1904.03323) paper contains four unique clinicalBERT models: initialized with BERT-Base (`cased_L-12_H-768_A-12`) or BioBERT (`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`) & trained on either all MIMIC notes or only discharge summaries.\n" + ] + }, + { + "cell_type": "markdown", + "id": "1c92755e", + "metadata": {}, + "source": [ + "This model card describes the Bio+Discharge Summary BERT model, which was initialized from [BioBERT](https://arxiv.org/abs/1901.08746) & trained on only discharge summaries from MIMIC.\n" + ] + }, + { + "cell_type": "markdown", + "id": "068ba168", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "24e8b203", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4bcd1b84", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"emilyalsentzer/Bio_Discharge_Summary_BERT\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "0cebe09b", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/emilyalsentzer/Bio_Discharge_Summary_BERT](https://huggingface.co/emilyalsentzer/Bio_Discharge_Summary_BERT)，并转换为飞桨模型格式。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/introduction_en.ipynb b/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/introduction_en.ipynb new file mode 100644 index 00000000..f879e822 --- /dev/null +++ b/modelcenter/community/emilyalsentzer/Bio_Discharge_Summary_BERT/introduction_en.ipynb @@ -0,0 +1,90 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a786c8f0", + "metadata": {}, + "source": [ + "# ClinicalBERT - Bio + Discharge Summary BERT Model\n" + ] + }, + { + "cell_type": "markdown", + "id": "4d8e4f1f", + "metadata": {}, + "source": [ + "The [Publicly Available Clinical BERT Embeddings](https://arxiv.org/abs/1904.03323) paper contains four unique clinicalBERT models: initialized with BERT-Base (`cased_L-12_H-768_A-12`) or BioBERT (`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`) & trained on either all MIMIC notes or only discharge summaries.\n" + ] + }, + { + "cell_type": "markdown", + "id": "83bf8287", + "metadata": {}, + "source": [ + "This model card describes the Bio+Discharge Summary BERT model, which was initialized from [BioBERT](https://arxiv.org/abs/1901.08746) & trained on only discharge summaries from MIMIC.\n" + ] + }, + { + "cell_type": "markdown", + "id": "ee7d03ef", + "metadata": {}, + "source": [ + "## How to use the model\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3ef75bb2", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c04f99b3", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"emilyalsentzer/Bio_Discharge_Summary_BERT\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "e4459a1c", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/emilyalsentzer/Bio_Discharge_Summary_BERT](https://huggingface.co/emilyalsentzer/Bio_Discharge_Summary_BERT) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-base-lm-adapt/info.yaml b/modelcenter/community/google/t5-base-lm-adapt/info.yaml index 90a398a8..7afa8d89 100644 --- a/modelcenter/community/google/t5-base-lm-adapt/info.yaml +++ b/modelcenter/community/google/t5-base-lm-adapt/info.yaml @@ -1,26 +1,23 @@ +Datasets: c4 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "google/t5-base-lm-adapt" - description: "Version 1.1 - LM-Adapted" - description_en: "Version 1.1 - LM-Adapted" - icon: "" - from_repo: "https://huggingface.co/google/t5-base-lm-adapt" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4" -Publisher: "google" -License: "apache-2.0" -Language: "English" + description: Version 1.1 - LM-Adapted + description_en: Version 1.1 - LM-Adapted + from_repo: https://huggingface.co/google/t5-base-lm-adapt + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: google/t5-base-lm-adapt Paper: - - title: 'GLU Variants Improve Transformer' - url: 'http://arxiv.org/abs/2002.05202v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: GLU Variants Improve Transformer + url: http://arxiv.org/abs/2002.05202v1 +- title: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer + url: http://arxiv.org/abs/1910.10683v3 +Publisher: google +Task: +- sub_tag: 文本生成 + sub_tag_en: Text2Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/google/t5-base-lm-adapt/introduction_cn.ipynb b/modelcenter/community/google/t5-base-lm-adapt/introduction_cn.ipynb new file mode 100644 index 00000000..8e4a107f --- /dev/null +++ b/modelcenter/community/google/t5-base-lm-adapt/introduction_cn.ipynb @@ -0,0 +1,102 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "c59ed826", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1 - LM-Adapted\n", + "\n", + "\n", + "## Version 1.1 - LM-Adapted\n", + "\n", + "[T5 Version 1.1 - LM Adapted](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k) includes the following improvements compared to the original T5 model:\n", + "\n", + "- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "and is pretrained on both the denoising and language modeling objective.\n", + "\n", + "More specifically, this checkpoint is initialized from T5 Version 1.1 - Base\n", + "and then trained for an additional 100K steps on the LM objective discussed in the [T5 paper](https://arxiv.org/pdf/1910.10683.pdf).\n", + "This adaptation improves the ability of the model to be used for prompt tuning.\n", + "\n", + "**Note**: A popular fine-tuned version of the *T5 Version 1.1 - LM Adapted* model is BigScience's T0pp.\n", + "\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6db9a194", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b70ecb24", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-base-lm-adapt\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "46aee335", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/google/t5-base-lm-adapt](https://huggingface.co/google/t5-base-lm-adapt)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-base-lm-adapt/introduction_en.ipynb b/modelcenter/community/google/t5-base-lm-adapt/introduction_en.ipynb new file mode 100644 index 00000000..c1a17778 --- /dev/null +++ b/modelcenter/community/google/t5-base-lm-adapt/introduction_en.ipynb @@ -0,0 +1,103 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "35f226d7", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1 - LM-Adapted\n", + "\n", + "\n", + "## Version 1.1 - LM-Adapted\n", + "\n", + "[T5 Version 1.1 - LM Adapted](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k) includes the following improvements compared to the original T5 model:\n", + "\n", + "- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "and is pretrained on both the denoising and language modeling objective.\n", + "\n", + "More specifically, this checkpoint is initialized from T5 Version 1.1 - Base\n", + "and then trained for an additional 100K steps on the LM objective discussed in the [T5 paper](https://arxiv.org/pdf/1910.10683.pdf).\n", + "This adaptation improves the ability of the model to be used for prompt tuning.\n", + "\n", + "**Note**: A popular fine-tuned version of the *T5 Version 1.1 - LM Adapted* model is BigScience's T0pp.\n", + "\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b471855d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f74ec3ef", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-base-lm-adapt\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "e431d080", + "metadata": {}, + "source": [ + "\n", + "> The model introduction and model weights originate from [https://huggingface.co/google/t5-base-lm-adapt](https://huggingface.co/google/t5-base-lm-adapt) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-large-lm-adapt/info.yaml b/modelcenter/community/google/t5-large-lm-adapt/info.yaml index db5c0320..417bbe71 100644 --- a/modelcenter/community/google/t5-large-lm-adapt/info.yaml +++ b/modelcenter/community/google/t5-large-lm-adapt/info.yaml @@ -1,26 +1,23 @@ +Datasets: c4 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "google/t5-large-lm-adapt" - description: "Version 1.1 - LM-Adapted" - description_en: "Version 1.1 - LM-Adapted" - icon: "" - from_repo: "https://huggingface.co/google/t5-large-lm-adapt" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4" -Publisher: "google" -License: "apache-2.0" -Language: "English" + description: Version 1.1 - LM-Adapted + description_en: Version 1.1 - LM-Adapted + from_repo: https://huggingface.co/google/t5-large-lm-adapt + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: google/t5-large-lm-adapt Paper: - - title: 'GLU Variants Improve Transformer' - url: 'http://arxiv.org/abs/2002.05202v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: GLU Variants Improve Transformer + url: http://arxiv.org/abs/2002.05202v1 +- title: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer + url: http://arxiv.org/abs/1910.10683v3 +Publisher: google +Task: +- sub_tag: 文本生成 + sub_tag_en: Text2Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/google/t5-large-lm-adapt/introduction_cn.ipynb b/modelcenter/community/google/t5-large-lm-adapt/introduction_cn.ipynb new file mode 100644 index 00000000..ce6a76d6 --- /dev/null +++ b/modelcenter/community/google/t5-large-lm-adapt/introduction_cn.ipynb @@ -0,0 +1,111 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "71352026", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1 - LM-Adapted\n", + "\n", + "\n", + "## Version 1.1 - LM-Adapted\n", + "\n", + "[T5 Version 1.1 - LM Adapted](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k) includes the following improvements compared to the original [T5 model](https://huggingface.co/t5-large):\n", + "\n", + "- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "and is pretrained on both the denoising and language modeling objective.\n", + "\n", + "More specifically, this checkpoint is initialized from T5 Version 1.1 - Large\n", + "and then trained for an additional 100K steps on the LM objective discussed in the [T5 paper](https://arxiv.org/pdf/1910.10683.pdf).\n", + "This adaptation improves the ability of the model to be used for prompt tuning.\n", + "\n", + "**Note**: A popular fine-tuned version of the *T5 Version 1.1 - LM Adapted* model is BigScience's T0pp.\n", + "\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "d41870db", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b9451fd3", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6392031c", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-large-lm-adapt\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "3b35551f", + "metadata": {}, + "source": [ + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/google/t5-large-lm-adapt](https://huggingface.co/google/t5-large-lm-adapt)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-large-lm-adapt/introduction_en.ipynb b/modelcenter/community/google/t5-large-lm-adapt/introduction_en.ipynb new file mode 100644 index 00000000..fdc71cd1 --- /dev/null +++ b/modelcenter/community/google/t5-large-lm-adapt/introduction_en.ipynb @@ -0,0 +1,111 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "70cb903f", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1 - LM-Adapted\n", + "\n", + "\n", + "## Version 1.1 - LM-Adapted\n", + "\n", + "[T5 Version 1.1 - LM Adapted](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k) includes the following improvements compared to the original [T5 model](https://huggingface.co/t5-large):\n", + "\n", + "- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "and is pretrained on both the denoising and language modeling objective.\n", + "\n", + "More specifically, this checkpoint is initialized from T5 Version 1.1 - Large\n", + "and then trained for an additional 100K steps on the LM objective discussed in the [T5 paper](https://arxiv.org/pdf/1910.10683.pdf).\n", + "This adaptation improves the ability of the model to be used for prompt tuning.\n", + "\n", + "**Note**: A popular fine-tuned version of the *T5 Version 1.1 - LM Adapted* model is BigScience's T0pp.\n", + "\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "cab0c1ea", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e37976b5", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0673661a", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-large-lm-adapt\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "7b24e77f", + "metadata": {}, + "source": [ + "\n", + "> The model introduction and model weights originate from [https://huggingface.co/google/t5-large-lm-adapt](https://huggingface.co/google/t5-large-lm-adapt) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-large-ssm/download_cn.md b/modelcenter/community/google/t5-large-ssm/download_cn.md deleted file mode 100644 index 344edafe..00000000 --- a/modelcenter/community/google/t5-large-ssm/download_cn.md +++ /dev/null @@ -1,23 +0,0 @@ -# 模型列表 - -## google/t5-large-ssm - -| 模型名称 | 模型介绍 | 模型大小 | 模型下载 | -| --- | --- | --- | --- | -|google/t5-large-ssm| | 3.12G | [model_config.json](https://bj.bcebos.com/paddlenlp/models/community/google/t5-large-ssm/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/google/t5-large-ssm/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/google/t5-large-ssm/tokenizer_config.json) | - -也可以通过`paddlenlp` cli 工具来下载对应的模型权重，使用步骤如下所示： - -* 安装paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* 下载命令行 - -```shell -paddlenlp download --cache-dir ./pretrained_models google/t5-large-ssm -``` - -有任何下载的问题都可以到[PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP)中发Issue提问。 \ No newline at end of file diff --git a/modelcenter/community/google/t5-large-ssm/download_en.md b/modelcenter/community/google/t5-large-ssm/download_en.md deleted file mode 100644 index 000055d3..00000000 --- a/modelcenter/community/google/t5-large-ssm/download_en.md +++ /dev/null @@ -1,23 +0,0 @@ -# model list - -## - -| model | description | model_size | download | -| --- | --- | --- | --- | -|google/t5-large-ssm| | 3.12G | [model_config.json](https://bj.bcebos.com/paddlenlp/models/community/google/t5-large-ssm/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/google/t5-large-ssm/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/google/t5-large-ssm/tokenizer_config.json) | - -or you can download all of model file with the following steps: - -* install paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* download model with cli tool - -```shell -paddlenlp download --cache-dir ./pretrained_models google/t5-large-ssm -``` - -If you have any problems with it, you can post issue on [PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP) to get support. diff --git a/modelcenter/community/google/t5-large-ssm/info.yaml b/modelcenter/community/google/t5-large-ssm/info.yaml deleted file mode 100644 index 5a7cafb1..00000000 --- a/modelcenter/community/google/t5-large-ssm/info.yaml +++ /dev/null @@ -1,26 +0,0 @@ -Model_Info: - name: "google/t5-large-ssm" - description: "Abstract" - description_en: "Abstract" - icon: "" - from_repo: "https://huggingface.co/google/t5-large-ssm" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4,wikipedia" -Publisher: "google" -License: "apache-2.0" -Language: "English" -Paper: - - title: 'REALM: Retrieval-Augmented Language Model Pre-Training' - url: 'http://arxiv.org/abs/2002.08909v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file diff --git a/modelcenter/community/google/t5-small-lm-adapt/info.yaml b/modelcenter/community/google/t5-small-lm-adapt/info.yaml index db6f3a7e..9e9b014a 100644 --- a/modelcenter/community/google/t5-small-lm-adapt/info.yaml +++ b/modelcenter/community/google/t5-small-lm-adapt/info.yaml @@ -1,26 +1,23 @@ +Datasets: c4 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "google/t5-small-lm-adapt" - description: "Version 1.1 - LM-Adapted" - description_en: "Version 1.1 - LM-Adapted" - icon: "" - from_repo: "https://huggingface.co/google/t5-small-lm-adapt" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4" -Publisher: "google" -License: "apache-2.0" -Language: "English" + description: Version 1.1 - LM-Adapted + description_en: Version 1.1 - LM-Adapted + from_repo: https://huggingface.co/google/t5-small-lm-adapt + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: google/t5-small-lm-adapt Paper: - - title: 'GLU Variants Improve Transformer' - url: 'http://arxiv.org/abs/2002.05202v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: GLU Variants Improve Transformer + url: http://arxiv.org/abs/2002.05202v1 +- title: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer + url: http://arxiv.org/abs/1910.10683v3 +Publisher: google +Task: +- sub_tag: 文本生成 + sub_tag_en: Text2Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/google/t5-small-lm-adapt/introduction_cn.ipynb b/modelcenter/community/google/t5-small-lm-adapt/introduction_cn.ipynb new file mode 100644 index 00000000..641edd7a --- /dev/null +++ b/modelcenter/community/google/t5-small-lm-adapt/introduction_cn.ipynb @@ -0,0 +1,111 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "9dca1445", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1 - LM-Adapted\n", + "\n", + "\n", + "## Version 1.1 - LM-Adapted\n", + "\n", + "[T5 Version 1.1 - LM Adapted](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k) includes the following improvements compared to the original [T5 model](https://huggingface.co/t5-small):\n", + "\n", + "- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "and is pretrained on both the denoising and language modeling objective.\n", + "\n", + "More specifically, this checkpoint is initialized from T5 Version 1.1 - Small\n", + "and then trained for an additional 100K steps on the LM objective discussed in the [T5 paper](https://arxiv.org/pdf/1910.10683.pdf).\n", + "This adaptation improves the ability of the model to be used for prompt tuning.\n", + "\n", + "**Note**: A popular fine-tuned version of the *T5 Version 1.1 - LM Adapted* model is BigScience's T0pp.\n", + "\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "4c63de98", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8030fcb4", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8f6f14dd", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-small-lm-adapt\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "b8dd698b", + "metadata": {}, + "source": [ + "\n", + "> 此模型介绍及权重来源于[https://huggingface.co/google/t5-small-lm-adapt](https://huggingface.co/google/t5-small-lm-adapt)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-small-lm-adapt/introduction_en.ipynb b/modelcenter/community/google/t5-small-lm-adapt/introduction_en.ipynb new file mode 100644 index 00000000..21c1e824 --- /dev/null +++ b/modelcenter/community/google/t5-small-lm-adapt/introduction_en.ipynb @@ -0,0 +1,111 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "42de6200", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1 - LM-Adapted\n", + "\n", + "\n", + "## Version 1.1 - LM-Adapted\n", + "\n", + "[T5 Version 1.1 - LM Adapted](https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#lm-adapted-t511lm100k) includes the following improvements compared to the original [T5 model](https://huggingface.co/t5-small):\n", + "\n", + "- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "and is pretrained on both the denoising and language modeling objective.\n", + "\n", + "More specifically, this checkpoint is initialized from [T5 Version 1.1 - Small](https://huggingface.co/google/https://huggingface.co/google/t5-v1_1-small)\n", + "and then trained for an additional 100K steps on the LM objective discussed in the [T5 paper](https://arxiv.org/pdf/1910.10683.pdf).\n", + "This adaptation improves the ability of the model to be used for prompt tuning.\n", + "\n", + "**Note**: A popular fine-tuned version of the *T5 Version 1.1 - LM Adapted* model is BigScience's T0pp.\n", + "\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "39071317", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "31a774de", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a1cd7a50", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-small-lm-adapt\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "4d283d3d", + "metadata": {}, + "source": [ + "\n", + "> The model introduction and model weights originate from [https://huggingface.co/google/t5-small-lm-adapt](https://huggingface.co/google/t5-small-lm-adapt) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-v1_1-base/info.yaml b/modelcenter/community/google/t5-v1_1-base/info.yaml index 3cedf8d0..971ea7d8 100644 --- a/modelcenter/community/google/t5-v1_1-base/info.yaml +++ b/modelcenter/community/google/t5-v1_1-base/info.yaml @@ -1,26 +1,23 @@ +Datasets: c4 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "google/t5-v1_1-base" - description: "Version 1.1" - description_en: "Version 1.1" - icon: "" - from_repo: "https://huggingface.co/google/t5-v1_1-base" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4" -Publisher: "google" -License: "apache-2.0" -Language: "English" + description: Version 1.1 + description_en: Version 1.1 + from_repo: https://huggingface.co/google/t5-v1_1-base + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: google/t5-v1_1-base Paper: - - title: 'GLU Variants Improve Transformer' - url: 'http://arxiv.org/abs/2002.05202v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: GLU Variants Improve Transformer + url: http://arxiv.org/abs/2002.05202v1 +- title: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer + url: http://arxiv.org/abs/1910.10683v3 +Publisher: google +Task: +- sub_tag: 文本生成 + sub_tag_en: Text2Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/google/t5-v1_1-base/introduction_cn.ipynb b/modelcenter/community/google/t5-v1_1-base/introduction_cn.ipynb new file mode 100644 index 00000000..fb4d969a --- /dev/null +++ b/modelcenter/community/google/t5-v1_1-base/introduction_cn.ipynb @@ -0,0 +1,101 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "83a0cdfd", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1\n", + "\n", + "\n", + "## Version 1.1\n", + "\n", + "[T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "**Note**: T5 Version 1.1 was only pre-trained on C4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task.\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "6196f74c", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6c13cb82", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1d62f626", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-v1_1-base\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "016545f2", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/google/t5-v1_1-base](https://huggingface.co/google/t5-v1_1-base)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-v1_1-base/introduction_en.ipynb b/modelcenter/community/google/t5-v1_1-base/introduction_en.ipynb new file mode 100644 index 00000000..4084d70f --- /dev/null +++ b/modelcenter/community/google/t5-v1_1-base/introduction_en.ipynb @@ -0,0 +1,101 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "2656a571", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1\n", + "\n", + "\n", + "## Version 1.1\n", + "\n", + "[T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "**Note**: T5 Version 1.1 was only pre-trained on C4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task.\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "c0cd9f02", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9323615d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0b9994b3", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-v1_1-base\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "8daa264b", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/google/t5-v1_1-base](https://huggingface.co/google/t5-v1_1-base) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-v1_1-large/info.yaml b/modelcenter/community/google/t5-v1_1-large/info.yaml index b9315915..46201b82 100644 --- a/modelcenter/community/google/t5-v1_1-large/info.yaml +++ b/modelcenter/community/google/t5-v1_1-large/info.yaml @@ -1,26 +1,23 @@ +Datasets: c4 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "google/t5-v1_1-large" - description: "Version 1.1" - description_en: "Version 1.1" - icon: "" - from_repo: "https://huggingface.co/google/t5-v1_1-large" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4" -Publisher: "google" -License: "apache-2.0" -Language: "English" + description: Version 1.1 + description_en: Version 1.1 + from_repo: https://huggingface.co/google/t5-v1_1-large + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: google/t5-v1_1-large Paper: - - title: 'GLU Variants Improve Transformer' - url: 'http://arxiv.org/abs/2002.05202v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: GLU Variants Improve Transformer + url: http://arxiv.org/abs/2002.05202v1 +- title: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer + url: http://arxiv.org/abs/1910.10683v3 +Publisher: google +Task: +- sub_tag: 文本生成 + sub_tag_en: Text2Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/google/t5-v1_1-large/introduction_cn.ipynb b/modelcenter/community/google/t5-v1_1-large/introduction_cn.ipynb new file mode 100644 index 00000000..1ad787e7 --- /dev/null +++ b/modelcenter/community/google/t5-v1_1-large/introduction_cn.ipynb @@ -0,0 +1,101 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "11d36429", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1\n", + "\n", + "\n", + "## Version 1.1\n", + "\n", + "[T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "**Note**: T5 Version 1.1 was only pre-trained on C4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task.\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "c11ad8cf", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "480104f2", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e1323ff9", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-v1_1-large\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "4348828e", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/google/t5-v1_1-large](https://huggingface.co/google/t5-v1_1-large)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-v1_1-large/introduction_en.ipynb b/modelcenter/community/google/t5-v1_1-large/introduction_en.ipynb new file mode 100644 index 00000000..4efcfb2f --- /dev/null +++ b/modelcenter/community/google/t5-v1_1-large/introduction_en.ipynb @@ -0,0 +1,101 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "5f0c769f", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1\n", + "\n", + "\n", + "## Version 1.1\n", + "\n", + "[T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "**Note**: T5 Version 1.1 was only pre-trained on C4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task.\n", + "Pretraining Dataset: C4\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "27e206b9", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3cf23148", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "467b7ff7", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-v1_1-large\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "a5616bca", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/google/t5-v1_1-large](https://huggingface.co/google/t5-v1_1-large) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-v1_1-small/info.yaml b/modelcenter/community/google/t5-v1_1-small/info.yaml index ffaf1b74..8cd8aced 100644 --- a/modelcenter/community/google/t5-v1_1-small/info.yaml +++ b/modelcenter/community/google/t5-v1_1-small/info.yaml @@ -1,26 +1,23 @@ +Datasets: c4 +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "google/t5-v1_1-small" - description: "Version 1.1" - description_en: "Version 1.1" - icon: "" - from_repo: "https://huggingface.co/google/t5-v1_1-small" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Text2Text Generation" - sub_tag: "文本生成" - -Example: - -Datasets: "c4" -Publisher: "google" -License: "apache-2.0" -Language: "English" + description: Version 1.1 + description_en: Version 1.1 + from_repo: https://huggingface.co/google/t5-v1_1-small + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: google/t5-v1_1-small Paper: - - title: 'GLU Variants Improve Transformer' - url: 'http://arxiv.org/abs/2002.05202v1' - - title: 'Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer' - url: 'http://arxiv.org/abs/1910.10683v3' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: GLU Variants Improve Transformer + url: http://arxiv.org/abs/2002.05202v1 +- title: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer + url: http://arxiv.org/abs/1910.10683v3 +Publisher: google +Task: +- sub_tag: 文本生成 + sub_tag_en: Text2Text Generation + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/google/t5-v1_1-small/introduction_cn.ipynb b/modelcenter/community/google/t5-v1_1-small/introduction_cn.ipynb new file mode 100644 index 00000000..ebcc34ea --- /dev/null +++ b/modelcenter/community/google/t5-v1_1-small/introduction_cn.ipynb @@ -0,0 +1,103 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "51d7e9ca", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1\n", + "\n", + "\n", + "## Version 1.1\n", + "\n", + "[T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "**Note**: T5 Version 1.1 was only pre-trained on C4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task.\n", + "Pretraining Dataset: [C4](https://huggingface.co/datasets/c4)\n", + "\n", + "Other Community Checkpoints: [here](https://huggingface.co/models?search=t5-v1_1)\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "b4b5fc59", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ae31cbc9", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "81d25d09", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-v1_1-small\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "6e0459f7", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/google/t5-v1_1-small](https://huggingface.co/google/t5-v1_1-small)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/google/t5-v1_1-small/introduction_en.ipynb b/modelcenter/community/google/t5-v1_1-small/introduction_en.ipynb new file mode 100644 index 00000000..e89ecb34 --- /dev/null +++ b/modelcenter/community/google/t5-v1_1-small/introduction_en.ipynb @@ -0,0 +1,103 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "95b64b6f", + "metadata": {}, + "source": [ + "[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) Version 1.1\n", + "\n", + "\n", + "## Version 1.1\n", + "\n", + "[T5 Version 1.1](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511) includes the following improvements compared to the original T5 model- GEGLU activation in feed-forward hidden layer, rather than ReLU - see [here](https://arxiv.org/abs/2002.05202).\n", + "\n", + "- Dropout was turned off in pre-training (quality win). Dropout should be re-enabled during fine-tuning.\n", + "\n", + "- Pre-trained on C4 only without mixing in the downstream tasks.\n", + "\n", + "- no parameter sharing between embedding and classifier layer\n", + "\n", + "- \"xl\" and \"xxl\" replace \"3B\" and \"11B\". The model shapes are a bit different - larger `d_model` and smaller `num_heads` and `d_ff`.\n", + "\n", + "**Note**: T5 Version 1.1 was only pre-trained on C4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task.\n", + "Pretraining Dataset: [C4](https://huggingface.co/datasets/c4)\n", + "\n", + "Other Community Checkpoints: [here](https://huggingface.co/models?search=t5-v1_1)\n", + "\n", + "Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf)\n", + "\n", + "Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu*\n", + "\n", + "\n", + "## Abstract\n", + "\n", + "Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "88ec53f3", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "082cae7f", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "05f7f4d0", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"google/t5-v1_1-small\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "c7a95cdf", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/google/t5-v1_1-small](https://huggingface.co/google/t5-v1_1-small) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-bert-wwm-ext/info.yaml b/modelcenter/community/hfl/chinese-bert-wwm-ext/info.yaml index 8474c936..2e118e75 100644 --- a/modelcenter/community/hfl/chinese-bert-wwm-ext/info.yaml +++ b/modelcenter/community/hfl/chinese-bert-wwm-ext/info.yaml @@ -1,26 +1,23 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Chinese +License: apache-2.0 Model_Info: - name: "hfl/chinese-bert-wwm-ext" - description: "Chinese BERT with Whole Word Masking" - description_en: "Chinese BERT with Whole Word Masking" - icon: "" - from_repo: "https://huggingface.co/hfl/chinese-bert-wwm-ext" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "hfl" -License: "apache-2.0" -Language: "Chinese" + description: Chinese BERT with Whole Word Masking + description_en: Chinese BERT with Whole Word Masking + from_repo: https://huggingface.co/hfl/chinese-bert-wwm-ext + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: hfl/chinese-bert-wwm-ext Paper: - - title: 'Pre-Training with Whole Word Masking for Chinese BERT' - url: 'http://arxiv.org/abs/1906.08101v3' - - title: 'Revisiting Pre-Trained Models for Chinese Natural Language Processing' - url: 'http://arxiv.org/abs/2004.13922v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Pre-Training with Whole Word Masking for Chinese BERT + url: http://arxiv.org/abs/1906.08101v3 +- title: Revisiting Pre-Trained Models for Chinese Natural Language Processing + url: http://arxiv.org/abs/2004.13922v2 +Publisher: hfl +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/hfl/chinese-bert-wwm-ext/introduction_cn.ipynb b/modelcenter/community/hfl/chinese-bert-wwm-ext/introduction_cn.ipynb new file mode 100644 index 00000000..7d8a9de3 --- /dev/null +++ b/modelcenter/community/hfl/chinese-bert-wwm-ext/introduction_cn.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a5e1e8bd", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "be498a8f", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0199d11d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b71b0698", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-bert-wwm-ext\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "5d6bd99f", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "456616b7", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "9784d9b7", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "15ed9adf", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3593ecc9", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/hfl/chinese-bert-wwm-ext](https://huggingface.co/hfl/chinese-bert-wwm-ext)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-bert-wwm-ext/introduction_en.ipynb b/modelcenter/community/hfl/chinese-bert-wwm-ext/introduction_en.ipynb new file mode 100644 index 00000000..debab5d4 --- /dev/null +++ b/modelcenter/community/hfl/chinese-bert-wwm-ext/introduction_en.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "faeb5f50", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "fbf98c0e", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f6b3ac7", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f380cab7", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-bert-wwm-ext\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "a39bca7c", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "5cff4b49", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "a8781cbe", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "b7acc10f", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "86de1995", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/hfl/chinese-bert-wwm-ext](https://huggingface.co/hfl/chinese-bert-wwm-ext) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-bert-wwm/info.yaml b/modelcenter/community/hfl/chinese-bert-wwm/info.yaml index 06014fe9..78271b2f 100644 --- a/modelcenter/community/hfl/chinese-bert-wwm/info.yaml +++ b/modelcenter/community/hfl/chinese-bert-wwm/info.yaml @@ -1,26 +1,23 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Chinese +License: apache-2.0 Model_Info: - name: "hfl/chinese-bert-wwm" - description: "Chinese BERT with Whole Word Masking" - description_en: "Chinese BERT with Whole Word Masking" - icon: "" - from_repo: "https://huggingface.co/hfl/chinese-bert-wwm" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "hfl" -License: "apache-2.0" -Language: "Chinese" + description: Chinese BERT with Whole Word Masking + description_en: Chinese BERT with Whole Word Masking + from_repo: https://huggingface.co/hfl/chinese-bert-wwm + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: hfl/chinese-bert-wwm Paper: - - title: 'Pre-Training with Whole Word Masking for Chinese BERT' - url: 'http://arxiv.org/abs/1906.08101v3' - - title: 'Revisiting Pre-Trained Models for Chinese Natural Language Processing' - url: 'http://arxiv.org/abs/2004.13922v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Pre-Training with Whole Word Masking for Chinese BERT + url: http://arxiv.org/abs/1906.08101v3 +- title: Revisiting Pre-Trained Models for Chinese Natural Language Processing + url: http://arxiv.org/abs/2004.13922v2 +Publisher: hfl +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/hfl/chinese-bert-wwm/introduction_cn.ipynb b/modelcenter/community/hfl/chinese-bert-wwm/introduction_cn.ipynb new file mode 100644 index 00000000..6de568ac --- /dev/null +++ b/modelcenter/community/hfl/chinese-bert-wwm/introduction_cn.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a5e1e8bd", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "be498a8f", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0199d11d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b71b0698", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-bert-wwm\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "5d6bd99f", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "376186df", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "9784d9b7", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "478fe6be", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3593ecc9", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/hfl/chinese-bert-wwm](https://huggingface.co/hfl/chinese-bert-wwm)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-bert-wwm/introduction_en.ipynb b/modelcenter/community/hfl/chinese-bert-wwm/introduction_en.ipynb new file mode 100644 index 00000000..bba2578c --- /dev/null +++ b/modelcenter/community/hfl/chinese-bert-wwm/introduction_en.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "faeb5f50", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "fbf98c0e", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f6b3ac7", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f380cab7", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-bert-wwm\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "a39bca7c", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "0ebe185e", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "a8781cbe", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "85d2437a", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "86de1995", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/hfl/chinese-bert-wwm-ext](https://huggingface.co/hfl/chinese-bert-wwm-ext) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/info.yaml b/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/info.yaml index 537a4123..fc792db2 100644 --- a/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/info.yaml +++ b/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/info.yaml @@ -1,26 +1,23 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Chinese +License: apache-2.0 Model_Info: - name: "hfl/chinese-roberta-wwm-ext-large" - description: "Please use 'Bert' related functions to load this model!" - description_en: "Please use 'Bert' related functions to load this model!" - icon: "" - from_repo: "https://huggingface.co/hfl/chinese-roberta-wwm-ext-large" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "hfl" -License: "apache-2.0" -Language: "Chinese" + description: Please use 'Bert' related functions to load this model! + description_en: Please use 'Bert' related functions to load this model! + from_repo: https://huggingface.co/hfl/chinese-roberta-wwm-ext-large + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: hfl/chinese-roberta-wwm-ext-large Paper: - - title: 'Pre-Training with Whole Word Masking for Chinese BERT' - url: 'http://arxiv.org/abs/1906.08101v3' - - title: 'Revisiting Pre-Trained Models for Chinese Natural Language Processing' - url: 'http://arxiv.org/abs/2004.13922v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Pre-Training with Whole Word Masking for Chinese BERT + url: http://arxiv.org/abs/1906.08101v3 +- title: Revisiting Pre-Trained Models for Chinese Natural Language Processing + url: http://arxiv.org/abs/2004.13922v2 +Publisher: hfl +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/introduction_cn.ipynb b/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/introduction_cn.ipynb new file mode 100644 index 00000000..d0394493 --- /dev/null +++ b/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/introduction_cn.ipynb @@ -0,0 +1,156 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a5e1e8bd", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "\n", + "### Please use 'Bert' related functions to load this model!\n", + "\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "be498a8f", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0199d11d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b71b0698", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-roberta-wwm-ext-large\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "5d6bd99f", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "9429c396", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "9784d9b7", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "eb3e56a1", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3593ecc9", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/hfl/chinese-roberta-wwm-ext-large](https://huggingface.co/hfl/chinese-roberta-wwm-ext-large)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/introduction_en.ipynb b/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/introduction_en.ipynb new file mode 100644 index 00000000..43826e43 --- /dev/null +++ b/modelcenter/community/hfl/chinese-roberta-wwm-ext-large/introduction_en.ipynb @@ -0,0 +1,156 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "faeb5f50", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "\n", + "### Please use 'Bert' related functions to load this model!\n", + "\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "fbf98c0e", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f6b3ac7", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f380cab7", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-roberta-wwm-ext-large\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "a39bca7c", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "b01c1973", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "a8781cbe", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "7ad8a810", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "86de1995", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/hfl/chinese-roberta-wwm-ext-large](https://huggingface.co/hfl/chinese-roberta-wwm-ext-large) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-roberta-wwm-ext/info.yaml b/modelcenter/community/hfl/chinese-roberta-wwm-ext/info.yaml index c0e9a0a4..79b37c73 100644 --- a/modelcenter/community/hfl/chinese-roberta-wwm-ext/info.yaml +++ b/modelcenter/community/hfl/chinese-roberta-wwm-ext/info.yaml @@ -1,26 +1,23 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Chinese +License: apache-2.0 Model_Info: - name: "hfl/chinese-roberta-wwm-ext" - description: "Please use 'Bert' related functions to load this model!" - description_en: "Please use 'Bert' related functions to load this model!" - icon: "" - from_repo: "https://huggingface.co/hfl/chinese-roberta-wwm-ext" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "hfl" -License: "apache-2.0" -Language: "Chinese" + description: Please use 'Bert' related functions to load this model! + description_en: Please use 'Bert' related functions to load this model! + from_repo: https://huggingface.co/hfl/chinese-roberta-wwm-ext + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: hfl/chinese-roberta-wwm-ext Paper: - - title: 'Pre-Training with Whole Word Masking for Chinese BERT' - url: 'http://arxiv.org/abs/1906.08101v3' - - title: 'Revisiting Pre-Trained Models for Chinese Natural Language Processing' - url: 'http://arxiv.org/abs/2004.13922v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Pre-Training with Whole Word Masking for Chinese BERT + url: http://arxiv.org/abs/1906.08101v3 +- title: Revisiting Pre-Trained Models for Chinese Natural Language Processing + url: http://arxiv.org/abs/2004.13922v2 +Publisher: hfl +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/hfl/chinese-roberta-wwm-ext/introduction_cn.ipynb b/modelcenter/community/hfl/chinese-roberta-wwm-ext/introduction_cn.ipynb new file mode 100644 index 00000000..f55e7b22 --- /dev/null +++ b/modelcenter/community/hfl/chinese-roberta-wwm-ext/introduction_cn.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a5e1e8bd", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "be498a8f", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0199d11d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b71b0698", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-roberta-wwm-ext\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "5d6bd99f", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "737822b2", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "9784d9b7", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "22d0c28d", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3593ecc9", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/hfl/chinese-roberta-wwm-ext](https://huggingface.co/hfl/chinese-roberta-wwm-ext)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/chinese-roberta-wwm-ext/introduction_en.ipynb b/modelcenter/community/hfl/chinese-roberta-wwm-ext/introduction_en.ipynb new file mode 100644 index 00000000..50668500 --- /dev/null +++ b/modelcenter/community/hfl/chinese-roberta-wwm-ext/introduction_en.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "faeb5f50", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "fbf98c0e", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f6b3ac7", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f380cab7", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/chinese-roberta-wwm-ext\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "a39bca7c", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "f495aec9", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "a8781cbe", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "8eebfbf4", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "86de1995", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/hfl/chinese-roberta-wwm-ext](https://huggingface.co/hfl/chinese-roberta-wwm-ext) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/rbt3/info.yaml b/modelcenter/community/hfl/rbt3/info.yaml index 36c3c8d5..b03acf55 100644 --- a/modelcenter/community/hfl/rbt3/info.yaml +++ b/modelcenter/community/hfl/rbt3/info.yaml @@ -1,26 +1,23 @@ +Datasets: '' +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: Chinese +License: apache-2.0 Model_Info: - name: "hfl/rbt3" - description: "This is a re-trained 3-layer RoBERTa-wwm-ext model." - description_en: "This is a re-trained 3-layer RoBERTa-wwm-ext model." - icon: "" - from_repo: "https://huggingface.co/hfl/rbt3" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "" -Publisher: "hfl" -License: "apache-2.0" -Language: "Chinese" + description: This is a re-trained 3-layer RoBERTa-wwm-ext model. + description_en: This is a re-trained 3-layer RoBERTa-wwm-ext model. + from_repo: https://huggingface.co/hfl/rbt3 + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: hfl/rbt3 Paper: - - title: 'Pre-Training with Whole Word Masking for Chinese BERT' - url: 'http://arxiv.org/abs/1906.08101v3' - - title: 'Revisiting Pre-Trained Models for Chinese Natural Language Processing' - url: 'http://arxiv.org/abs/2004.13922v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: Pre-Training with Whole Word Masking for Chinese BERT + url: http://arxiv.org/abs/1906.08101v3 +- title: Revisiting Pre-Trained Models for Chinese Natural Language Processing + url: http://arxiv.org/abs/2004.13922v2 +Publisher: hfl +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/hfl/rbt3/introduction_cn.ipynb b/modelcenter/community/hfl/rbt3/introduction_cn.ipynb new file mode 100644 index 00000000..eb219922 --- /dev/null +++ b/modelcenter/community/hfl/rbt3/introduction_cn.ipynb @@ -0,0 +1,156 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a5e1e8bd", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "\n", + "### Please use 'Bert' related functions to load this model!\n", + "\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "be498a8f", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0199d11d", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b71b0698", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/rbt3\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "5d6bd99f", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "73e04675", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "9784d9b7", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "068895c6", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "3593ecc9", + "metadata": {}, + "source": [ + "> 此模型介绍及权重来源于[https://huggingface.co/hfl/rbt3](https://huggingface.co/hfl/rbt3)，并转换为飞桨模型格式。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/hfl/rbt3/introduction_en.ipynb b/modelcenter/community/hfl/rbt3/introduction_en.ipynb new file mode 100644 index 00000000..4d118772 --- /dev/null +++ b/modelcenter/community/hfl/rbt3/introduction_en.ipynb @@ -0,0 +1,156 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "faeb5f50", + "metadata": {}, + "source": [ + "## Chinese BERT with Whole Word Masking\n", + "\n", + "### Please use 'Bert' related functions to load this model!\n", + "\n", + "For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**.\n", + "\n", + "**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**\n", + "Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu\n", + "\n", + "This repository is developed based on：https://github.com/google-research/bert\n", + "\n", + "You may also interested in,\n", + "- Chinese BERT series: https://github.com/ymcui/Chinese-BERT-wwm\n", + "- Chinese MacBERT: https://github.com/ymcui/MacBERT\n", + "- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA\n", + "- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet\n", + "- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer\n", + "\n", + "More resources by HFL: https://github.com/ymcui/HFL-Anthology\n" + ] + }, + { + "cell_type": "markdown", + "id": "fbf98c0e", + "metadata": {}, + "source": [ + "## How to Use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f6b3ac7", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f380cab7", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"hfl/rbt3\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "a39bca7c", + "metadata": {}, + "source": [ + "\n", + "## Citation\n", + "If you find the technical report or resource is useful, please cite the following technical report in your paper.\n", + "- Primary: https://arxiv.org/abs/2004.13922" + ] + }, + { + "cell_type": "markdown", + "id": "370bfe67", + "metadata": {}, + "source": [ + "```\n", + "@inproceedings{cui-etal-2020-revisiting,\n", + "title = \"Revisiting Pre-Trained Models for {C}hinese Natural Language Processing\",\n", + "author = \"Cui, Yiming and\n", + "Che, Wanxiang and\n", + "Liu, Ting and\n", + "Qin, Bing and\n", + "Wang, Shijin and\n", + "Hu, Guoping\",\n", + "booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings\",\n", + "month = nov,\n", + "year = \"2020\",\n", + "address = \"Online\",\n", + "publisher = \"Association for Computational Linguistics\",\n", + "url = \"https://www.aclweb.org/anthology/2020.findings-emnlp.58\",\n", + "pages = \"657--668\",\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "a8781cbe", + "metadata": {}, + "source": [ + "- Secondary: https://arxiv.org/abs/1906.08101\n" + ] + }, + { + "cell_type": "markdown", + "id": "4a1fe5aa", + "metadata": {}, + "source": [ + "```\n", + "@article{chinese-bert-wwm,\n", + "title={Pre-Training with Whole Word Masking for Chinese BERT},\n", + "author={Cui, Yiming and Che, Wanxiang and Liu, Ting and Qin, Bing and Yang, Ziqing and Wang, Shijin and Hu, Guoping},\n", + "journal={arXiv preprint arXiv:1906.08101},\n", + "year={2019}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "86de1995", + "metadata": {}, + "source": [ + "> The model introduction and model weights originate from [https://huggingface.co/hfl/rbt3](https://huggingface.co/hfl/rbt3) and were converted to PaddlePaddle format for ease of use in PaddleNLP." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.13" + }, + "vscode": { + "interpreter": { + "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5" + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modelcenter/community/huggingface/bert-base-cased/info.yaml b/modelcenter/community/huggingface/bert-base-cased/info.yaml index 20c36035..467ef27a 100644 --- a/modelcenter/community/huggingface/bert-base-cased/info.yaml +++ b/modelcenter/community/huggingface/bert-base-cased/info.yaml @@ -1,24 +1,21 @@ +Datasets: bookcorpus,wikipedia +Example: null +IfOnlineDemo: 0 +IfTraining: 0 +Language: English +License: apache-2.0 Model_Info: - name: "bert-base-cased" - description: "BERT base model (cased)" - description_en: "BERT base model (cased)" - icon: "" - from_repo: "https://huggingface.co/bert-base-cased" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Fill-Mask" - sub_tag: "槽位填充" - -Example: - -Datasets: "bookcorpus,wikipedia" -Publisher: "huggingface" -License: "apache-2.0" -Language: "English" + description: BERT base model (cased) + description_en: BERT base model (cased) + from_repo: https://huggingface.co/bert-base-cased + icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png + name: bert-base-cased Paper: - - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding' - url: 'http://arxiv.org/abs/1810.04805v2' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file +- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding' + url: http://arxiv.org/abs/1810.04805v2 +Publisher: huggingface +Task: +- sub_tag: 槽位填充 + sub_tag_en: Fill-Mask + tag: 自然语言处理 + tag_en: Natural Language Processing diff --git a/modelcenter/community/huggingface/bert-base-cased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-base-cased/introduction_cn.ipynb new file mode 100644 index 00000000..66f9747f --- /dev/null +++ b/modelcenter/community/huggingface/bert-base-cased/introduction_cn.ipynb @@ -0,0 +1,170 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "bb008e6f", + "metadata": {}, + "source": [ + "# BERT base model (cased)\n", + "\n", + "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。" + ] + }, + { + "cell_type": "markdown", + "id": "079266fb", + "metadata": {}, + "source": [ + "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n", + "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n", + "[this repository](https://github.com/google-research/bert). This model is case-sensitive: it makes a difference between\n", + "english and English.\n" + ] + }, + { + "cell_type": "markdown", + "id": "5c8220aa", + "metadata": {}, + "source": [ + "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n", + "the Hugging Face team.\n" + ] + }, + { + "cell_type": "markdown", + "id": "8564477f", + "metadata": {}, + "source": [ + "## Model description\n" + ] + }, + { + "cell_type": "markdown", + "id": "7365685d", + "metadata": {}, + "source": [ + "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n", + "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n", + "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n", + "was pretrained with two objectives:\n" + ] + }, + { + "cell_type": "markdown", + "id": "1c979d12", + "metadata": {}, + "source": [ + "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n", + "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n", + "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n", + "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n", + "sentence.\n", + "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n", + "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n", + "predict if the two sentences were following each other or not.\n" + ] + }, + { + "cell_type": "markdown", + "id": "cdc00722", + "metadata": {}, + "source": [ + "This way, the model learns an inner representation of the English language that can then be used to extract features\n", + "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n", + "classifier using the features produced by the BERT model as inputs.\n" + ] + }, + { + "cell_type": "markdown", + "id": "9253a517", + "metadata": {}, + "source": [ + "## How to use" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9fbfcd0a", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install --upgrade paddlenlp" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6185db74", + "metadata": {}, + "outputs": [], + "source": [ + "import paddle\n", + "from paddlenlp.transformers import AutoModel\n", + "\n", + "model = AutoModel.from_pretrained(\"bert-base-cased\")\n", + "input_ids = paddle.randint(100, 200, shape=[1, 20])\n", + "print(model(input_ids))" + ] + }, + { + "cell_type": "markdown", + "id": "8e0ca3bd", + "metadata": {}, + "source": [ + "```\n", + "@article{DBLP:journals/corr/abs-1810-04805,\n", + "author = {Jacob Devlin and\n", + "Ming{-}Wei Chang and\n", + "Kenton Lee and\n", + "Kristina Toutanova},\n", + "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n", + "Understanding},\n", + "journal = {CoRR},\n", + "volume = {abs/1810.04805},\n", + "year = {2018},\n", + "url = {http://arxiv.org/abs/1810.04805},\n", + "archivePrefix = {arXiv},\n", + "eprint = {1810.04805},\n", + "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n", + "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n", + "bibsource = {dblp computer science bibliography, https://dblp.org}\n", + "}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "f14e9f06", + "metadata": {}, + "source": [ + "\n", + "
![](\"https://cdn-media.huggingface.co/exbert/button.png\")
\n",
+ "\n",
+ "\n",
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-base-cased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-cased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-base-cased/introduction_en.ipynb
new file mode 100644
index 00000000..40c4e44f
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-cased/introduction_en.ipynb
@@ -0,0 +1,171 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "58235e68",
+ "metadata": {},
+ "source": [
+ "# BERT base model (cased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "36c7d585",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is case-sensitive: it makes a difference between\n",
+ "english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d361a880",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "47b0cf99",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d1911491",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "94e45c66",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9fec6197",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5e17ee3b",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "62ae31d8",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "3c52bdd5",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "da7c4875",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "86873e48",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> The model introduction and model weights originate from https://huggingface.co/bert-base-cased and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-german-cased/info.yaml b/modelcenter/community/huggingface/bert-base-german-cased/info.yaml
index 56dceb1f..f549130c 100644
--- a/modelcenter/community/huggingface/bert-base-german-cased/info.yaml
+++ b/modelcenter/community/huggingface/bert-base-german-cased/info.yaml
@@ -1,23 +1,19 @@
+Datasets: ''
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: German
+License: mit
Model_Info:
- name: "bert-base-german-cased"
- description: "German BERT"
- description_en: "German BERT"
- icon: ""
- from_repo: "https://huggingface.co/bert-base-german-cased"
-
+ description: German BERT
+ description_en: German BERT
+ from_repo: https://huggingface.co/bert-base-german-cased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-base-german-cased
+Paper: null
+Publisher: huggingface
Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: ""
-Publisher: "huggingface"
-License: "mit"
-Language: "German"
-Paper:
-
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-base-german-cased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-base-german-cased/introduction_cn.ipynb
new file mode 100644
index 00000000..007d6df6
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-german-cased/introduction_cn.ipynb
@@ -0,0 +1,128 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "0870a629",
+ "metadata": {},
+ "source": [
+ "# German BERT\n",
+ "\n",
+ "## Overview\n",
+ "**Language model:** bert-base-cased\n",
+ "**Language:** German\n",
+ "**Training data:** Wiki, OpenLegalData, News (~ 12GB)\n",
+ "**Eval data:** Conll03 (NER), GermEval14 (NER), GermEval18 (Classification), GNAD (Classification)\n",
+ "**Infrastructure**: 1x TPU v2\n",
+ "**Published**: Jun 14th, 2019\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b2a6c897",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "1790135e",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "99c714ac",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-german-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "54c2f398",
+ "metadata": {},
+ "source": [
+ "## Authors\n",
+ "- Branden Chan: `branden.chan [at] deepset.ai`\n",
+ "- Timo Möller: `timo.moeller [at] deepset.ai`\n",
+ "- Malte Pietsch: `malte.pietsch [at] deepset.ai`\n",
+ "- Tanay Soni: `tanay.soni [at] deepset.ai`\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "94b669bc",
+ "metadata": {},
+ "source": [
+ "## About us\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ce90710a",
+ "metadata": {},
+ "source": [
+ "We bring NLP to the industry via open source!\n",
+ "Our focus: Industry specific language models & large scale QA systems.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5dc8ba63",
+ "metadata": {},
+ "source": [
+ "Some of our work:\n",
+ "- [German BERT (aka \"bert-base-german-cased\")](https://deepset.ai/german-bert)\n",
+ "- [FARM](https://github.com/deepset-ai/FARM)\n",
+ "- [Haystack](https://github.com/deepset-ai/haystack/)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "56a1a360",
+ "metadata": {},
+ "source": [
+ "Get in touch:\n",
+ "[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai)\n",
+ "\n",
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-base-german-cased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-german-cased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-base-german-cased/introduction_en.ipynb
new file mode 100644
index 00000000..60d78c67
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-german-cased/introduction_en.ipynb
@@ -0,0 +1,137 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "7aa268f7",
+ "metadata": {},
+ "source": [
+ "# German BERT\n",
+ "\n",
+ "## Overview\n",
+ "**Language model:** bert-base-cased\n",
+ "**Language:** German\n",
+ "**Training data:** Wiki, OpenLegalData, News (~ 12GB)\n",
+ "**Eval data:** Conll03 (NER), GermEval14 (NER), GermEval18 (Classification), GNAD (Classification)\n",
+ "**Infrastructure**: 1x TPU v2\n",
+ "**Published**: Jun 14th, 2019\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f407e80e",
+ "metadata": {},
+ "source": [
+ "**Update April 3rd, 2020**: we updated the vocabulary file on deepset's s3 to conform with the default tokenization of punctuation tokens.\n",
+ "For details see the related [FARM issue](https://github.com/deepset-ai/FARM/issues/60). If you want to use the old vocab we have also uploaded a deepset/bert-base-german-cased-oldvocab model.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "18d2ad8e",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "b80052bd",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "4ea9d4e3",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-german-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9d560e75",
+ "metadata": {},
+ "source": [
+ "## Authors\n",
+ "- Branden Chan: `branden.chan [at] deepset.ai`\n",
+ "- Timo Möller: `timo.moeller [at] deepset.ai`\n",
+ "- Malte Pietsch: `malte.pietsch [at] deepset.ai`\n",
+ "- Tanay Soni: `tanay.soni [at] deepset.ai`\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a0e43273",
+ "metadata": {},
+ "source": [
+ "## About us\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c1b05e60",
+ "metadata": {},
+ "source": [
+ "We bring NLP to the industry via open source!\n",
+ "Our focus: Industry specific language models & large scale QA systems.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5196bee9",
+ "metadata": {},
+ "source": [
+ "Some of our work:\n",
+ "- [German BERT (aka \"bert-base-german-cased\")](https://deepset.ai/german-bert)\n",
+ "- [FARM](https://github.com/deepset-ai/FARM)\n",
+ "- [Haystack](https://github.com/deepset-ai/haystack/)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "18fe01d5",
+ "metadata": {},
+ "source": [
+ "Get in touch:\n",
+ "[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai)\n",
+ "\n",
+ "> The model introduction and model weights originate from https://huggingface.co/bert-base-german-cased and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-multilingual-cased/info.yaml b/modelcenter/community/huggingface/bert-base-multilingual-cased/info.yaml
index 6f6e9996..30aa91f6 100644
--- a/modelcenter/community/huggingface/bert-base-multilingual-cased/info.yaml
+++ b/modelcenter/community/huggingface/bert-base-multilingual-cased/info.yaml
@@ -1,24 +1,21 @@
+Datasets: wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: ''
+License: apache-2.0
Model_Info:
- name: "bert-base-multilingual-cased"
- description: "BERT multilingual base model (cased)"
- description_en: "BERT multilingual base model (cased)"
- icon: ""
- from_repo: "https://huggingface.co/bert-base-multilingual-cased"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: ""
+ description: BERT multilingual base model (cased)
+ description_en: BERT multilingual base model (cased)
+ from_repo: https://huggingface.co/bert-base-multilingual-cased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-base-multilingual-cased
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-base-multilingual-cased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-base-multilingual-cased/introduction_cn.ipynb
new file mode 100644
index 00000000..6ef00469
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-multilingual-cased/introduction_cn.ipynb
@@ -0,0 +1,166 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "92e18984",
+ "metadata": {},
+ "source": [
+ "# BERT multilingual base model (cased)\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cc38bad3",
+ "metadata": {},
+ "source": [
+ "Pretrained model on the top 104 languages with the largest Wikipedia using a masked language modeling (MLM) objective.\n",
+ "It was introduced in [this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is case sensitive: it makes a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a54cdf6e",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3be641ef",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "93fd337b",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of multilingual data in a self-supervised fashion. This means\n",
+ "it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2222d4b6",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2f9ea64e",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the languages in the training set that can then be used to\n",
+ "extract features useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a\n",
+ "standard classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7363abb0",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "780c0123",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "5a325830",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-multilingual-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "81ca575a",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "216555c3",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-base-multilingual-cased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-multilingual-cased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-base-multilingual-cased/introduction_en.ipynb
new file mode 100644
index 00000000..614f4494
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-multilingual-cased/introduction_en.ipynb
@@ -0,0 +1,167 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "19d62907",
+ "metadata": {},
+ "source": [
+ "# BERT multilingual base model (cased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "09809b94",
+ "metadata": {},
+ "source": [
+ "Pretrained model on the top 104 languages with the largest Wikipedia using a masked language modeling (MLM) objective.\n",
+ "It was introduced in [this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is case sensitive: it makes a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d3a52162",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f67f02dc",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bf05022f",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of multilingual data in a self-supervised fashion. This means\n",
+ "it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "081a7a88",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "79e6eda9",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the languages in the training set that can then be used to\n",
+ "extract features useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a\n",
+ "standard classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1696fb24",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "4f7d20fd",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "3c369c9a",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-multilingual-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6338f981",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c55dc64e",
+ "metadata": {},
+ "source": [
+ "\n",
+ "> The model introduction and model weights originate from https://huggingface.co/bert-base-multilingual-cased and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-multilingual-uncased/info.yaml b/modelcenter/community/huggingface/bert-base-multilingual-uncased/info.yaml
index eea48a95..d4008a99 100644
--- a/modelcenter/community/huggingface/bert-base-multilingual-uncased/info.yaml
+++ b/modelcenter/community/huggingface/bert-base-multilingual-uncased/info.yaml
@@ -1,24 +1,21 @@
+Datasets: wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: ''
+License: apache-2.0
Model_Info:
- name: "bert-base-multilingual-uncased"
- description: "BERT multilingual base model (uncased)"
- description_en: "BERT multilingual base model (uncased)"
- icon: ""
- from_repo: "https://huggingface.co/bert-base-multilingual-uncased"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: ""
+ description: BERT multilingual base model (uncased)
+ description_en: BERT multilingual base model (uncased)
+ from_repo: https://huggingface.co/bert-base-multilingual-uncased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-base-multilingual-uncased
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-base-multilingual-uncased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-base-multilingual-uncased/introduction_cn.ipynb
new file mode 100644
index 00000000..81274e37
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-multilingual-uncased/introduction_cn.ipynb
@@ -0,0 +1,166 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "867cb6e6",
+ "metadata": {},
+ "source": [
+ "# BERT multilingual base model (uncased)\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "207ffc57",
+ "metadata": {},
+ "source": [
+ "Pretrained model on the top 102 languages with the largest Wikipedia using a masked language modeling (MLM) objective.\n",
+ "It was introduced in [this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8b2e2c13",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "40d071c9",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "af4a1260",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of multilingual data in a self-supervised fashion. This means\n",
+ "it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "81abfbcb",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "64988b6b",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the languages in the training set that can then be used to\n",
+ "extract features useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a\n",
+ "standard classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "79c3e104",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "d7b2d0ec",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "52f8d16d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-multilingual-uncased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f11b298a",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "548a9d6c",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-base-multilingual-uncased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-multilingual-uncased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-base-multilingual-uncased/introduction_en.ipynb
new file mode 100644
index 00000000..3c641136
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-multilingual-uncased/introduction_en.ipynb
@@ -0,0 +1,166 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "ff0c69a5",
+ "metadata": {},
+ "source": [
+ "# BERT multilingual base model (uncased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1ad499a9",
+ "metadata": {},
+ "source": [
+ "Pretrained model on the top 102 languages with the largest Wikipedia using a masked language modeling (MLM) objective.\n",
+ "It was introduced in [this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a8878d0c",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4581e670",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c8d5f59f",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of multilingual data in a self-supervised fashion. This means\n",
+ "it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "836834df",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bafe70e4",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the languages in the training set that can then be used to\n",
+ "extract features useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a\n",
+ "standard classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cf2a29e2",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "dc792d6e",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "7a6faf50",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-multilingual-uncased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5b616f23",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "67e01093",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/bert-base-multilingual-uncased and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-uncased/info.yaml b/modelcenter/community/huggingface/bert-base-uncased/info.yaml
index c5b7c80f..e60141f5 100644
--- a/modelcenter/community/huggingface/bert-base-uncased/info.yaml
+++ b/modelcenter/community/huggingface/bert-base-uncased/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-base-uncased"
- description: "BERT base model (uncased)"
- description_en: "BERT base model (uncased)"
- icon: ""
- from_repo: "https://huggingface.co/bert-base-uncased"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT base model (uncased)
+ description_en: BERT base model (uncased)
+ from_repo: https://huggingface.co/bert-base-uncased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-base-uncased
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-base-uncased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-base-uncased/introduction_cn.ipynb
new file mode 100644
index 00000000..c848895e
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-uncased/introduction_cn.ipynb
@@ -0,0 +1,214 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "a14866e7",
+ "metadata": {},
+ "source": [
+ "# BERT base model (uncased)\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d348c680",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9a790b40",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "985d2894",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "985bd7ee",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labeling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2e1ee5f4",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally masks the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ae584a51",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences, for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a4d48848",
+ "metadata": {},
+ "source": [
+ "## Model variations\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dcb46068",
+ "metadata": {},
+ "source": [
+ "BERT has originally been released in base and large variations, for cased and uncased input text. The uncased models also strips out an accent markers.\n",
+ "Chinese and multilingual uncased and cased versions followed shortly after.\n",
+ "Modified preprocessing with whole word masking has replaced subpiece masking in a following work, with the release of two models.\n",
+ "Other 24 smaller models are released afterward.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bdf3ec7e",
+ "metadata": {},
+ "source": [
+ "The detailed release history can be found on the [google-research/bert readme](https://github.com/google-research/bert/blob/master/README.md) on github.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d66e6fc4",
+ "metadata": {},
+ "source": [
+ "| Model | #params | Language |\n",
+ "|------------------------|--------------------------------|-------|\n",
+ "| [`bert-base-uncased`](https://huggingface.co/bert-base-uncased) | 110M | English |\n",
+ "| [`bert-large-uncased`](https://huggingface.co/bert-large-uncased) | 340M | English | sub\n",
+ "| [`bert-base-cased`](https://huggingface.co/bert-base-cased) | 110M | English |\n",
+ "| [`bert-large-cased`](https://huggingface.co/bert-large-cased) | 340M | English |\n",
+ "| [`bert-base-chinese`](https://huggingface.co/bert-base-chinese) | 110M | Chinese |\n",
+ "| [`bert-base-multilingual-cased`](https://huggingface.co/bert-base-multilingual-cased) | 110M | Multiple |\n",
+ "| [`bert-large-uncased-whole-word-masking`](https://huggingface.co/bert-large-uncased-whole-word-masking) | 340M | English |\n",
+ "| [`bert-large-cased-whole-word-masking`](https://huggingface.co/bert-large-cased-whole-word-masking) | 340M | English |\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "93c97712",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "e4daab88",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "09dec4f3",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-uncased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "85541d34",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "82898490",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-base-uncased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-base-uncased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-base-uncased/introduction_en.ipynb
new file mode 100644
index 00000000..ea2c73ba
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-base-uncased/introduction_en.ipynb
@@ -0,0 +1,214 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "86c2dd31",
+ "metadata": {},
+ "source": [
+ "# BERT base model (uncased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e25590e2",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "632646c9",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6d37733d",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "20eb0099",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labeling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a43bc44c",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally masks the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3ea31760",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences, for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c44e01b0",
+ "metadata": {},
+ "source": [
+ "## Model variations\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6cb3e530",
+ "metadata": {},
+ "source": [
+ "BERT has originally been released in base and large variations, for cased and uncased input text. The uncased models also strips out an accent markers.\n",
+ "Chinese and multilingual uncased and cased versions followed shortly after.\n",
+ "Modified preprocessing with whole word masking has replaced subpiece masking in a following work, with the release of two models.\n",
+ "Other 24 smaller models are released afterward.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "557a417a",
+ "metadata": {},
+ "source": [
+ "The detailed release history can be found on the [google-research/bert readme](https://github.com/google-research/bert/blob/master/README.md) on github.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0f4bf9e0",
+ "metadata": {},
+ "source": [
+ "| Model | #params | Language |\n",
+ "|------------------------|--------------------------------|-------|\n",
+ "| [`bert-base-uncased`](https://huggingface.co/bert-base-uncased) | 110M | English |\n",
+ "| [`bert-large-uncased`](https://huggingface.co/bert-large-uncased) | 340M | English | sub\n",
+ "| [`bert-base-cased`](https://huggingface.co/bert-base-cased) | 110M | English |\n",
+ "| [`bert-large-cased`](https://huggingface.co/bert-large-cased) | 340M | English |\n",
+ "| [`bert-base-chinese`](https://huggingface.co/bert-base-chinese) | 110M | Chinese |\n",
+ "| [`bert-base-multilingual-cased`](https://huggingface.co/bert-base-multilingual-cased) | 110M | Multiple |\n",
+ "| [`bert-large-uncased-whole-word-masking`](https://huggingface.co/bert-large-uncased-whole-word-masking) | 340M | English |\n",
+ "| [`bert-large-cased-whole-word-masking`](https://huggingface.co/bert-large-cased-whole-word-masking) | 340M | English |\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "909c1c8d",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "68db3da7",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "04d6a56d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-base-uncased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "76d1a4dc",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1bcee897",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> The model introduction and model weights originate from https://huggingface.co/bert-base-uncased and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/info.yaml b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/info.yaml
index f1718d55..a7ac22ba 100644
--- a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/info.yaml
+++ b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-large-cased-whole-word-masking-finetuned-squad"
- description: "BERT large model (cased) whole word masking finetuned on SQuAD"
- description_en: "BERT large model (cased) whole word masking finetuned on SQuAD"
- icon: ""
- from_repo: "https://huggingface.co/bert-large-cased-whole-word-masking-finetuned-squad"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Question Answering"
- sub_tag: "回答问题"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT large model (cased) whole word masking finetuned on SQuAD
+ description_en: BERT large model (cased) whole word masking finetuned on SQuAD
+ from_repo: https://huggingface.co/bert-large-cased-whole-word-masking-finetuned-squad
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-large-cased-whole-word-masking-finetuned-squad
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 回答问题
+ sub_tag_en: Question Answering
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/introduction_cn.ipynb
new file mode 100644
index 00000000..d291733e
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/introduction_cn.ipynb
@@ -0,0 +1,208 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "7b02f8e4",
+ "metadata": {},
+ "source": [
+ "# BERT large model (cased) whole word masking finetuned on SQuAD\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7804aeec",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is cased: it makes a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9ee7c4ee",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2198ff25",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "159c04c3",
+ "metadata": {},
+ "source": [
+ "After pre-training, this model was fine-tuned on the SQuAD dataset with one of our fine-tuning scripts. See below for more information regarding this fine-tuning.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cec53443",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0a6d113e",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3776a729",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8ef5e147",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f494c97f",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "eeffccad",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7754e7ed",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dc30e3d4",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "6c0a8e7e",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "3eb39f84",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-cased-whole-word-masking-finetuned-squad\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "82b3ff37",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cb789a5a",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-large-cased-whole-word-masking-finetuned-squad ，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/introduction_en.ipynb b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/introduction_en.ipynb
new file mode 100644
index 00000000..022a59dc
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking-finetuned-squad/introduction_en.ipynb
@@ -0,0 +1,208 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "1dfd11c1",
+ "metadata": {},
+ "source": [
+ "# BERT large model (cased) whole word masking finetuned on SQuAD\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7105fb8c",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is cased: it makes a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e3d8b394",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "be078628",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "278aee7f",
+ "metadata": {},
+ "source": [
+ "After pre-training, this model was fine-tuned on the SQuAD dataset with one of our fine-tuning scripts. See below for more information regarding this fine-tuning.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ce69aca2",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "89b52c17",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2ebe9e94",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7131c024",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4a8e4aea",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "717dd1f6",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6778930f",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1ffc0609",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "678acd58",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "c5318a0c",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-cased-whole-word-masking-finetuned-squad\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f930fd97",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b3240bd3",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/bert-large-cased-whole-word-masking-finetuned-squad and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/info.yaml b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/info.yaml
index b7bf0bde..cbf482fb 100644
--- a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/info.yaml
+++ b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-large-cased-whole-word-masking"
- description: "BERT large model (cased) whole word masking"
- description_en: "BERT large model (cased) whole word masking"
- icon: ""
- from_repo: "https://huggingface.co/bert-large-cased-whole-word-masking"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT large model (cased) whole word masking
+ description_en: BERT large model (cased) whole word masking
+ from_repo: https://huggingface.co/bert-large-cased-whole-word-masking
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-large-cased-whole-word-masking
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/introduction_cn.ipynb
new file mode 100644
index 00000000..e9d6ca52
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/introduction_cn.ipynb
@@ -0,0 +1,201 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "1d5ffd6a",
+ "metadata": {},
+ "source": [
+ "# BERT large model (cased) whole word masking\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9e7590bd",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is cased: it makes a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2456751c",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "204d6ee6",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "743ff269",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bce1ffcc",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d5d83b7c",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "38a98598",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "89b5e554",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3f205174",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6b9cf751",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "74a0400e",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "d8952fcc",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "365e04c2",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-cased-whole-word-masking\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1cef8f18",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0d54ff2d",
+ "metadata": {},
+ "source": [
+ "\n",
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-large-cased-whole-word-masking ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/introduction_en.ipynb b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/introduction_en.ipynb
new file mode 100644
index 00000000..9617a15c
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-cased-whole-word-masking/introduction_en.ipynb
@@ -0,0 +1,237 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "58f64e54",
+ "metadata": {},
+ "source": [
+ "# BERT large model (cased) whole word masking\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6814fe73",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is cased: it makes a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7a6b1b28",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e6c8ddc5",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dfcd9c6b",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d758dbd9",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c4e44287",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d07abc2a",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5fcb83d6",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1be2f6a5",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cd047a65",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "93925c79",
+ "metadata": {},
+ "source": [
+ "## Intended uses & limitations\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f6c1f9b9",
+ "metadata": {},
+ "source": [
+ "You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to\n",
+ "be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=bert) to look for\n",
+ "fine-tuned versions on a task that interests you.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a682ee5c",
+ "metadata": {},
+ "source": [
+ "Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked)\n",
+ "to make decisions, such as sequence classification, token classification or question answering. For tasks such as text\n",
+ "generation you should look at model like GPT2.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "394e6456",
+ "metadata": {},
+ "source": [
+ "### How to use\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9e5fdb9a",
+ "metadata": {},
+ "source": [
+ "You can use this model directly with a pipeline for masked language modeling:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "77af91fe",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "ae5caf8d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-cased-whole-word-masking\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0f43705d",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "54ae4165",
+ "metadata": {},
+ "source": [
+ "\n",
+ "> The model introduction and model weights originate from https://huggingface.co/bert-large-cased-whole-word-masking and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-cased/info.yaml b/modelcenter/community/huggingface/bert-large-cased/info.yaml
index ae878bc6..89d74497 100644
--- a/modelcenter/community/huggingface/bert-large-cased/info.yaml
+++ b/modelcenter/community/huggingface/bert-large-cased/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-large-cased"
- description: "BERT large model (cased)"
- description_en: "BERT large model (cased)"
- icon: ""
- from_repo: "https://huggingface.co/bert-large-cased"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT large model (cased)
+ description_en: BERT large model (cased)
+ from_repo: https://huggingface.co/bert-large-cased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-large-cased
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-large-cased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-large-cased/introduction_cn.ipynb
new file mode 100644
index 00000000..6a92402e
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-cased/introduction_cn.ipynb
@@ -0,0 +1,213 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "360e146a",
+ "metadata": {},
+ "source": [
+ "# BERT large model (cased)\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bb3eb868",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is cased: it makes a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0f512012",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8dfae0e4",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "29d97a32",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "84dd3c36",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dbb66981",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4a3d9a5c",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "85a286cd",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e9f5c5f1",
+ "metadata": {},
+ "source": [
+ "## Intended uses & limitations\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d3ae1617",
+ "metadata": {},
+ "source": [
+ "You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to\n",
+ "be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=bert) to look for\n",
+ "fine-tuned versions on a task that interests you.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1d814aa3",
+ "metadata": {},
+ "source": [
+ "Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked)\n",
+ "to make decisions, such as sequence classification, token classification or question answering. For tasks such as text\n",
+ "generation you should look at model like GPT2.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7c9cb698",
+ "metadata": {},
+ "source": [
+ "### How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "266349de",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "e0d0fb84",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d58fffcd",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8591ee7f",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-large-cased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-cased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-large-cased/introduction_en.ipynb
new file mode 100644
index 00000000..e4795deb
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-cased/introduction_en.ipynb
@@ -0,0 +1,185 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "2460ffb6",
+ "metadata": {},
+ "source": [
+ "# BERT large model (cased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "07c2aecf",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is cased: it makes a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "fb6201f0",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ffd4c0b9",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0b465123",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7a5eb557",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d40678bb",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8fc24335",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "355e9553",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "47e2e497",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "c4d80b50",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f73f3925",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2873617b",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bc4aea4d",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/bert-large-cased and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/info.yaml b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/info.yaml
index 0a4f922c..a7510ff8 100644
--- a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/info.yaml
+++ b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-large-uncased-whole-word-masking-finetuned-squad"
- description: "BERT large model (uncased) whole word masking finetuned on SQuAD"
- description_en: "BERT large model (uncased) whole word masking finetuned on SQuAD"
- icon: ""
- from_repo: "https://huggingface.co/bert-large-uncased-whole-word-masking-finetuned-squad"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Question Answering"
- sub_tag: "回答问题"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT large model (uncased) whole word masking finetuned on SQuAD
+ description_en: BERT large model (uncased) whole word masking finetuned on SQuAD
+ from_repo: https://huggingface.co/bert-large-uncased-whole-word-masking-finetuned-squad
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-large-uncased-whole-word-masking-finetuned-squad
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 回答问题
+ sub_tag_en: Question Answering
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/introduction_cn.ipynb
new file mode 100644
index 00000000..93f59ac9
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/introduction_cn.ipynb
@@ -0,0 +1,207 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "aad9532a",
+ "metadata": {},
+ "source": [
+ "# BERT large model (uncased) whole word masking finetuned on SQuAD\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "724df271",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f2b9e3bf",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6566eb12",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7b45422b",
+ "metadata": {},
+ "source": [
+ "After pre-training, this model was fine-tuned on the SQuAD dataset with one of our fine-tuning scripts. See below for more information regarding this fine-tuning.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c9957f91",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "43cba468",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "457bfeee",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "77c83270",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0ba87de6",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f363132f",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "83a4e49f",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "68565c6d",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "457a1c54",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "a9369c0d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-uncased-whole-word-masking-finetuned-squad\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c5fefb8f",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "654c0920",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-large-uncased-whole-word-masking-finetuned-squad ，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/introduction_en.ipynb b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/introduction_en.ipynb
new file mode 100644
index 00000000..f940e050
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking-finetuned-squad/introduction_en.ipynb
@@ -0,0 +1,209 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "2d4f4368",
+ "metadata": {},
+ "source": [
+ "# BERT large model (uncased) whole word masking finetuned on SQuAD\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "afef45e0",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c94536b9",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "50254dea",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4b482be9",
+ "metadata": {},
+ "source": [
+ "After pre-training, this model was fine-tuned on the SQuAD dataset with one of our fine-tuning scripts. See below for more information regarding this fine-tuning.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "adfc36af",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "22f554a7",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "eccd3048",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3d4098e8",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1047d1ad",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7046db0c",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "09659088",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "65769919",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "4449cfac",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "1e8dcf70",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-uncased-whole-word-masking-finetuned-squad\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "49471f4b",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d783c8fc",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/bert-large-uncased-whole-word-masking-finetuned-squad and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/info.yaml b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/info.yaml
index 82a56eb1..8629aba0 100644
--- a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/info.yaml
+++ b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-large-uncased-whole-word-masking"
- description: "BERT large model (uncased) whole word masking"
- description_en: "BERT large model (uncased) whole word masking"
- icon: ""
- from_repo: "https://huggingface.co/bert-large-uncased-whole-word-masking"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT large model (uncased) whole word masking
+ description_en: BERT large model (uncased) whole word masking
+ from_repo: https://huggingface.co/bert-large-uncased-whole-word-masking
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-large-uncased-whole-word-masking
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/introduction_cn.ipynb
new file mode 100644
index 00000000..a686b3e0
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/introduction_cn.ipynb
@@ -0,0 +1,201 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "cf43e770",
+ "metadata": {},
+ "source": [
+ "# BERT large model (uncased) whole word masking\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "af8c3816",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c103e84b",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "19a76368",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "67f11a2c",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "778cf97d",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dddbb307",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "40becad1",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3fc265b6",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "65e4a308",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6d0b86c1",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dd94b8be",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "bc669f99",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "4580650d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-uncased-whole-word-masking\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "475fd35d",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f09b9b09",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-large-uncased-whole-word-masking ，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/introduction_en.ipynb b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/introduction_en.ipynb
new file mode 100644
index 00000000..2557af1b
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-uncased-whole-word-masking/introduction_en.ipynb
@@ -0,0 +1,237 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "ceefe52d",
+ "metadata": {},
+ "source": [
+ "# BERT large model (uncased) whole word masking\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "14552c09",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "78d1e4a0",
+ "metadata": {},
+ "source": [
+ "Differently to other BERT models, this model was trained with a new technique: Whole Word Masking. In this case, all of the tokens corresponding to a word are masked at once. The overall masking rate remains the same.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cdbe484a",
+ "metadata": {},
+ "source": [
+ "The training is identical -- each masked WordPiece token is predicted independently.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "fdbba80d",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "aba33624",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "459ca6e6",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "65f2ae1a",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "86e8d7eb",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b81821d8",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3a576172",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0038f06c",
+ "metadata": {},
+ "source": [
+ "## Intended uses & limitations\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ba8c18de",
+ "metadata": {},
+ "source": [
+ "You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to\n",
+ "be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=bert) to look for\n",
+ "fine-tuned versions on a task that interests you.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bb72ad39",
+ "metadata": {},
+ "source": [
+ "Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked)\n",
+ "to make decisions, such as sequence classification, token classification or question answering. For tasks such as text\n",
+ "generation you should look at model like GPT2.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "54b59ca8",
+ "metadata": {},
+ "source": [
+ "### How to use\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a0ff2a80",
+ "metadata": {},
+ "source": [
+ "You can use this model directly with a pipeline for masked language modeling:\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "990ce14a",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "7d468ffb",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-uncased-whole-word-masking\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "93d6e9e4",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c9d05272",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/bert-large-uncased-whole-word-masking and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-uncased/info.yaml b/modelcenter/community/huggingface/bert-large-uncased/info.yaml
index 41677ffc..cf05a3fb 100644
--- a/modelcenter/community/huggingface/bert-large-uncased/info.yaml
+++ b/modelcenter/community/huggingface/bert-large-uncased/info.yaml
@@ -1,24 +1,21 @@
+Datasets: bookcorpus,wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "bert-large-uncased"
- description: "BERT large model (uncased)"
- description_en: "BERT large model (uncased)"
- icon: ""
- from_repo: "https://huggingface.co/bert-large-uncased"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "bookcorpus,wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: BERT large model (uncased)
+ description_en: BERT large model (uncased)
+ from_repo: https://huggingface.co/bert-large-uncased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: bert-large-uncased
Paper:
- - title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
- url: 'http://arxiv.org/abs/1810.04805v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'
+ url: http://arxiv.org/abs/1810.04805v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/bert-large-uncased/introduction_cn.ipynb b/modelcenter/community/huggingface/bert-large-uncased/introduction_cn.ipynb
new file mode 100644
index 00000000..dabec910
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-uncased/introduction_cn.ipynb
@@ -0,0 +1,185 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "c000df74",
+ "metadata": {},
+ "source": [
+ "# BERT large model (uncased)\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bd7436a9",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "87c430c2",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e2004f07",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ad86c301",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8f12ab3c",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3fc80525",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c31d15b4",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "822f7f40",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7fcdeb04",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "db4ceaa3",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "dc6a0473",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-uncased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1156d387",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9d07ca08",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/bert-large-uncased ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/bert-large-uncased/introduction_en.ipynb b/modelcenter/community/huggingface/bert-large-uncased/introduction_en.ipynb
new file mode 100644
index 00000000..e73afd86
--- /dev/null
+++ b/modelcenter/community/huggingface/bert-large-uncased/introduction_en.ipynb
@@ -0,0 +1,185 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "a4fae520",
+ "metadata": {},
+ "source": [
+ "# BERT large model (uncased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c410d1ae",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in\n",
+ "[this paper](https://arxiv.org/abs/1810.04805) and first released in\n",
+ "[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference\n",
+ "between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "40166ab8",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by\n",
+ "the Hugging Face team.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dacb968e",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c519206d",
+ "metadata": {},
+ "source": [
+ "BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it\n",
+ "was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of\n",
+ "publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it\n",
+ "was pretrained with two objectives:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2dd87a78",
+ "metadata": {},
+ "source": [
+ "- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run\n",
+ "the entire masked sentence through the model and has to predict the masked words. This is different from traditional\n",
+ "recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like\n",
+ "GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the\n",
+ "sentence.\n",
+ "- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes\n",
+ "they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to\n",
+ "predict if the two sentences were following each other or not.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "507ce60a",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard\n",
+ "classifier using the features produced by the BERT model as inputs.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7fb7a8a0",
+ "metadata": {},
+ "source": [
+ "This model has the following configuration:\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ebe2c593",
+ "metadata": {},
+ "source": [
+ "- 24-layer\n",
+ "- 1024 hidden dimension\n",
+ "- 16 attention heads\n",
+ "- 336M parameters.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "547e3cc8",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "669cb05f",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "09a4bc02",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"bert-large-uncased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3ae36313",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "@article{DBLP:journals/corr/abs-1810-04805,\n",
+ "author = {Jacob Devlin and\n",
+ "Ming{-}Wei Chang and\n",
+ "Kenton Lee and\n",
+ "Kristina Toutanova},\n",
+ "title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language\n",
+ "Understanding},\n",
+ "journal = {CoRR},\n",
+ "volume = {abs/1810.04805},\n",
+ "year = {2018},\n",
+ "url = {http://arxiv.org/abs/1810.04805},\n",
+ "archivePrefix = {arXiv},\n",
+ "eprint = {1810.04805},\n",
+ "timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},\n",
+ "biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},\n",
+ "bibsource = {dblp computer science bibliography, https://dblp.org}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bed31ba3",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/bert-large-uncased and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/distilbert-base-multilingual-cased/info.yaml b/modelcenter/community/huggingface/distilbert-base-multilingual-cased/info.yaml
index 24ef1d56..3e9ab3a6 100644
--- a/modelcenter/community/huggingface/distilbert-base-multilingual-cased/info.yaml
+++ b/modelcenter/community/huggingface/distilbert-base-multilingual-cased/info.yaml
@@ -1,26 +1,23 @@
+Datasets: wikipedia
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: ''
+License: apache-2.0
Model_Info:
- name: "distilbert-base-multilingual-cased"
- description: "Model Card for DistilBERT base multilingual (cased)"
- description_en: "Model Card for DistilBERT base multilingual (cased)"
- icon: ""
- from_repo: "https://huggingface.co/distilbert-base-multilingual-cased"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "wikipedia"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: ""
+ description: Model Card for DistilBERT base multilingual (cased)
+ description_en: Model Card for DistilBERT base multilingual (cased)
+ from_repo: https://huggingface.co/distilbert-base-multilingual-cased
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: distilbert-base-multilingual-cased
Paper:
- - title: 'DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter'
- url: 'http://arxiv.org/abs/1910.01108v4'
- - title: 'Quantifying the Carbon Emissions of Machine Learning'
- url: 'http://arxiv.org/abs/1910.09700v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter'
+ url: http://arxiv.org/abs/1910.01108v4
+- title: Quantifying the Carbon Emissions of Machine Learning
+ url: http://arxiv.org/abs/1910.09700v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/distilbert-base-multilingual-cased/introduction_cn.ipynb b/modelcenter/community/huggingface/distilbert-base-multilingual-cased/introduction_cn.ipynb
new file mode 100644
index 00000000..32ecf21b
--- /dev/null
+++ b/modelcenter/community/huggingface/distilbert-base-multilingual-cased/introduction_cn.ipynb
@@ -0,0 +1,143 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "922fd8e5",
+ "metadata": {},
+ "source": [
+ "# Model Card for DistilBERT base multilingual (cased)\n",
+ "\n",
+ "详细内容请看[Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a1024bec",
+ "metadata": {},
+ "source": [
+ "## Model Description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bcdfe024",
+ "metadata": {},
+ "source": [
+ "This model is a distilled version of the [BERT base multilingual model](https://huggingface.co/bert-base-multilingual-cased/). The code for the distillation process can be found [here](https://github.com/huggingface/transformers/tree/main/examples/research_projects/distillation). This model is cased: it does make a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5051aaa6",
+ "metadata": {},
+ "source": [
+ "The model is trained on the concatenation of Wikipedia in 104 different languages listed [here](https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages).\n",
+ "The model has 6 layers, 768 dimension and 12 heads, totalizing 134M parameters (compared to 177M parameters for mBERT-base).\n",
+ "On average, this model, referred to as DistilmBERT, is twice as fast as mBERT-base.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "cdddc273",
+ "metadata": {},
+ "source": [
+ "We encourage potential users of this model to check out the [BERT base multilingual model card](https://huggingface.co/bert-base-multilingual-cased) to learn more about usage, limitations and potential biases.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8eebedbf",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** Victor Sanh, Lysandre Debut, Julien Chaumond, Thomas Wolf (Hugging Face)\n",
+ "- **Model type:** Transformer-based language model\n",
+ "- **Language(s) (NLP):** 104 languages; see full list [here](https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages)\n",
+ "- **License:** Apache 2.0\n",
+ "- **Related Models:** [BERT base multilingual model](https://huggingface.co/bert-base-multilingual-cased)\n",
+ "- **Resources for more information:**\n",
+ "- [GitHub Repository](https://github.com/huggingface/transformers/blob/main/examples/research_projects/distillation/README.md)\n",
+ "- [Associated Paper](https://arxiv.org/abs/1910.01108)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e9f48c0b",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f4dde273",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "b940cddf",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"distilbert-base-multilingual-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7ab62874",
+ "metadata": {},
+ "source": [
+ "# Citation\n",
+ "\n",
+ "```\n",
+ "@article{Sanh2019DistilBERTAD,\n",
+ " title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},\n",
+ " author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf},\n",
+ " journal={ArXiv},\n",
+ " year={2019},\n",
+ " volume={abs/1910.01108}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8bdb4ee1",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于 https://huggingface.co/distilbert-base-multilingual-cased ，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/distilbert-base-multilingual-cased/introduction_en.ipynb b/modelcenter/community/huggingface/distilbert-base-multilingual-cased/introduction_en.ipynb
new file mode 100644
index 00000000..5fbfcda3
--- /dev/null
+++ b/modelcenter/community/huggingface/distilbert-base-multilingual-cased/introduction_en.ipynb
@@ -0,0 +1,121 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "4260a150",
+ "metadata": {},
+ "source": [
+ "# Model Card for DistilBERT base multilingual (cased)\n",
+ "\n",
+ "You can get more details from [Bert in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/bert/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "53f1b1c2",
+ "metadata": {},
+ "source": [
+ "This model is a distilled version of the [BERT base multilingual model](https://huggingface.co/bert-base-multilingual-cased/). The code for the distillation process can be found [here](https://github.com/huggingface/transformers/tree/main/examples/research_projects/distillation). This model is cased: it does make a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f417583b",
+ "metadata": {},
+ "source": [
+ "The model is trained on the concatenation of Wikipedia in 104 different languages listed [here](https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages).\n",
+ "The model has 6 layers, 768 dimension and 12 heads, totalizing 134M parameters (compared to 177M parameters for mBERT-base).\n",
+ "On average, this model, referred to as DistilmBERT, is twice as fast as mBERT-base.\n",
+ "\n",
+ "- **Developed by:** Victor Sanh, Lysandre Debut, Julien Chaumond, Thomas Wolf (Hugging Face)\n",
+ "- **Model type:** Transformer-based language model\n",
+ "- **Language(s) (NLP):** 104 languages; see full list [here](https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages)\n",
+ "- **License:** Apache 2.0\n",
+ "- **Related Models:** [BERT base multilingual model](https://huggingface.co/bert-base-multilingual-cased)\n",
+ "- **Resources for more information:**\n",
+ "- [GitHub Repository](https://github.com/huggingface/transformers/blob/main/examples/research_projects/distillation/README.md)\n",
+ "- [Associated Paper](https://arxiv.org/abs/1910.01108)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f47ce9b7",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "a1353b5f",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "e23a860f",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"distilbert-base-multilingual-cased\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "38c30ea4",
+ "metadata": {},
+ "source": [
+ "# Citation\n",
+ "\n",
+ "```\n",
+ "@article{Sanh2019DistilBERTAD,\n",
+ " title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},\n",
+ " author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf},\n",
+ " journal={ArXiv},\n",
+ " year={2019},\n",
+ " volume={abs/1910.01108}\n",
+ "}\n",
+ "```\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0ee03d6a",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/distilbert-base-multilingual-cased and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/distilgpt2/info.yaml b/modelcenter/community/huggingface/distilgpt2/info.yaml
index dcaa018e..070a0794 100644
--- a/modelcenter/community/huggingface/distilgpt2/info.yaml
+++ b/modelcenter/community/huggingface/distilgpt2/info.yaml
@@ -1,32 +1,30 @@
+Datasets: openwebtext
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "distilgpt2"
- description: "DistilGPT2"
- description_en: "DistilGPT2"
- icon: ""
- from_repo: "https://huggingface.co/distilgpt2"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Text Generation"
- sub_tag: "文本生成"
-
-Example:
-
-Datasets: "openwebtext"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: DistilGPT2
+ description_en: DistilGPT2
+ from_repo: https://huggingface.co/distilgpt2
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: distilgpt2
Paper:
- - title: 'DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter'
- url: 'http://arxiv.org/abs/1910.01108v4'
- - title: 'Can Model Compression Improve NLP Fairness'
- url: 'http://arxiv.org/abs/2201.08542v1'
- - title: 'Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal'
- url: 'http://arxiv.org/abs/2203.12574v1'
- - title: 'Quantifying the Carbon Emissions of Machine Learning'
- url: 'http://arxiv.org/abs/1910.09700v2'
- - title: 'Distilling the Knowledge in a Neural Network'
- url: 'http://arxiv.org/abs/1503.02531v1'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter'
+ url: http://arxiv.org/abs/1910.01108v4
+- title: Can Model Compression Improve NLP Fairness
+ url: http://arxiv.org/abs/2201.08542v1
+- title: Mitigating Gender Bias in Distilled Language Models via Counterfactual Role
+ Reversal
+ url: http://arxiv.org/abs/2203.12574v1
+- title: Quantifying the Carbon Emissions of Machine Learning
+ url: http://arxiv.org/abs/1910.09700v2
+- title: Distilling the Knowledge in a Neural Network
+ url: http://arxiv.org/abs/1503.02531v1
+Publisher: huggingface
+Task:
+- sub_tag: 文本生成
+ sub_tag_en: Text Generation
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/distilgpt2/introduction_cn.ipynb b/modelcenter/community/huggingface/distilgpt2/introduction_cn.ipynb
new file mode 100644
index 00000000..082df8e1
--- /dev/null
+++ b/modelcenter/community/huggingface/distilgpt2/introduction_cn.ipynb
@@ -0,0 +1,142 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "72047643",
+ "metadata": {},
+ "source": [
+ "# DistilGPT2\n",
+ "\n",
+ "详细内容请看[GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "20c299c9",
+ "metadata": {},
+ "source": [
+ "DistilGPT2 (short for Distilled-GPT2) is an English-language model pre-trained with the supervision of the smallest version of Generative Pre-trained Transformer 2 (GPT-2). Like GPT-2, DistilGPT2 can be used to generate text. Users of this model card should also consider information about the design, training, and limitations of GPT-2.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c624b3d1",
+ "metadata": {},
+ "source": [
+ "## Model Details\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "92002396",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** Hugging Face\n",
+ "- **Model type:** Transformer-based Language Model\n",
+ "- **Language:** English\n",
+ "- **License:** Apache 2.0\n",
+ "- **Model Description:** DistilGPT2 is an English-language model pre-trained with the supervision of the 124 million parameter version of GPT-2. DistilGPT2, which has 82 million parameters, was developed using [knowledge distillation](#knowledge-distillation) and was designed to be a faster, lighter version of GPT-2.\n",
+ "- **Resources for more information:** See this repository for more about Distil\\* (a class of compressed models including Distilled-GPT2), [Sanh et al. (2019)](https://arxiv.org/abs/1910.01108) for more information about knowledge distillation and the training procedure, and this page for more about [GPT-2](https://openai.com/blog/better-language-models/).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a1a84778",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f9c6043d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "a9f0754d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"distilgpt2\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "03d3d465",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@inproceedings{sanh2019distilbert,\n",
+ "title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},\n",
+ "author={Sanh, Victor and Debut, Lysandre and Chaumond, Julien and Wolf, Thomas},\n",
+ "booktitle={NeurIPS EMC^2 Workshop},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7966636a",
+ "metadata": {},
+ "source": [
+ "## Glossary\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "533038ef",
+ "metadata": {},
+ "source": [
+ "-\t**Knowledge Distillation**: As described in [Sanh et al. (2019)](https://arxiv.org/pdf/1910.01108.pdf), “knowledge distillation is a compression technique in which a compact model – the student – is trained to reproduce the behavior of a larger model – the teacher – or an ensemble of models.” Also see [Bucila et al. (2006)](https://www.cs.cornell.edu/~caruana/compression.kdd06.pdf) and [Hinton et al. (2015)](https://arxiv.org/abs/1503.02531).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a7ff7cc1",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> 此模型介绍及权重来源于[https://huggingface.co/distilgpt2](https://huggingface.co/distilgpt2)，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/distilgpt2/introduction_en.ipynb b/modelcenter/community/huggingface/distilgpt2/introduction_en.ipynb
new file mode 100644
index 00000000..d30a33f9
--- /dev/null
+++ b/modelcenter/community/huggingface/distilgpt2/introduction_en.ipynb
@@ -0,0 +1,134 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "1b34fb8a",
+ "metadata": {},
+ "source": [
+ "# DistilGPT2\n",
+ "\n",
+ "You can get more details from [GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f3ab8949",
+ "metadata": {},
+ "source": [
+ "DistilGPT2 (short for Distilled-GPT2) is an English-language model pre-trained with the supervision of the smallest version of Generative Pre-trained Transformer 2 (GPT-2). Like GPT-2, DistilGPT2 can be used to generate text. Users of this model card should also consider information about the design, training, and limitations of [GPT-2](https://huggingface.co/gpt2).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c6fbc1da",
+ "metadata": {},
+ "source": [
+ "## Model Details\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e2929e2f",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** Hugging Face\n",
+ "- **Model type:** Transformer-based Language Model\n",
+ "- **Language:** English\n",
+ "- **License:** Apache 2.0\n",
+ "- **Model Description:** DistilGPT2 is an English-language model pre-trained with the supervision of the 124 million parameter version of GPT-2. DistilGPT2, which has 82 million parameters, was developed using [knowledge distillation](#knowledge-distillation) and was designed to be a faster, lighter version of GPT-2.\n",
+ "- **Resources for more information:** See [this repository](https://github.com/huggingface/transformers/tree/main/examples/research_projects/distillation) for more about Distil\\* (a class of compressed models including Distilled-GPT2), [Sanh et al. (2019)](https://arxiv.org/abs/1910.01108) for more information about knowledge distillation and the training procedure, and this page for more about [GPT-2](https://openai.com/blog/better-language-models/).\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "5e226406",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "51f32d75",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"distilgpt2\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "adb84dc8",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@inproceedings{sanh2019distilbert,\n",
+ "title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},\n",
+ "author={Sanh, Victor and Debut, Lysandre and Chaumond, Julien and Wolf, Thomas},\n",
+ "booktitle={NeurIPS EMC^2 Workshop},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7d2aaec2",
+ "metadata": {},
+ "source": [
+ "## Glossary\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "004026dd",
+ "metadata": {},
+ "source": [
+ "-\t**Knowledge Distillation**: As described in [Sanh et al. (2019)](https://arxiv.org/pdf/1910.01108.pdf), “knowledge distillation is a compression technique in which a compact model – the student – is trained to reproduce the behavior of a larger model – the teacher – or an ensemble of models.” Also see [Bucila et al. (2006)](https://www.cs.cornell.edu/~caruana/compression.kdd06.pdf) and [Hinton et al. (2015)](https://arxiv.org/abs/1503.02531).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f8d12799",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> The model introduction and model weights originate from [https://huggingface.co/distilgpt2](https://huggingface.co/distilgpt2) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/distilroberta-base/info.yaml b/modelcenter/community/huggingface/distilroberta-base/info.yaml
index eceb32f2..dd53f5e4 100644
--- a/modelcenter/community/huggingface/distilroberta-base/info.yaml
+++ b/modelcenter/community/huggingface/distilroberta-base/info.yaml
@@ -1,26 +1,23 @@
+Datasets: openwebtext
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: apache-2.0
Model_Info:
- name: "distilroberta-base"
- description: "Model Card for DistilRoBERTa base"
- description_en: "Model Card for DistilRoBERTa base"
- icon: ""
- from_repo: "https://huggingface.co/distilroberta-base"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Fill-Mask"
- sub_tag: "槽位填充"
-
-Example:
-
-Datasets: "openwebtext"
-Publisher: "huggingface"
-License: "apache-2.0"
-Language: "English"
+ description: Model Card for DistilRoBERTa base
+ description_en: Model Card for DistilRoBERTa base
+ from_repo: https://huggingface.co/distilroberta-base
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: distilroberta-base
Paper:
- - title: 'DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter'
- url: 'http://arxiv.org/abs/1910.01108v4'
- - title: 'Quantifying the Carbon Emissions of Machine Learning'
- url: 'http://arxiv.org/abs/1910.09700v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter'
+ url: http://arxiv.org/abs/1910.01108v4
+- title: Quantifying the Carbon Emissions of Machine Learning
+ url: http://arxiv.org/abs/1910.09700v2
+Publisher: huggingface
+Task:
+- sub_tag: 槽位填充
+ sub_tag_en: Fill-Mask
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/distilroberta-base/introduction_cn.ipynb b/modelcenter/community/huggingface/distilroberta-base/introduction_cn.ipynb
new file mode 100644
index 00000000..db6d2867
--- /dev/null
+++ b/modelcenter/community/huggingface/distilroberta-base/introduction_cn.ipynb
@@ -0,0 +1,126 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "7f49bb4b",
+ "metadata": {},
+ "source": [
+ "# Model Card for DistilRoBERTa base\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "88c832ab",
+ "metadata": {},
+ "source": [
+ "## Model Description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3a2333a1",
+ "metadata": {},
+ "source": [
+ "This model is a distilled version of the RoBERTa-base model. It follows the same training procedure as DistilBERT.\n",
+ "The code for the distillation process can be found [here](https://github.com/huggingface/transformers/tree/master/examples/distillation).\n",
+ "This model is case-sensitive: it makes a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9ac70255",
+ "metadata": {},
+ "source": [
+ "The model has 6 layers, 768 dimension and 12 heads, totalizing 82M parameters (compared to 125M parameters for RoBERTa-base).\n",
+ "On average DistilRoBERTa is twice as fast as Roberta-base.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a0757c23",
+ "metadata": {},
+ "source": [
+ "We encourage users of this model card to check out the RoBERTa-base model card to learn more about usage, limitations and potential biases.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2865466d",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** Victor Sanh, Lysandre Debut, Julien Chaumond, Thomas Wolf (Hugging Face)\n",
+ "- **Model type:** Transformer-based language model\n",
+ "- **Language(s) (NLP):** English\n",
+ "- **License:** Apache 2.0\n",
+ "- **Related Models:** RoBERTa-base model card\n",
+ "- [Associated Paper](https://arxiv.org/abs/1910.01108)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a204fad3",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "b2e488ed",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "43d7726b",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"distilroberta-base\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e30fb0eb",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> 此模型介绍及权重来源于[https://huggingface.co/distilroberta-base](https://huggingface.co/distilroberta-base)，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/distilroberta-base/introduction_en.ipynb b/modelcenter/community/huggingface/distilroberta-base/introduction_en.ipynb
new file mode 100644
index 00000000..588b1d0b
--- /dev/null
+++ b/modelcenter/community/huggingface/distilroberta-base/introduction_en.ipynb
@@ -0,0 +1,128 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "4bd898ca",
+ "metadata": {},
+ "source": [
+ "# Model Card for DistilRoBERTa base\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7d39a086",
+ "metadata": {},
+ "source": [
+ "## Model Description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e2043d14",
+ "metadata": {},
+ "source": [
+ "This model is a distilled version of the [RoBERTa-base model](https://huggingface.co/roberta-base). It follows the same training procedure as [DistilBERT](https://huggingface.co/distilbert-base-uncased).\n",
+ "The code for the distillation process can be found [here](https://github.com/huggingface/transformers/tree/master/examples/distillation).\n",
+ "This model is case-sensitive: it makes a difference between english and English.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "10aefe84",
+ "metadata": {},
+ "source": [
+ "The model has 6 layers, 768 dimension and 12 heads, totalizing 82M parameters (compared to 125M parameters for RoBERTa-base).\n",
+ "On average DistilRoBERTa is twice as fast as Roberta-base.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d7ebd775",
+ "metadata": {},
+ "source": [
+ "We encourage users of this model card to check out the [RoBERTa-base model card](https://huggingface.co/roberta-base) to learn more about usage, limitations and potential biases.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "423d28b1",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** Victor Sanh, Lysandre Debut, Julien Chaumond, Thomas Wolf (Hugging Face)\n",
+ "- **Model type:** Transformer-based language model\n",
+ "- **Language(s) (NLP):** English\n",
+ "- **License:** Apache 2.0\n",
+ "- **Related Models:** [RoBERTa-base model card](https://huggingface.co/roberta-base)\n",
+ "- **Resources for more information:**\n",
+ "- [GitHub Repository](https://github.com/huggingface/transformers/blob/main/examples/research_projects/distillation/README.md)\n",
+ "- [Associated Paper](https://arxiv.org/abs/1910.01108)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "715b4360",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "9ad9b1a9",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "94e4d093",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"distilroberta-base\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e258a20c",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> The model introduction and model weights originate from [https://huggingface.co/distilroberta-base](https://huggingface.co/distilroberta-base) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/gpt2-large/info.yaml b/modelcenter/community/huggingface/gpt2-large/info.yaml
index 5877d17d..b158fe17 100644
--- a/modelcenter/community/huggingface/gpt2-large/info.yaml
+++ b/modelcenter/community/huggingface/gpt2-large/info.yaml
@@ -1,24 +1,21 @@
+Datasets: ''
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: mit
Model_Info:
- name: "gpt2-large"
- description: "GPT-2 Large"
- description_en: "GPT-2 Large"
- icon: ""
- from_repo: "https://huggingface.co/gpt2-large"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Text Generation"
- sub_tag: "文本生成"
-
-Example:
-
-Datasets: ""
-Publisher: "huggingface"
-License: "mit"
-Language: "English"
+ description: GPT-2 Large
+ description_en: GPT-2 Large
+ from_repo: https://huggingface.co/gpt2-large
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: gpt2-large
Paper:
- - title: 'Quantifying the Carbon Emissions of Machine Learning'
- url: 'http://arxiv.org/abs/1910.09700v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: Quantifying the Carbon Emissions of Machine Learning
+ url: http://arxiv.org/abs/1910.09700v2
+Publisher: huggingface
+Task:
+- sub_tag: 文本生成
+ sub_tag_en: Text Generation
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/gpt2-large/introduction_cn.ipynb b/modelcenter/community/huggingface/gpt2-large/introduction_cn.ipynb
new file mode 100644
index 00000000..0a394483
--- /dev/null
+++ b/modelcenter/community/huggingface/gpt2-large/introduction_cn.ipynb
@@ -0,0 +1,158 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "32b8730a",
+ "metadata": {},
+ "source": [
+ "# GPT-2 Large\n",
+ "\n",
+ "详细内容请看[GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "de66cac3",
+ "metadata": {},
+ "source": [
+ "## Table of Contents\n",
+ "- [Model Details](#model-details)\n",
+ "- [How To Get Started With the Model](#how-to-get-started-with-the-model)\n",
+ "- [Uses](#uses)\n",
+ "- [Risks, Limitations and Biases](#risks-limitations-and-biases)\n",
+ "- [Training](#training)\n",
+ "- [Evaluation](#evaluation)\n",
+ "- [Environmental Impact](#environmental-impact)\n",
+ "- [Technical Specifications](#technical-specifications)\n",
+ "- [Citation Information](#citation-information)\n",
+ "- [Model Card Authors](#model-card-author)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8afa58ef",
+ "metadata": {},
+ "source": [
+ "## Model Details\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e4e46496",
+ "metadata": {},
+ "source": [
+ "**Model Description:** GPT-2 Large is the **774M parameter** version of GPT-2, a transformer-based language model created and released by OpenAI. The model is a pretrained model on English language using a causal language modeling (CLM) objective.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "15b8f634",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** OpenAI, see [associated research paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) and [GitHub repo](https://github.com/openai/gpt-2) for model developers.\n",
+ "- **Model Type:** Transformer-based language model\n",
+ "- **Language(s):** English\n",
+ "- **License:** [Modified MIT License](https://github.com/openai/gpt-2/blob/master/LICENSE)\n",
+ "- **Related Models:** [GPT-2](https://huggingface.co/gpt2), [GPT-Medium](https://huggingface.co/gpt2-medium) and [GPT-XL](https://huggingface.co/gpt2-xl)\n",
+ "- **Resources for more information:**\n",
+ "- [Research Paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\n",
+ "- [OpenAI Blog Post](https://openai.com/blog/better-language-models/)\n",
+ "- [GitHub Repo](https://github.com/openai/gpt-2)\n",
+ "- [OpenAI Model Card for GPT-2](https://github.com/openai/gpt-2/blob/master/model_card.md)\n",
+ "- Test the full generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6c2023d9",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "b17e6efb",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "33c1f565",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"gpt2-large\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8060d283",
+ "metadata": {},
+ "source": [
+ "## Citatioin\n",
+ "\n",
+ "```\n",
+ "@article{radford2019language,\n",
+ "title={Language models are unsupervised multitask learners},\n",
+ "author={Radford, Alec and Wu, Jeffrey and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya and others},\n",
+ "journal={OpenAI blog},\n",
+ "volume={1},\n",
+ "number={8},\n",
+ "pages={9},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "083f0d9c",
+ "metadata": {},
+ "source": [
+ "## Model Card Authors\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f9e4bb43",
+ "metadata": {},
+ "source": [
+ "This model card was written by the Hugging Face team.\n",
+ "\n",
+ "> 此模型介绍及权重来源于[https://huggingface.co/gpt2-large](https://huggingface.co/gpt2-large)，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/gpt2-large/introduction_en.ipynb b/modelcenter/community/huggingface/gpt2-large/introduction_en.ipynb
new file mode 100644
index 00000000..e3dce42d
--- /dev/null
+++ b/modelcenter/community/huggingface/gpt2-large/introduction_en.ipynb
@@ -0,0 +1,157 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "dc26013b",
+ "metadata": {},
+ "source": [
+ "# GPT-2 Large\n",
+ "\n",
+ "You can get more details from [GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "38e29e37",
+ "metadata": {},
+ "source": [
+ "## Table of Contents\n",
+ "- [Model Details](#model-details)\n",
+ "- [How To Get Started With the Model](#how-to-get-started-with-the-model)\n",
+ "- [Uses](#uses)\n",
+ "- [Risks, Limitations and Biases](#risks-limitations-and-biases)\n",
+ "- [Training](#training)\n",
+ "- [Evaluation](#evaluation)\n",
+ "- [Environmental Impact](#environmental-impact)\n",
+ "- [Technical Specifications](#technical-specifications)\n",
+ "- [Citation Information](#citation-information)\n",
+ "- [Model Card Authors](#model-card-author)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "590c3fbd",
+ "metadata": {},
+ "source": [
+ "## Model Details\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1a2cd621",
+ "metadata": {},
+ "source": [
+ "**Model Description:** GPT-2 Large is the **774M parameter** version of GPT-2, a transformer-based language model created and released by OpenAI. The model is a pretrained model on English language using a causal language modeling (CLM) objective.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0155f43f",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** OpenAI, see [associated research paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) and [GitHub repo](https://github.com/openai/gpt-2) for model developers.\n",
+ "- **Model Type:** Transformer-based language model\n",
+ "- **Language(s):** English\n",
+ "- **License:** [Modified MIT License](https://github.com/openai/gpt-2/blob/master/LICENSE)\n",
+ "- **Related Models:** https://huggingface.co/gpt2, GPT-Medium and GPT-XL\n",
+ "- **Resources for more information:**\n",
+ "- [Research Paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\n",
+ "- [OpenAI Blog Post](https://openai.com/blog/better-language-models/)\n",
+ "- [GitHub Repo](https://github.com/openai/gpt-2)\n",
+ "- [OpenAI Model Card for GPT-2](https://github.com/openai/gpt-2/blob/master/model_card.md)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "18e2772d",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "30207821",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "2ae65fe6",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"gpt2-large\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e8b7c92b",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@article{radford2019language,\n",
+ "title={Language models are unsupervised multitask learners},\n",
+ "author={Radford, Alec and Wu, Jeffrey and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya and others},\n",
+ "journal={OpenAI blog},\n",
+ "volume={1},\n",
+ "number={8},\n",
+ "pages={9},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7cded70d",
+ "metadata": {},
+ "source": [
+ "## Model Card Authors\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ff9ab2d4",
+ "metadata": {},
+ "source": [
+ "This model card was written by the Hugging Face team.\n",
+ "\n",
+ "> The model introduction and model weights originate from [https://huggingface.co/gpt2-large](https://huggingface.co/gpt2-large) and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/gpt2-medium/info.yaml b/modelcenter/community/huggingface/gpt2-medium/info.yaml
index 2b8d53c8..3e03ccdf 100644
--- a/modelcenter/community/huggingface/gpt2-medium/info.yaml
+++ b/modelcenter/community/huggingface/gpt2-medium/info.yaml
@@ -1,24 +1,21 @@
+Datasets: ''
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: mit
Model_Info:
- name: "gpt2-medium"
- description: "GPT-2 Medium"
- description_en: "GPT-2 Medium"
- icon: ""
- from_repo: "https://huggingface.co/gpt2-medium"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Text Generation"
- sub_tag: "文本生成"
-
-Example:
-
-Datasets: ""
-Publisher: "huggingface"
-License: "mit"
-Language: "English"
+ description: GPT-2 Medium
+ description_en: GPT-2 Medium
+ from_repo: https://huggingface.co/gpt2-medium
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: gpt2-medium
Paper:
- - title: 'Quantifying the Carbon Emissions of Machine Learning'
- url: 'http://arxiv.org/abs/1910.09700v2'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: Quantifying the Carbon Emissions of Machine Learning
+ url: http://arxiv.org/abs/1910.09700v2
+Publisher: huggingface
+Task:
+- sub_tag: 文本生成
+ sub_tag_en: Text Generation
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/gpt2-medium/introduction_cn.ipynb b/modelcenter/community/huggingface/gpt2-medium/introduction_cn.ipynb
new file mode 100644
index 00000000..cfd546c3
--- /dev/null
+++ b/modelcenter/community/huggingface/gpt2-medium/introduction_cn.ipynb
@@ -0,0 +1,140 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "25324e9c",
+ "metadata": {},
+ "source": [
+ "# GPT-2 Medium\n",
+ "\n",
+ "详细内容请看[GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "806177e3",
+ "metadata": {},
+ "source": [
+ "## Model Details\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dbcaecb0",
+ "metadata": {},
+ "source": [
+ "**Model Description:** GPT-2 Medium is the **355M parameter** version of GPT-2, a transformer-based language model created and released by OpenAI. The model is a pretrained model on English language using a causal language modeling (CLM) objective.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ab73e9f0",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** OpenAI, see [associated research paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) and [GitHub repo](https://github.com/openai/gpt-2) for model developers.\n",
+ "- **Model Type:** Transformer-based language model\n",
+ "- **Language(s):** English\n",
+ "- **License:** [Modified MIT License](https://github.com/openai/gpt-2/blob/master/LICENSE)\n",
+ "- **Related Models:** [GPT2](https://huggingface.co/gpt2), [GPT2-Large](https://huggingface.co/gpt2-large) and [GPT2-XL](https://huggingface.co/gpt2-xl)\n",
+ "- **Resources for more information:**\n",
+ "- [Research Paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\n",
+ "- [OpenAI Blog Post](https://openai.com/blog/better-language-models/)\n",
+ "- [GitHub Repo](https://github.com/openai/gpt-2)\n",
+ "- [OpenAI Model Card for GPT-2](https://github.com/openai/gpt-2/blob/master/model_card.md)\n",
+ "- Test the full generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "70c3fd36",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "1bae5ee0",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "11b32577",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"gpt2-medium\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "08f90ea0",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@article{radford2019language,\n",
+ "title={Language models are unsupervised multitask learners},\n",
+ "author={Radford, Alec and Wu, Jeffrey and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya and others},\n",
+ "journal={OpenAI blog},\n",
+ "volume={1},\n",
+ "number={8},\n",
+ "pages={9},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "64d79312",
+ "metadata": {},
+ "source": [
+ "## Model Card Authors\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d14dd2ac",
+ "metadata": {},
+ "source": [
+ "This model card was written by the Hugging Face team.\n",
+ "\n",
+ "> 此模型介绍及权重来源于 https://huggingface.co/gpt2-medium ，并转换为飞桨模型格式。"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/gpt2-medium/introduction_en.ipynb b/modelcenter/community/huggingface/gpt2-medium/introduction_en.ipynb
new file mode 100644
index 00000000..a324c5b9
--- /dev/null
+++ b/modelcenter/community/huggingface/gpt2-medium/introduction_en.ipynb
@@ -0,0 +1,146 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "46995787",
+ "metadata": {},
+ "source": [
+ "# GPT-2 Medium\n",
+ "\n",
+ "You can get more details from [GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f695ad73",
+ "metadata": {},
+ "source": [
+ "## Model Details\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "5a8170d9",
+ "metadata": {},
+ "source": [
+ "**Model Description:** GPT-2 Medium is the **355M parameter** version of GPT-2, a transformer-based language model created and released by OpenAI. The model is a pretrained model on English language using a causal language modeling (CLM) objective.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1d0dc244",
+ "metadata": {},
+ "source": [
+ "- **Developed by:** OpenAI, see [associated research paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) and [GitHub repo](https://github.com/openai/gpt-2) for model developers.\n",
+ "- **Model Type:** Transformer-based language model\n",
+ "- **Language(s):** English\n",
+ "- **License:** [Modified MIT License](https://github.com/openai/gpt-2/blob/master/LICENSE)\n",
+ "- **Related Models:** [GPT2](https://huggingface.co/gpt2), [GPT2-Large](https://huggingface.co/gpt2-large) and [GPT2-XL](https://huggingface.co/gpt2-xl)\n",
+ "- **Resources for more information:**\n",
+ "- [Research Paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\n",
+ "- [OpenAI Blog Post](https://openai.com/blog/better-language-models/)\n",
+ "- [GitHub Repo](https://github.com/openai/gpt-2)\n",
+ "- [OpenAI Model Card for GPT-2](https://github.com/openai/gpt-2/blob/master/model_card.md)\n",
+ "- Test the full generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "adc5a3f9",
+ "metadata": {},
+ "source": [
+ "## How to Get Started with the Model\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7566eafd",
+ "metadata": {},
+ "source": [
+ "Use the code below to get started with the model. \n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "ab4c71ee",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "b0167528",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"gpt2-medium\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "52cdcf9e",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@article{radford2019language,\n",
+ "title={Language models are unsupervised multitask learners},\n",
+ "author={Radford, Alec and Wu, Jeffrey and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya and others},\n",
+ "journal={OpenAI blog},\n",
+ "volume={1},\n",
+ "number={8},\n",
+ "pages={9},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "eb327c10",
+ "metadata": {},
+ "source": [
+ "## Model Card Authors\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "50fb7de8",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from https://huggingface.co/gpt2-medium and were converted to PaddlePaddle format for ease of use in PaddleNLP."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/gpt2/info.yaml b/modelcenter/community/huggingface/gpt2/info.yaml
index 6663ffdf..ce027f72 100644
--- a/modelcenter/community/huggingface/gpt2/info.yaml
+++ b/modelcenter/community/huggingface/gpt2/info.yaml
@@ -1,23 +1,19 @@
+Datasets: ''
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: English
+License: mit
Model_Info:
- name: "gpt2"
- description: "GPT-2"
- description_en: "GPT-2"
- icon: ""
- from_repo: "https://huggingface.co/gpt2"
-
+ description: GPT-2
+ description_en: GPT-2
+ from_repo: https://huggingface.co/gpt2
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: gpt2
+Paper: null
+Publisher: huggingface
Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Text Generation"
- sub_tag: "文本生成"
-
-Example:
-
-Datasets: ""
-Publisher: "huggingface"
-License: "mit"
-Language: "English"
-Paper:
-
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- sub_tag: 文本生成
+ sub_tag_en: Text Generation
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/huggingface/gpt2/introduction_cn.ipynb b/modelcenter/community/huggingface/gpt2/introduction_cn.ipynb
new file mode 100644
index 00000000..a7849400
--- /dev/null
+++ b/modelcenter/community/huggingface/gpt2/introduction_cn.ipynb
@@ -0,0 +1,160 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "a4cd103f",
+ "metadata": {},
+ "source": [
+ "# GPT-2\n",
+ "\n",
+ "详细内容请看[GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)。"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e10dfe6d",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in\n",
+ "[this paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\n",
+ "and first released at [this page](https://openai.com/blog/better-language-models/).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d1b13043",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing GPT-2 also wrote a\n",
+ "[model card](https://github.com/openai/gpt-2/blob/master/model_card.md) for their model. Content from this model card\n",
+ "has been written by the Hugging Face team to complete the information they provided and give specific examples of bias.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "016271a5",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e3a53155",
+ "metadata": {},
+ "source": [
+ "GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This\n",
+ "means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots\n",
+ "of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely,\n",
+ "it was trained to guess the next word in sentences.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6836ad17",
+ "metadata": {},
+ "source": [
+ "More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence,\n",
+ "shifted one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the\n",
+ "predictions for the token `i` only uses the inputs from `1` to `i` but not the future tokens.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "26946ce6",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks. The model is best at what it was pretrained for however, which is generating texts from a\n",
+ "prompt.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "571b41cf",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "a6233e8e",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "2e906136",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"gpt2\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "78f26b7f",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@article{radford2019language,\n",
+ "title={Language Models are Unsupervised Multitask Learners},\n",
+ "author={Radford, Alec and Wu, Jeff and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2f646c57",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> 此模型介绍及权重来源于[https://huggingface.co/gpt2](https://huggingface.co/gpt2)，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ },
+ "vscode": {
+ "interpreter": {
+ "hash": "606ea184b8fed3419d714b545dc1784fad6c99d0cc940b6b9d787dccf225faa5"
+ }
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/huggingface/gpt2/introduction_en.ipynb b/modelcenter/community/huggingface/gpt2/introduction_en.ipynb
new file mode 100644
index 00000000..fe5a7baa
--- /dev/null
+++ b/modelcenter/community/huggingface/gpt2/introduction_en.ipynb
@@ -0,0 +1,164 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "2d373572",
+ "metadata": {},
+ "source": [
+ "# GPT-2\n",
+ "\n",
+ "You can get more details from [GPT2 in PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP/blob/develop/model_zoo/gpt/README.md)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "00be5831",
+ "metadata": {},
+ "source": [
+ "Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b5857cc2",
+ "metadata": {},
+ "source": [
+ "Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in\n",
+ "[this paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)\n",
+ "and first released at [this page](https://openai.com/blog/better-language-models/).\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b0abac76",
+ "metadata": {},
+ "source": [
+ "Disclaimer: The team releasing GPT-2 also wrote a\n",
+ "[model card](https://github.com/openai/gpt-2/blob/master/model_card.md) for their model. Content from this model card\n",
+ "has been written by the Hugging Face team to complete the information they provided and give specific examples of bias.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "fa2c7f4b",
+ "metadata": {},
+ "source": [
+ "## Model description\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "294521bd",
+ "metadata": {},
+ "source": [
+ "GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This\n",
+ "means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots\n",
+ "of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely,\n",
+ "it was trained to guess the next word in sentences.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b1204c32",
+ "metadata": {},
+ "source": [
+ "More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence,\n",
+ "shifted one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the\n",
+ "predictions for the token `i` only uses the inputs from `1` to `i` but not the future tokens.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a019cc9e",
+ "metadata": {},
+ "source": [
+ "This way, the model learns an inner representation of the English language that can then be used to extract features\n",
+ "useful for downstream tasks. The model is best at what it was pretrained for however, which is generating texts from a\n",
+ "prompt.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "54ae8500",
+ "metadata": {},
+ "source": [
+ "## How to use"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "d33fddda",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "d0e160c6",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"gpt2\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "fcb8a843",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@article{radford2019language,\n",
+ "title={Language Models are Unsupervised Multitask Learners},\n",
+ "author={Radford, Alec and Wu, Jeff and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya},\n",
+ "year={2019}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "513848f8",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "> The model introduction and model weights originate from [https://huggingface.co/gpt2](https://huggingface.co/gpt2) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/indobenchmark/indobert-base-p1/info.yaml b/modelcenter/community/indobenchmark/indobert-base-p1/info.yaml
index a27d7a18..4bf70a48 100644
--- a/modelcenter/community/indobenchmark/indobert-base-p1/info.yaml
+++ b/modelcenter/community/indobenchmark/indobert-base-p1/info.yaml
@@ -1,24 +1,22 @@
+Datasets: indonlu
+Example: null
+IfOnlineDemo: 0
+IfTraining: 0
+Language: Indonesian
+License: mit
Model_Info:
- name: "indobenchmark/indobert-base-p1"
- description: "IndoBERT Base Model (phase1 - uncased)"
- description_en: "IndoBERT Base Model (phase1 - uncased)"
- icon: ""
- from_repo: "https://huggingface.co/indobenchmark/indobert-base-p1"
-
-Task:
-- tag_en: "Natural Language Processing"
- tag: "自然语言处理"
- sub_tag_en: "Feature Extraction"
- sub_tag: "特征抽取"
-
-Example:
-
-Datasets: "indonlu"
-Publisher: "indobenchmark"
-License: "mit"
-Language: "Indonesian"
+ description: IndoBERT Base Model (phase1 - uncased)
+ description_en: IndoBERT Base Model (phase1 - uncased)
+ from_repo: https://huggingface.co/indobenchmark/indobert-base-p1
+ icon: https://paddlenlp.bj.bcebos.com/models/community/transformer-layer.png
+ name: indobenchmark/indobert-base-p1
Paper:
- - title: 'IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding'
- url: 'http://arxiv.org/abs/2009.05387v3'
-IfTraining: 0
-IfOnlineDemo: 0
\ No newline at end of file
+- title: 'IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language
+ Understanding'
+ url: http://arxiv.org/abs/2009.05387v3
+Publisher: indobenchmark
+Task:
+- sub_tag: 特征抽取
+ sub_tag_en: Feature Extraction
+ tag: 自然语言处理
+ tag_en: Natural Language Processing
diff --git a/modelcenter/community/indobenchmark/indobert-base-p1/introduction_cn.ipynb b/modelcenter/community/indobenchmark/indobert-base-p1/introduction_cn.ipynb
new file mode 100644
index 00000000..cc12f561
--- /dev/null
+++ b/modelcenter/community/indobenchmark/indobert-base-p1/introduction_cn.ipynb
@@ -0,0 +1,124 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "3f5a12e4",
+ "metadata": {},
+ "source": [
+ "# IndoBERT Base Model (phase1 - uncased)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e2fcac01",
+ "metadata": {},
+ "source": [
+ "[IndoBERT](https://arxiv.org/abs/2009.05387) is a state-of-the-art language model for Indonesian based on the BERT model. The pretrained model is trained using a masked language modeling (MLM) objective and next sentence prediction (NSP) objective.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6a9d6a02",
+ "metadata": {},
+ "source": [
+ "## All Pre-trained Models\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3020975b",
+ "metadata": {},
+ "source": [
+ "| Model | #params | Arch. | Training data |\n",
+ "|--------------------------------|--------------------------------|-------|-----------------------------------|\n",
+ "| `indobenchmark/indobert-base-p1` | 124.5M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-base-p2` | 124.5M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-large-p1` | 335.2M | Large | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-large-p2` | 335.2M | Large | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-base-p1` | 11.7M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-base-p2` | 11.7M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-large-p1` | 17.7M | Large | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-large-p2` | 17.7M | Large | Indo4B (23.43 GB of text) |\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d0e3771a",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f1f38760",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "a11bc38f",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"indobenchmark/indobert-base-p1\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d1fe4366",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@inproceedings{wilie2020indonlu,\n",
+ "title={IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding},\n",
+ "author={Bryan Wilie and Karissa Vincentio and Genta Indra Winata and Samuel Cahyawijaya and X. Li and Zhi Yuan Lim and S. Soleman and R. Mahendra and Pascale Fung and Syafri Bahar and A. Purwarianti},\n",
+ "booktitle={Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing},\n",
+ "year={2020}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "95f83dc9",
+ "metadata": {},
+ "source": [
+ "> 此模型介绍及权重来源于[https://huggingface.co/indobenchmark/indobert-base-p1](https://huggingface.co/indobenchmark/indobert-base-p1)，并转换为飞桨模型格式。\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/indobenchmark/indobert-base-p1/introduction_en.ipynb b/modelcenter/community/indobenchmark/indobert-base-p1/introduction_en.ipynb
new file mode 100644
index 00000000..ec38f774
--- /dev/null
+++ b/modelcenter/community/indobenchmark/indobert-base-p1/introduction_en.ipynb
@@ -0,0 +1,124 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "d6793868",
+ "metadata": {},
+ "source": [
+ "# IndoBERT Base Model (phase1 - uncased)\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "48b35590",
+ "metadata": {},
+ "source": [
+ "[IndoBERT](https://arxiv.org/abs/2009.05387) is a state-of-the-art language model for Indonesian based on the BERT model. The pretrained model is trained using a masked language modeling (MLM) objective and next sentence prediction (NSP) objective.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e5dc323c",
+ "metadata": {},
+ "source": [
+ "## All Pre-trained Models\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7db5d6e5",
+ "metadata": {},
+ "source": [
+ "| Model | #params | Arch. | Training data |\n",
+ "|--------------------------------|--------------------------------|-------|-----------------------------------|\n",
+ "| `indobenchmark/indobert-base-p1` | 124.5M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-base-p2` | 124.5M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-large-p1` | 335.2M | Large | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-large-p2` | 335.2M | Large | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-base-p1` | 11.7M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-base-p2` | 11.7M | Base | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-large-p1` | 17.7M | Large | Indo4B (23.43 GB of text) |\n",
+ "| `indobenchmark/indobert-lite-large-p2` | 17.7M | Large | Indo4B (23.43 GB of text) |\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "fc8827fd",
+ "metadata": {},
+ "source": [
+ "## How to use\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "e5b6e205",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "!pip install --upgrade paddlenlp"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "6701163d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import paddle\n",
+ "from paddlenlp.transformers import AutoModel\n",
+ "\n",
+ "model = AutoModel.from_pretrained(\"indobenchmark/indobert-base-p1\")\n",
+ "input_ids = paddle.randint(100, 200, shape=[1, 20])\n",
+ "print(model(input_ids))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "fb28cf5b",
+ "metadata": {},
+ "source": [
+ "## Citation\n",
+ "\n",
+ "```\n",
+ "@inproceedings{wilie2020indonlu,\n",
+ "title={IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding},\n",
+ "author={Bryan Wilie and Karissa Vincentio and Genta Indra Winata and Samuel Cahyawijaya and X. Li and Zhi Yuan Lim and S. Soleman and R. Mahendra and Pascale Fung and Syafri Bahar and A. Purwarianti},\n",
+ "booktitle={Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing},\n",
+ "year={2020}\n",
+ "}\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e155d1ce",
+ "metadata": {},
+ "source": [
+ "> The model introduction and model weights originate from [https://huggingface.co/indobenchmark/indobert-base-p1](https://huggingface.co/indobenchmark/indobert-base-p1) and were converted to PaddlePaddle format for ease of use in PaddleNLP.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.13"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/modelcenter/community/johngiorgi/declutr-base/download_cn.md b/modelcenter/community/johngiorgi/declutr-base/download_cn.md
deleted file mode 100644
index 90761ce3..00000000
--- a/modelcenter/community/johngiorgi/declutr-base/download_cn.md
+++ /dev/null
@@ -1,23 +0,0 @@
-# 模型列表
-
-## johngiorgi/declutr-base
-
-| 模型名称 | 模型介绍 | 模型大小 | 模型下载 |
-| --- | --- | --- | --- |
-|johngiorgi/declutr-base| | 625.22MB | [merges.txt](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/merges.txt)[model_config.json](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/tokenizer_config.json)
[vocab.json](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/vocab.json)
[vocab.txt](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/vocab.txt) | - -也可以通过`paddlenlp` cli 工具来下载对应的模型权重，使用步骤如下所示： - -* 安装paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* 下载命令行 - -```shell -paddlenlp download --cache-dir ./pretrained_models johngiorgi/declutr-base -``` - -有任何下载的问题都可以到[PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP)中发Issue提问。 \ No newline at end of file diff --git a/modelcenter/community/johngiorgi/declutr-base/download_en.md b/modelcenter/community/johngiorgi/declutr-base/download_en.md deleted file mode 100644 index f25c8efd..00000000 --- a/modelcenter/community/johngiorgi/declutr-base/download_en.md +++ /dev/null @@ -1,23 +0,0 @@ -# model list - -## - -| model | description | model_size | download | -| --- | --- | --- | --- | -|johngiorgi/declutr-base| | 625.22MB | [merges.txt](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/merges.txt)
[model_config.json](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/model_config.json)
[model_state.pdparams](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/model_state.pdparams)
[tokenizer_config.json](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/tokenizer_config.json)
[vocab.json](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/vocab.json)
[vocab.txt](https://bj.bcebos.com/paddlenlp/models/community/johngiorgi/declutr-base/vocab.txt) | - -or you can download all of model file with the following steps: - -* install paddlenlp - -```shell -pip install --upgrade paddlenlp -``` - -* download model with cli tool - -```shell -paddlenlp download --cache-dir ./pretrained_models johngiorgi/declutr-base -``` - -If you have any problems with it, you can post issue on [PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP) to get support. diff --git a/modelcenter/community/johngiorgi/declutr-base/info.yaml b/modelcenter/community/johngiorgi/declutr-base/info.yaml deleted file mode 100644 index 86e42194..00000000 --- a/modelcenter/community/johngiorgi/declutr-base/info.yaml +++ /dev/null @@ -1,28 +0,0 @@ -Model_Info: - name: "johngiorgi/declutr-base" - description: "DeCLUTR-base" - description_en: "DeCLUTR-base" - icon: "" - from_repo: "https://huggingface.co/johngiorgi/declutr-base" - -Task: -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Sentence Similarity" - sub_tag: "句子相似度" -- tag_en: "Natural Language Processing" - tag: "自然语言处理" - sub_tag_en: "Feature Extraction" - sub_tag: "特征抽取" - -Example: - -Datasets: "openwebtext" -Publisher: "johngiorgi" -License: "apache-2.0" -Language: "English" -Paper: - - title: 'DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations' - url: 'http://arxiv.org/abs/2006.03659v4' -IfTraining: 0 -IfOnlineDemo: 0 \ No newline at end of file -- GitLab