Skip to content

P
PaddleHub

PaddlePaddle / PaddleHub
大约 2 年 前同步成功

通知 285
Star 12117
Fork 2091
P
PaddleHub
“a6e8731ea96868d084b1a640b76f981bd90a3482”上不存在“mobile/src/operators/kernel/arm/transpose_kernel.cpp”
未验证 提交 2b17b1f8 编写于 作者: D Daniel Yang 提交者： GitHub
浏览文件
操作

update readme and add the docs_en

上级 e5cb6178
显示空白变更内容
内联 并排
Showing with 3860 addition and 59 deletion
README.md
浏览文件 @ 2b17b1f8
...@@ -2,7 +2,6 @@ ...@@ -2,7 +2,6 @@
<img src="./docs/imgs/paddlehub_logo.jpg" align="middle" <img src="./docs/imgs/paddlehub_logo.jpg" align="middle"
</p> </p>
------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------
[![License](https://img.shields.io/badge/license-Apache%202-red.svg)](LICENSE) [![License](https://img.shields.io/badge/license-Apache%202-red.svg)](LICENSE)
...@@ -24,10 +23,10 @@ ...@@ -24,10 +23,10 @@
- **2020.09.27**，新增文本生成模型6个，图像分割模型1个，预训练模型总量到达 **【154】** 个。 - **2020.09.27**，新增文本生成模型6个，图像分割模型1个，预训练模型总量到达 **【154】** 个。
- **2020.08.13**，发布v1.8.1，新增人像分割模型Humanseg，支持EMNLP2019-Sentence-BERT作为文本匹配任务网络，预训练模型总量到达 **【147】** 个。 - **2020.08.13**，发布v1.8.1，新增人像分割模型Humanseg，支持EMNLP2019-Sentence-BERT作为文本匹配任务网络，预训练模型总量到达 **【147】** 个。
- **2020.07.29**，发布v1.8.0，新增AI对联和AI写诗、jieba切词，文本数据LDA、语义相似度计算，新增目标检测，短视频分类模型，超轻量中英文OCR，新增行人检测、车辆检测、动物识别等工业级模型，支持VisualDL可视化训练，预训练模型总量到达 **【135】** 个。 - **2020.07.29**，发布v1.8.0，新增AI对联和AI写诗、jieba切词，文本数据LDA、语义相似度计算，新增目标检测，短视频分类模型，超轻量中英文OCR，新增行人检测、车辆检测、动物识别等工业级模型，支持VisualDL可视化训练，预训练模型总量到达 **【135】** 个。
- [More](./docs/release.md) - [More](./docs/docs_ch/release.md)
## [特性](./docs/figures.md) ## [特性](./docs/docs_ch/figures.md)
- **【丰富的预训练模型】**：涵盖CV、NLP、Audio、Video主流四大品类的 180+ 预训练模型，全部开源下载，离线可运行。 - **【丰富的预训练模型】**：涵盖CV、NLP、Audio、Video主流四大品类的 180+ 预训练模型，全部开源下载，离线可运行。
- **【一键模型快速预测】**：通过一行命令行或者极简的Python API实现模型调用，可快速体验模型效果。 - **【一键模型快速预测】**：通过一行命令行或者极简的Python API实现模型调用，可快速体验模型效果。
- **【一键模型转服务化】**：一行命令，搭建深度学习模型API服务化部署能力。 - **【一键模型转服务化】**：一行命令，搭建深度学习模型API服务化部署能力。
...@@ -193,13 +192,13 @@ ...@@ -193,13 +192,13 @@
## 文档教程 ## 文档教程
- [PIP安装](./docs/installation.md) - [PIP安装](./docs/docs_ch/installation.md)
- 快速开始 - 快速开始
- [命令行调用](./docs/quick_experience/cmd_quick_run.md) - [命令行调用](./docs/docs_ch/quick_experience/cmd_quick_run.md)
- [Python API调用](./docs/quick_experience/python_use_hub.md) - [Python API调用](./docs/docs_ch/quick_experience/python_use_hub.md)
- [示例体验项目demo](./docs/quick_experience/more_demos.md) - [示例体验项目demo](./docs/docs_ch/quick_experience/more_demos.md)
- 丰富的预训练模型 182 个 - 丰富的预训练模型 182
- [精品特色模型](./docs/figures.md) - [精品特色模型](./docs/docs_ch/figures.md)
- 计算机视觉 126 个 - 计算机视觉 126 个
- [图像分类 64 个](./modules/image/classification/README.md) - [图像分类 64 个](./modules/image/classification/README.md)
- [目标检测 13 个](./modules/image/object_detection/README.md) - [目标检测 13 个](./modules/image/object_detection/README.md)
...@@ -221,19 +220,18 @@ ...@@ -221,19 +220,18 @@
- 视频5个 - 视频5个
- [视频分类 5 个](./modules/video/README.md) - [视频分类 5 个](./modules/video/README.md)
- 部署 - 部署
- [本地Inference部署](./docs/quick_experience/python_use_hub.md) - [本地Inference部署](./docs/docs_ch/quick_experience/python_use_hub.md)
- [一行代码服务化部署](./docs/tutorial/serving.md) - [一行代码服务化部署](./docs/docs_ch/tutorial/serving.md)
- [移动端 Lite 部署（跳转Lite教程）](https://paddle-lite.readthedocs.io/zh/latest/quick_start/tutorial.html) - [移动端 Lite 部署（跳转Lite教程）](https://paddle-lite.readthedocs.io/zh/latest/quick_start/tutorial.html)
- 进阶文档 - 进阶文档
- [命令行工具详解](./docs/tutorial/cmdintro.md) - [命令行工具详解](./docs/docs_ch/tutorial/cmdintro.md)
- [自定义数据迁移学习](./docs/tutorial/how_to_load_data.md) - [自定义数据迁移学习](./docs/docs_ch/tutorial/how_to_load_data.md)
- [文本Embedding任务](./docs/tutorial/bert_service.md)
- 社区交流 - 社区交流
- [加入技术交流群](#欢迎加入PaddleHub技术交流群) - [加入技术交流群](#欢迎加入PaddleHub技术交流群)
- [贡献预训练模型](./docs/contribution/contri_pretrained_model.md) - [贡献预训练模型](./docs/docs_ch/contribution/contri_pretrained_model.md)
- [贡献代码](./docs/contribution/contri_pr.md) - [贡献代码](./docs/docs_ch/contribution/contri_pr.md)
- [FAQ](./docs/faq.md) - [FAQ](./docs/docs_ch/faq.md)
- [更新历史](./docs/release.md) - [更新历史](./docs/docs_ch/release.md)
- [许可证书](#许可证书) - [许可证书](#许可证书)
- [致谢](#致谢) - [致谢](#致谢)
......
README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
<p align="center">
<img src="./docs/imgs/paddlehub_logo.jpg" align="middle"
</p>
------------------------------------------------------------------------------------------
[![License](https://img.shields.io/badge/license-Apache%202-red.svg)](LICENSE)
[![Version](https://img.shields.io/github/release/PaddlePaddle/PaddleHub.svg)](https://github.com/PaddlePaddle/PaddleHub/releases)
![python version](https://img.shields.io/badge/python-3.6+-orange.svg)
![support os](https://img.shields.io/badge/os-linux%2C%20win%2C%20mac-yellow.svg)
## Introduction
- PaddleHub aims to provide developers with rich, high-quality, and directly usable pre-trained models.
- **【No need for deep learning background, no data and training process】**，you can use AI models quickly and enjoy the dividends of the artificial intelligence era.
- Covers the four major categories of CV, NLP, Audio, and Video, and supports **one-click prediction**, **one-click service deployment** and **migration learning**
- All models are open source download, **offline can run**.
**Recent updates**
- 2020.11.20: Release 2.0-beta version, fully migrate the dynamic graph programming mode, and upgrade the service deployment Serving capability; add 1 hand key detection model, 12 image animation models, 3 image editing models, 3 speech synthesis models, syntax Analyzing one, the total number of pre-trained models reaches **【182】**.
- 2020.10.09: Added 4 new OCR multi-language series models, 4 image editing models, and the total number of pre-trained models reached **【162】**.
- 2020.09.27: 6 new text generation models and 1 image segmentation model were added, and the total number of pre-trained models reached **【154】**.
- 2020.08.13: Released v1.8.1, added a segmentation model, and supports EMNLP2019-Sentence-BERT as a text matching task network. The total number of pre-training models reaches **【147】**.
- 2020.07.29: Release v1.8.0, new AI couplets and AI writing poems, jieba word cutting, text data LDA, semantic similarity calculation, new target detection, short video classification model, ultra-lightweight Chinese and English OCR, new pedestrian detection, vehicle Industrial-grade models such as detection and animal recognition support VisualDL visualization training, and the total number of pre-training models reaches **【135】**.
## Features
- **【Abundant Pre-trained Models】**: 180+ pre-trained models covering the four major categories of CV, NLP, Audio, and Video, all open source downloads, and can be run offline.
- **【Quick Model Prediction】**: Model calls can be realized through a one-line command line or a minimalist Python API to quickly experience the model effect.
- **【Model As Service】**: A one-line command to build deep learning model API service deployment capabilities.
- **【Ten Lines of Code for Transfer Learning】**: Ten lines of code complete the transfer-learning task of image classification and text classification
- **【PIP installation 】**: Support PIP quick installation and use
- **【Cross-platform Compatibility】**: Can run on Linux, Windows, MacOS and other operating systems
## Visualization
### Text recognition
- Contains ultra-lightweight Chinese and English OCR models, high-precision Chinese and English, multilingual German, French, Japanese, Korean OCR recognition.
<div align="center">
<img src="./docs/imgs/Readme_Related/Image_Ocr.gif" width = "800" height = "400" />
</div>
### Face Detection
- Including face detection, mask face detection, multiple algorithms are optional.
<div align="center">
<img src="./docs/imgs/Readme_Related/Image_ObjectDetection_Face_Mask.gif" width = "588" height = "400" />
</div>
### Image editing
- 4 times the super-score effect, multiple super-score algorithms are optional.
- Coloring black and white pictures can be used to repair old photos.
<div align="center">
<table>
<thead>
</thead>
<tbody>
<tr>
<th>SuperResolution </th>
<th>Restoration </th>
</tr>
<tr>
<th>
<a>
<img src="./docs/imgs/Readme_Related/ImageEdit_SuperResolution.gif" width = "266" height = "400" /></a><br>
</th>
<th>
<a>
<img src="./docs/imgs/Readme_Related/ImageEdit_Restoration.gif" width = "300" height = "400" /></a><br>
</th>
</tr>
</tbody>
</table>
</div>
### Object Detection
- Including pedestrian detection, vehicle detection, and more industrial-grade ultra-large-scale pre-training models are optional.
<div align="center">
<img src="./docs/imgs/Readme_Related/Image_ObjectDetection_Pedestrian_Vehicle.gif" width = "642" height = "400" />
</div>
### Key Point Detection
- Including single person, multi-person body key point detection, face key point detection, hand key point detection.
<div align="center">
<img src="./docs/imgs/Readme_Related/Image_keypoint.gif" width = "458" height = "400" />
</div>
### Image Segmentation
- Contains excellent portrait cutout model, ACE2P human body analysis world champion model
<div align="center">
<img src="./docs/imgs/Readme_Related/ImageSeg_Human.gif" width = "642" height = "400" />
</div>
### Image Animation
- Many cartoonists including Hayao Miyazaki and Makoto Shinkai are migrating their styles, and a variety of algorithms are available
<div align="center">
<img src="./docs/imgs/Readme_Related/ImageGan_Anime.gif" width = "642" height = "400" />
</div>
### Image Classification
- Including animal classification, dish classification, wild animal product classification, multiple algorithms are available
<div align="center">
<img src="./docs/imgs/Readme_Related/ImageClas_animal_dish_wild.gif" width = "530" height = "400" />
</div>
### Lexical Analysis
- Including AI poem writing, AI couplets, AI love words, AI hidden poems, multiple algorithms are available.
<div align="center">
<img src="./docs/imgs/Readme_Related/Text_Textgen_poetry.gif" width = "850" height = "400" />
</div>
### Syntax Analysis
- Leading Chinese syntactic analysis model.
<div align="center">
<img src="./docs/imgs/Readme_Related/Text_SyntacticAnalysis.png" width = "640" height = "301" />
</div>
### Sementic Analysis
- Support Chinese comment sentiment analysis
<div align="center">
<img src="./docs/imgs/Readme_Related/Text_SentimentAnalysis.png" width = "640" height = "228" />
</div>
### Text Review
- Contains the review of Chinese pornographic text, and multiple algorithms are available.
<div align="center">
<img src="./docs/imgs/Readme_Related/Text_Textreview.png" width = "640" height = "140" />
</div>
### Speech Synthesis
- TTS speech synthesis algorithm, multiple algorithms are available
- Input: `Life was like a box of chocolates, you never know what you're gonna get.`
- The synthesis effect is as follows:
<div align="center">
<table>
<thead>
</thead>
<tbody>
<tr>
<th>deepvoice3 </th>
<th>fastspeech </th>
<th>transformer</th>
</tr>
<tr>
<th>
<a href="https://paddlehub.bj.bcebos.com/resources/deepvoice3_ljspeech-0.wav">
<img src="./docs/imgs/Readme_Related/audio_icon.png" width=250 /></a><br>
</th>
<th>
<a href="https://paddlehub.bj.bcebos.com/resources/fastspeech_ljspeech-0.wav">
<img src="./docs/imgs/Readme_Related/audio_icon.png" width=250 /></a><br>
</th>
<th>
<a href="https://paddlehub.bj.bcebos.com/resources/transformer_tts_ljspeech-0.wav">
<img src="./docs/imgs/Readme_Related/audio_icon.png" width=250 /></a><br>
</th>
</tr>
</tbody>
</table>
</div>
### Video Classification
- Contains short video classification, supports 3000+ tag types, can output TOP-K tags, and multiple algorithms are optional.
- `Example: Input a short video of swimming, the algorithm can output the result of "swimming"`
<div align="center">
<img src="./docs/imgs/Readme_Related/Text_Video.gif" width = "400" height = "400" />
</div>
## ===Key Points===
-All the above pre-trained models are all open source, and the number of models is continuously updated. Welcome Star to pay attention.
<div align="center">
<a href="https://github.com/PaddlePaddle/PaddleHub/stargazers">
<img src="./docs/imgs/Readme_Related/star_en.png" width = "411" height = "100" /></a>
</div>
<a name="Welcome_joinus"></a>
## Welcome to join PaddleHub technical group
- If you have any questions during the use of the model, you can join the official WeChat group to get more efficient questions and answers, and fully communicate with developers from all walks of life. We look forward to your joining.
<div align="center">
<img src="./docs/imgs/joinus.PNG" width = "200" height = "200" />
</div>
If you fail to scan the code, please add WeChat 15711058002 and note "Hub", the operating class will invite you to join the group.
## Documentation Tutorial
- [PIP Installation](./docs/docs_en/installation_en.md)
- Quick Start
- [Command Line](./docs/docs_en/quick_experience/cmd_quick_run_en.md)
- [Python API](./docs/docs_en/quick_experience/python_use_hub_en.md)
- [More_Demos](./docs/docs_en/quick_experience/more_demos_en.md)
- Rich Pre-trained Models 182
- [Boutique Featured Models](./docs/docs_en/figures_en.md)
- Computer vision 126
- [Image Classification 64 ](./modules/image/classification/README_en.md)
- [Object Detection 13 ](./modules/image/object_detection/README_en.md)
- [Face Detection 7 ](./modules/image/face_detection/README_en.md)
- [Key Point Detection 3 ](./modules/image/keypoint_detection/README_en.md)
- [Image Segmentation 7 ](./modules/image/semantic_segmentation/README_en.md)
- [Text Recognition 8 ](./modules/image/text_recognition/README_en.md)
- [Image Generation 17 ](./modules/image/Image_gan/README_en.md)
- [Image Editing 7 ](./modules/image/Image_editing/README_en.md)
- Natural language processing 48
- [Lexical Analysis 2 ](./modules/text/lexical_analysis/README_en.md)
- [Syntax Analysis 1 ](./modules/text/syntactic_analysis/README_en.md)
- [Emotion Analysis 7 ](./modules/text/sentiment_analysis/README_en.md)
- [Text Review 3 ](./modules/text/text_review/README_en.md)
- [Text Generation 9 ](./modules/text/text_generation/README_en.md)
- [Semantic Models 26 ](./modules/text/language_model/README_en.md)
- Audio 3
- [Speech Synthesis 3 ](./modules/audio/README_en.md)
- Video 5
- [Video Classification 5 ](./modules/video/README_en.md)
- Deploy
- [Local Inference Deployment](./docs/docs_en/quick_experience/python_use_hub_en.md)
- [One Line of Code Service deployment](./docs/docs_en/tutorial/serving_en.md)
- [Mobile Lite Deployment (link to Lite tutorial)](https://paddle-lite.readthedocs.io/zh/latest/quick_start/tutorial.html)
- Advanced documentation
- [Command_Line](./docs/docs_en/tutorial/cmdintro_en.md)
- [How to Load Data](./docs/docs_en/tutorial/how_to_load_data_en.md)
- Community
- [Join the technical group](#Welcome_joinus)
- [Contribute pre-trained models](./docs/docs_en/contribution/contri_pretrained_model_en.md)
- [Contribute code](./docs/docs_en/contribution/contri_pr_en.md)
- [License](#License)
- [Contribution](#Contribution)
<a name="License"></a>
## License
The release of this project is certified by the <a href="./LICENSE">Apache 2.0 license</a>.
<a name="Contribution"></a>
## Contribution
We welcome you to contribute code to PaddleHub, and thank you for your feedback.
* Many thanks to [Austendeng](https://github.com/Austendeng) for fixing the SequenceLabelReader
* Many thanks to [cclauss](https://github.com/cclauss) optimizing travis-ci check
* Many thanks to [奇想天外](http://www.cheerthink.com/)，Contributed a demo of mask detection
* Many thanks to [mhlwsk](https://github.com/mhlwsk)，Contributed the repair sequence annotation prediction demo
* Many thanks to [zbp-xxxp](https://github.com/zbp-xxxp)，Contributed modules for viewing pictures and writing poems
* Many thanks to [zbp-xxxp](https://github.com/zbp-xxxp) and [七年期限](https://github.com/1084667371),Jointly contributed to the Mid-Autumn Festival Special Edition Module
* Many thanks to [livingbody](https://github.com/livingbody)，Contributed models for style transfer based on PaddleHub's capabilities and Mid-Autumn Festival WeChat Mini Program
docs/contribution/contri_pretrained_model.md → docs/docs_ch/contribution/contri_pretrained_model.md
浏览文件 @ 2b17b1f8
...@@ -14,7 +14,7 @@ author_email: ...@@ -14,7 +14,7 @@ author_email:
type: nlp/sentiment_analysis type: nlp/sentiment_analysis
``` ```
**本示例代码可以参考[senta_module_sample](../../demo/senta_module_sample/senta_test)**
Module存在一个接口sentiment_classify，用于接收传入文本，并给出文本的情感倾向（正面/负面），支持python接口调用和命令行调用。 Module存在一个接口sentiment_classify，用于接收传入文本，并给出文本的情感倾向（正面/负面），支持python接口调用和命令行调用。
```python ```python
......
docs/figures.md → docs/docs_ch/figures.md
浏览文件 @ 2b17b1f8
...@@ -13,7 +13,7 @@ ...@@ -13,7 +13,7 @@
| 图像分割 | [人像分割](https://www.paddlepaddle.org.cn/hubdetail?name=deeplabv3p_xception65_humanseg&en_category=ImageSegmentation)、[人体解析](https://www.paddlepaddle.org.cn/hubdetail?name=ace2p&en_category=ImageSegmentation)、[肺炎CT影像分析](https://www.paddlepaddle.org.cn/hubdetail?name=Pneumonia_CT_LKM_PP&en_category=ImageSegmentation)、[-->More](../modules/image/semantic_segmentation/README.md) | | 图像分割 | [人像分割](https://www.paddlepaddle.org.cn/hubdetail?name=deeplabv3p_xception65_humanseg&en_category=ImageSegmentation)、[人体解析](https://www.paddlepaddle.org.cn/hubdetail?name=ace2p&en_category=ImageSegmentation)、[肺炎CT影像分析](https://www.paddlepaddle.org.cn/hubdetail?name=Pneumonia_CT_LKM_PP&en_category=ImageSegmentation)、[-->More](../modules/image/semantic_segmentation/README.md) |
| 关键点检测 | [人体关键点](https://www.paddlepaddle.org.cn/hubdetail?name=human_pose_estimation_resnet50_mpii&en_category=KeyPointDetection)、[人脸关键点](https://www.paddlepaddle.org.cn/hubdetail?name=face_landmark_localization&en_category=KeyPointDetection)、[手部关键点](https://www.paddlepaddle.org.cn/hubdetail?name=hand_pose_localization&en_category=KeyPointDetection)、[-->More](./modules/image/keypoint_detection/README.md) | | 关键点检测 | [人体关键点](https://www.paddlepaddle.org.cn/hubdetail?name=human_pose_estimation_resnet50_mpii&en_category=KeyPointDetection)、[人脸关键点](https://www.paddlepaddle.org.cn/hubdetail?name=face_landmark_localization&en_category=KeyPointDetection)、[手部关键点](https://www.paddlepaddle.org.cn/hubdetail?name=hand_pose_localization&en_category=KeyPointDetection)、[-->More](./modules/image/keypoint_detection/README.md) |
| 文本识别 | [超轻量中英文OCR文字识别](https://www.paddlepaddle.org.cn/hubdetail?name=chinese_ocr_db_crnn_mobile&en_category=TextRecognition)、[-->More](../modules/image/text_recognition/README.md) | | 文本识别 | [超轻量中英文OCR文字识别](https://www.paddlepaddle.org.cn/hubdetail?name=chinese_ocr_db_crnn_mobile&en_category=TextRecognition)、[-->More](../modules/image/text_recognition/README.md) |
| 图像生成 | [风格迁移](https://www.paddlepaddle.org.cn/hubdetail?name=stylepro_artistic&en_category=GANs)、[街景动漫画]()、[-->More](../modules/image/Image_gan/README.md) | | 图像生成 | [风格迁移](https://www.paddlepaddle.org.cn/hubdetail?name=stylepro_artistic&en_category=GANs)、[街景动漫画](https://www.paddlepaddle.org.cn/hubdetail?name=animegan_v2_hayao_99&en_category=GANs)、[-->More](../modules/image/Image_gan/README.md) |
| 图像编辑 | [超分辨率](https://www.paddlepaddle.org.cn/hubdetail?name=realsr&en_category=ImageEditing)、[黑白上色](https://www.paddlepaddle.org.cn/hubdetail?name=deoldify&en_category=ImageEditing)、[-->More](../modules/image/Image_editing/README.md) | | 图像编辑 | [超分辨率](https://www.paddlepaddle.org.cn/hubdetail?name=realsr&en_category=ImageEditing)、[黑白上色](https://www.paddlepaddle.org.cn/hubdetail?name=deoldify&en_category=ImageEditing)、[-->More](../modules/image/Image_editing/README.md) |
#### 1.2、文本 #### 1.2、文本
| | **精品模型举例** | | | **精品模型举例** |
...@@ -21,17 +21,17 @@ ...@@ -21,17 +21,17 @@
| 词句分析 | [词法分析 ](https://www.paddlepaddle.org.cn/hubdetail?name=lac&en_category=LexicalAnalysis)、[句法分析](https://www.paddlepaddle.org.cn/hubdetail?name=ddparser&en_category=SyntacticAnalysis)、[-->More](../modules/text/lexical_analysis/README.md) | | 词句分析 | [词法分析 ](https://www.paddlepaddle.org.cn/hubdetail?name=lac&en_category=LexicalAnalysis)、[句法分析](https://www.paddlepaddle.org.cn/hubdetail?name=ddparser&en_category=SyntacticAnalysis)、[-->More](../modules/text/lexical_analysis/README.md) |
| 情感分析 | [情感判断](https://www.paddlepaddle.org.cn/hubdetail?name=lac&en_category=LexicalAnalysis)、[情绪分析](https://www.paddlepaddle.org.cn/hubdetail?name=emotion_detection_textcnn&en_category=SentimentAnalysis) 、[-->More](../modules/text/sentiment_analysis/README.md)| | 情感分析 | [情感判断](https://www.paddlepaddle.org.cn/hubdetail?name=lac&en_category=LexicalAnalysis)、[情绪分析](https://www.paddlepaddle.org.cn/hubdetail?name=emotion_detection_textcnn&en_category=SentimentAnalysis) 、[-->More](../modules/text/sentiment_analysis/README.md)|
| 文本审核 | [色情审核](https://www.paddlepaddle.org.cn/hubdetail?name=porn_detection_gru&en_category=TextCensorship)、[-->More](../modules/text/text_review/README.md) | | 文本审核 | [色情审核](https://www.paddlepaddle.org.cn/hubdetail?name=porn_detection_gru&en_category=TextCensorship)、[-->More](../modules/text/text_review/README.md) |
| 文本生成 | [对联生成]()、[情话生成]()、[藏图诗生成]()、[土味情话]() 、[-->More](../modules/text/text_generation/README.md)| | 文本生成 | [对联生成](https://www.paddlepaddle.org.cn/hubdetail?name=ernie_tiny_couplet&en_category=TextGeneration)、[情话生成](https://www.paddlepaddle.org.cn/hubdetail?name=ernie_gen_poetry&en_category=TextGeneration)、[藏头诗生成](https://www.paddlepaddle.org.cn/hubdetail?name=ernie_gen_acrostic_poetry&en_category=TextGeneration)、[土味情话](https://www.paddlepaddle.org.cn/hubdetail?name=ernie_gen_lover_words&en_category=TextGeneration) 、[-->More](../modules/text/text_generation/README.md)|
| 语义模型 | [ERNIE](https://www.paddlepaddle.org.cn/hubdetail?name=ERNIE&en_category=SemanticModel)、[文本相似度](https://www.paddlepaddle.org.cn/hubdetail?name=simnet_bow&en_category=SemanticModel)、[-->More](../modules/text/language_model/README.md) | | 语义模型 | [ERNIE](https://www.paddlepaddle.org.cn/hubdetail?name=ERNIE&en_category=SemanticModel)、[文本相似度](https://www.paddlepaddle.org.cn/hubdetail?name=simnet_bow&en_category=SemanticModel)、[-->More](../modules/text/language_model/README.md) |
#### 1.3、语音 #### 1.3、语音
| | **精品模型举例** | | | **精品模型举例** |
| ---------- | :----------------------------------------------------------- | | ---------- | :----------------------------------------------------------- |
| 语音合成 | [语音合成]() 、[-->More](../modules/audio/README.md) | | 语音合成 | [语音合成](https://www.paddlepaddle.org.cn/hubdetail?name=deepvoice3_ljspeech&en_category=TextToSpeech) 、[-->More](../modules/audio/README.md) |
#### 1.4、视频 #### 1.4、视频
| | **精品模型举例** | | | **精品模型举例** |
| ---------- | :----------------------------------------------------------- | | ---------- | :----------------------------------------------------------- |
| 视频分类 | [视频分类]()、[-->More](../modules/video/README.md) | | 视频分类 | [视频分类](https://www.paddlepaddle.org.cn/hublist?filter=en_category&value=VideoClassificationhttps://www.paddlepaddle.org.cn/hublist?filter=en_category&value=VideoClassification)、[-->More](../modules/video/README.md) |
<a name="一键模型预测"></a> <a name="一键模型预测"></a>
...@@ -48,7 +48,7 @@ $ hub run chinese_ocr_db_crnn_mobile --input_path test_ocr.jpg --visualization=T ...@@ -48,7 +48,7 @@ $ hub run chinese_ocr_db_crnn_mobile --input_path test_ocr.jpg --visualization=T
* 预测结果图片保存在当前运行路径下ocr_result文件夹中，如下图所示。 * 预测结果图片保存在当前运行路径下ocr_result文件夹中，如下图所示。
<p align="center"> <p align="center">
<img src="./imgs/ocr_res.jpg" width='70%' align="middle" <img src="../imgs/ocr_res.jpg" width='70%' align="middle"
</p> </p>
* 使用词法分析模型LAC进行分词 * 使用词法分析模型LAC进行分词
...@@ -85,7 +85,7 @@ $ hub serving start -m chinese_ocr_db_crnn_mobile ...@@ -85,7 +85,7 @@ $ hub serving start -m chinese_ocr_db_crnn_mobile
* [Demo示例](../demo)提供丰富的Fine-tune API的使用代码，包括[图像分类](../demo/image_classification)、[图像着色](../demo/colorization)、[风格迁移](../demo/style_transfer)、等场景的模型迁移示例。 * [Demo示例](../demo)提供丰富的Fine-tune API的使用代码，包括[图像分类](../demo/image_classification)、[图像着色](../demo/colorization)、[风格迁移](../demo/style_transfer)、等场景的模型迁移示例。
<p align="center"> <p align="center">
<img src="./imgs/paddlehub_finetune.gif" align="middle" <img src="../imgs/paddlehub_finetune.gif" align="middle"
</p> </p>
<p align='center'> <p align='center'>
......
docs/release.md → docs/docs_ch/release.md
浏览文件 @ 2b17b1f8
...@@ -4,7 +4,7 @@ ...@@ -4,7 +4,7 @@
* 全面迁移动态图编程模式，模型开发调试更加方便，finetune接口更加灵活易用。 * 全面迁移动态图编程模式，模型开发调试更加方便，finetune接口更加灵活易用。
* 优化服务化部署Serving能力，支持多卡预测、自动负载均衡，性能大幅度提升。 * 优化服务化部署Serving能力，支持多卡预测、自动负载均衡，性能大幅度提升。
* 视觉类迁移学习能力全面升级，支持[图像分类](../demo/image_classification)、[图像着色](../demo/colorization)、[风格迁移](../demo/style_transfer)等多种视觉任务。 * 视觉类迁移学习能力全面升级，支持[图像分类](../../demo/image_classification)、[图像着色](../../demo/colorization)、[风格迁移](../../demo/style_transfer)等多种视觉任务。
## `v1.8.1` ## `v1.8.1`
......
docs/tutorial/cmdintro.md → docs/docs_ch/tutorial/cmdintro.md
浏览文件 @ 2b17b1f8
...@@ -103,6 +103,7 @@ $ hub run lac --input_file test.txt ...@@ -103,6 +103,7 @@ $ hub run lac --input_file test.txt
# 单张照片预测 # 单张照片预测
$ hub run resnet_v2_50_imagenet --input_path test.jpg $ hub run resnet_v2_50_imagenet --input_path test.jpg
``` ```
```shell ```shell
# 多张照片预测 # 多张照片预测
$ hub run resnet_v2_50_imagenet --input_file test.txt $ hub run resnet_v2_50_imagenet --input_file test.txt
......
docs/docs_ch/tutorial/contri_pretrained_model.md 0 → 100644
浏览文件 @ 2b17b1f8
# 如何编写一个PaddleHub Module
## 模型基本信息
我们准备编写一个PaddleHub Module，Module的基本信息如下：
```yaml
name="openpose_body_estimation",
type="CV/image_editing",
author="paddlepaddle",
author_email="",
summary="Openpose_body_estimation is a body pose estimation model based on Realtime Multi-Person 2D Pose \
Estimation using Part Affinity Fields.",
version="1.0.0"
```
Module存在一个接口predict，用于接收传入图片，并得到最终输出的结果，支持python接口调用和命令行调用。
```python
import paddlehub as hub
model = hub.Module(name="openpose_body_estimation")
result = model.predict("demo.jpg")
```
```cmd
hub run openpose_body_estimation --input_path demo.jpg
```
## Module创建
### step 1. 创建必要的目录与文件
创建一个openpose_body_estimation的目录，并在openpose_body_estimation目录下分别创建module.py, processor.py。其中
|文件名|用途|
|-|-|
|module.py|主模块，提供Module的实现代码|
|processor.py|辅助模块，提供词表加载的方法|
```cmd
➜ tree openpose_body_estimation
openpose_body_estimation/
├── module.py
└── processor.py
```
### step 2. 实现辅助模块processor
在processor.py中实现一些在module.py里面需要调用到的类和函数。例如在processor.py 中实现ResizeScaling类：
```python
class ResizeScaling:
"""Resize images by scaling method.
Args:
target(int): Target image size.
interpolation(Callable): Interpolation method.
"""
def __init__(self, target: int = 368, interpolation: Callable = cv2.INTER_CUBIC):
self.target = target
self.interpolation = interpolation
def __call__(self, img, scale_search):
scale = scale_search * self.target / img.shape[0]
resize_img = cv2.resize(img, (0, 0), fx=scale, fy=scale, interpolation=self.interpolation)
return resize_img
```
### step 3. 编写Module处理代码
module.py文件为Module的入口代码所在，我们需要在其中实现预测逻辑。
#### step 3_1. 引入必要的头文件
```python
import os
import time
import copy
import base64
import argparse
from typing import Union
from collections import OrderedDict
import cv2
import paddle
import paddle.nn as nn
import numpy as np
from paddlehub.module.module import moduleinfo, runnable, serving
import paddlehub.vision.transforms as T
import openpose_body_estimation.processor as P
```
**NOTE:** `paddlehub.vision.transforms`有常见的图像处理方法，可以方便调用。
#### step 3_2. 定义BodyPoseModel类
module.py中需要有一个继承了nn.Layer，该类负责实现预测逻辑，并使用moduleinfo填写基本信息。当使用hub.Module(name="openpose_body_estimation")加载Module时，PaddleHub会自动创建openpose_body_estimation的对象并返回。
```python
@moduleinfo(
name="openpose_body_estimation",
type="CV/image_editing",
author="paddlepaddle",
author_email="",
summary="Openpose_body_estimation is a body pose estimation model based on Realtime Multi-Person 2D Pose \
Estimation using Part Affinity Fields.",
version="1.0.0")
class BodyPoseModel(nn.Layer):
...
```
#### step 3_3. 执行必要的初始化及模型搭建
模型的初始化主要完成几个功能：待使用的类的声明，模型使用的类的声明及参数加载。
```python
def __init__(self, load_checkpoint: str = None):
super(BodyPoseModel, self).__init__()
#将会使用到的类的声明
self.resize_func = P.ResizeScaling()
self.norm_func = T.Normalize(std=[1, 1, 1])
#模型声明
self.input_nc = 4
self.output_nc = 2
model1 = (
Conv2D(self.input_nc, 64, 3, 1, 1),
nn.ReLU(),
Conv2D(64, 64, 3, 1, 1),
nn.ReLU(),
nn.BatchNorm(64),
)
self.model1 = nn.Sequential(*model1)
#参数加载
if load_checkpoint is not None:
self.model_dict = paddle.load(load_checkpoint)
self.set_dict(self.model_dict)
print("load custom checkpoint success")
else:
checkpoint = os.path.join(self.directory, 'model.pdparams')
self.model_dict = paddle.load(checkpoint)
self.set_dict(self.model_dict)
print("load pretrained checkpoint success")
```
模型的搭建主要在`forward`里面实现：
```python
def forward(self, input: paddle.Tensor) -> paddle.Tensor:
result = self.model1(input)
return result
```
#### step 3_4. 完善预测逻辑
```python
def predict(self, img:Union(np.ndarray,str), visualization: bool = True):
self.eval()
self.visualization = visualization
if isinstance(img, str):
orgImg = cv2.imread(img)
else:
orgImg = img
data = self.resize_func(self.norm_func(orgImg))
output = self.forward(paddle.to_tensor(data.astype('float32')))
output = paddle.clip(output[0].transpose((1, 2, 0)), 0, 255).numpy()
output = output.astype(np.uint8)
if self.visualization:
style_name = "body_" + str(time.time()) + ".png"
if not os.path.exists(save_path):
os.mkdir(save_path)
path = os.path.join(save_path, style_name)
cv2.imwrite(path, output)
return output
```
#### step 3_5. 支持命令行调用
如果希望Module可以支持命令行调用，则需要提供一个经过runnable修饰的接口，接口负责解析传入数据并进行预测，将结果返回。
```python
@runnable
def run_cmd(self, argvs):
"""
Run as a command.
"""
self.parser = argparse.ArgumentParser(
description="Run the {} module.".format(self.name),
prog='hub run {}'.format(self.name),
usage='%(prog)s',
add_help=True)
self.arg_input_group = self.parser.add_argument_group(
title="Input options", description="Input data. Required")
self.arg_config_group = self.parser.add_argument_group(
title="Config options",
description=
"Run configuration for controlling module behavior, not required.")
self.add_module_config_arg()
self.add_module_input_arg()
args = self.parser.parse_args(argvs)
results = self.predict(
img=args.input_path,
save_path=args.output_dir,
visualization=args.visualization)
return results
def add_module_config_arg(self):
"""
Add the command config options.
"""
self.arg_config_group.add_argument(
'--output_dir',
type=str,
default='openpose_body',
help="The directory to save output images.")
self.arg_config_group.add_argument(
'--save_dir',
type=str,
default='openpose_model',
help="The directory to save model.")
self.arg_config_group.add_argument(
'--visualization',
type=bool,
default=True,
help="whether to save output as images.")
def add_module_input_arg(self):
"""
Add the command input options.
"""
self.arg_input_group.add_argument(
'--input_path', type=str, help="path to image.")
```
#### step 3_6. 支持serving调用
如果希望Module可以支持PaddleHub Serving部署预测服务，则需要提供一个经过serving修饰的接口，接口负责解析传入数据并进行预测，将结果返回。
如果不需要提供PaddleHub Serving部署预测服务，则可以不需要加上serving修饰。
```python
@serving
def serving_method(self, images, **kwargs):
"""
Run as a service.
"""
images_decode = [base64_to_cv2(image) for image in images]
results = self.predict(img=images_decode[0], **kwargs)
final={}
final['data'] = P.cv2_to_base64(results)
return final
```
## 测试步骤
完成Module编写后，我们可以通过以下方式测试该Module
### 调用方法1
将Module安装到本机中，再通过Hub.Module(name=...)加载
```shell
hub install openpose_body_estimation
```
```python
import paddlehub as hub
if __name__ == "__main__":
model = hub.Module(name='openpose_hands_estimation')
result = model.predict("demo.jpg")
```
### 调用方法2
将Module安装到本机中，再通过hub run运行
```shell
hub install openpose_body_estimation
hub run openpose_body_estimation --input_path demo.jpg
```
### 测试serving方法
运行启动命令：
```shell
$ hub serving start -m openpose_body_estimation
```
发送预测请求，获取预测结果.
```python
import requests
import json
import cv2
import base64
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
# 发送HTTP请求
org_im = cv2.imread('/PATH/TO/IMAGE')
data = {'images':[cv2_to_base64(org_im)]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/openpose_body_estimation"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
canvas = base64_to_cv2(r.json()["results"]['data'])
cv2.imwrite('keypoint_body.png', canvas)
```
docs/docs_ch/tutorial/contri_pretrained_model.md.orig 0 → 100644
浏览文件 @ 2b17b1f8
# 如何编写一个PaddleHub Module
## 模型基本信息
我们准备编写一个PaddleHub Module，Module的基本信息如下：
```yaml
name="openpose_body_estimation",
type="CV/image_editing",
author="paddlepaddle",
author_email="",
summary="Openpose_body_estimation is a body pose estimation model based on Realtime Multi-Person 2D Pose \
Estimation using Part Affinity Fields.",
version="1.0.0"
```
Module存在一个接口predict，用于接收传入图片，并得到最终输出的结果，支持python接口调用和命令行调用。
```python
import paddlehub as hub
model = hub.Module(name="openpose_body_estimation")
result = model.predict("demo.jpg")
```
```cmd
hub run openpose_body_estimation --input_path demo.jpg
```
## Module创建
### step 1. 创建必要的目录与文件
创建一个openpose_body_estimation的目录，并在openpose_body_estimation目录下分别创建module.py, processor.py。其中
|文件名|用途|
|-|-|
|module.py|主模块，提供Module的实现代码|
|processor.py|辅助模块，提供词表加载的方法|
```cmd
➜ tree openpose_body_estimation
openpose_body_estimation/
├── module.py
└── processor.py
```
### step 2. 实现辅助模块processor
在processor.py中实现一些在module.py里面需要调用到的类和函数。例如在processor.py 中实现ResizeScaling类：
```python
class ResizeScaling:
"""Resize images by scaling method.
Args:
target(int): Target image size.
interpolation(Callable): Interpolation method.
"""
def __init__(self, target: int = 368, interpolation: Callable = cv2.INTER_CUBIC):
self.target = target
self.interpolation = interpolation
def __call__(self, img, scale_search):
scale = scale_search * self.target / img.shape[0]
resize_img = cv2.resize(img, (0, 0), fx=scale, fy=scale, interpolation=self.interpolation)
return resize_img
```
### step 3. 编写Module处理代码
module.py文件为Module的入口代码所在，我们需要在其中实现预测逻辑。
#### step 3_1. 引入必要的头文件
```python
import os
import time
import copy
import base64
import argparse
from typing import Union
from collections import OrderedDict
import cv2
import paddle
import paddle.nn as nn
import numpy as np
from paddlehub.module.module import moduleinfo, runnable, serving
import paddlehub.vision.transforms as T
import openpose_body_estimation.processor as P
```
**NOTE:** `paddlehub.vision.transforms`有常见的图像处理方法，可以方便调用。
#### step 3_2. 定义BodyPoseModel类
module.py中需要有一个继承了nn.Layer，该类负责实现预测逻辑，并使用moduleinfo填写基本信息。当使用hub.Module(name="openpose_body_estimation")加载Module时，PaddleHub会自动创建openpose_body_estimation的对象并返回。
```python
@moduleinfo(
name="openpose_body_estimation",
type="CV/image_editing",
author="paddlepaddle",
author_email="",
summary="Openpose_body_estimation is a body pose estimation model based on Realtime Multi-Person 2D Pose \
Estimation using Part Affinity Fields.",
version="1.0.0")
class BodyPoseModel(nn.Layer):
...
```
#### step 3_3. 执行必要的初始化及模型搭建
模型的初始化主要完成几个功能：待使用的类的声明，模型使用的类的声明及参数加载。
```python
def __init__(self, load_checkpoint: str = None):
super(BodyPoseModel, self).__init__()
#将会使用到的类的声明
self.resize_func = P.ResizeScaling()
self.norm_func = T.Normalize(std=[1, 1, 1])
#模型声明
self.input_nc = 4
self.output_nc = 2
model1 = (
Conv2D(self.input_nc, 64, 3, 1, 1),
nn.ReLU(),
Conv2D(64, 64, 3, 1, 1),
nn.ReLU(),
nn.BatchNorm(64),
)
self.model1 = nn.Sequential(*model1)
#参数加载
if load_checkpoint is not None:
self.model_dict = paddle.load(load_checkpoint)
self.set_dict(self.model_dict)
print("load custom checkpoint success")
else:
checkpoint = os.path.join(self.directory, 'model.pdparams')
self.model_dict = paddle.load(checkpoint)
self.set_dict(self.model_dict)
print("load pretrained checkpoint success")
```
模型的搭建主要在`forward`里面实现：
```python
def forward(self, input: paddle.Tensor) -> paddle.Tensor:
result = self.model1(input)
return result
```
#### step 3_4. 完善预测逻辑
```python
def predict(self, img:Union(np.ndarray,str), visualization: bool = True):
self.eval()
self.visualization = visualization
if isinstance(img, str):
orgImg = cv2.imread(img)
else:
orgImg = img
data = self.resize_func(self.norm_func(orgImg))
output = self.forward(paddle.to_tensor(data.astype('float32')))
output = paddle.clip(output[0].transpose((1, 2, 0)), 0, 255).numpy()
output = output.astype(np.uint8)
if self.visualization:
style_name = "body_" + str(time.time()) + ".png"
if not os.path.exists(save_path):
os.mkdir(save_path)
path = os.path.join(save_path, style_name)
cv2.imwrite(path, output)
return output
```
#### step 3_5. 支持命令行调用
如果希望Module可以支持命令行调用，则需要提供一个经过runnable修饰的接口，接口负责解析传入数据并进行预测，将结果返回。
```python
@runnable
def run_cmd(self, argvs):
"""
Run as a command.
"""
self.parser = argparse.ArgumentParser(
description="Run the {} module.".format(self.name),
prog='hub run {}'.format(self.name),
usage='%(prog)s',
add_help=True)
self.arg_input_group = self.parser.add_argument_group(
title="Input options", description="Input data. Required")
self.arg_config_group = self.parser.add_argument_group(
title="Config options",
description=
"Run configuration for controlling module behavior, not required.")
self.add_module_config_arg()
self.add_module_input_arg()
args = self.parser.parse_args(argvs)
results = self.predict(
img=args.input_path,
save_path=args.output_dir,
visualization=args.visualization)
return results
def add_module_config_arg(self):
"""
Add the command config options.
"""
self.arg_config_group.add_argument(
'--output_dir',
type=str,
default='openpose_body',
help="The directory to save output images.")
self.arg_config_group.add_argument(
'--save_dir',
type=str,
default='openpose_model',
help="The directory to save model.")
self.arg_config_group.add_argument(
'--visualization',
type=bool,
default=True,
help="whether to save output as images.")
def add_module_input_arg(self):
"""
Add the command input options.
"""
self.arg_input_group.add_argument(
'--input_path', type=str, help="path to image.")
```
#### step 3_6. 支持serving调用
如果希望Module可以支持PaddleHub Serving部署预测服务，则需要提供一个经过serving修饰的接口，接口负责解析传入数据并进行预测，将结果返回。
如果不需要提供PaddleHub Serving部署预测服务，则可以不需要加上serving修饰。
```python
@serving
def serving_method(self, images, **kwargs):
"""
Run as a service.
"""
images_decode = [base64_to_cv2(image) for image in images]
results = self.predict(img=images_decode[0], **kwargs)
final={}
final['data'] = P.cv2_to_base64(results)
return final
```
## 测试步骤
完成Module编写后，我们可以通过以下方式测试该Module
### 调用方法1
将Module安装到本机中，再通过Hub.Module(name=...)加载
```shell
hub install openpose_body_estimation
```
```python
import paddlehub as hub
if __name__ == "__main__":
model = hub.Module(name='openpose_hands_estimation')
result = model.predict("demo.jpg")
```
### 调用方法2
将Module安装到本机中，再通过hub run运行
```shell
hub install openpose_body_estimation
hub run openpose_body_estimation --input_path demo.jpg
```
### 测试serving方法
运行启动命令：
```shell
$ hub serving start -m openpose_body_estimation
```
发送预测请求，获取预测结果.
```python
import requests
import json
import cv2
import base64
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
# 发送HTTP请求
org_im = cv2.imread('/PATH/TO/IMAGE')
data = {'images':[cv2_to_base64(org_im)]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/openpose_body_estimation"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
canvas = base64_to_cv2(r.json()["results"]['data'])
cv2.imwrite('keypoint_body.png', canvas)
```
docs/docs_ch/tutorial/how_to_finetune.md 0 → 100644
浏览文件 @ 2b17b1f8
## **迁移学习**
### **概述**
迁移学习 (Transfer Learning) 是属于深度学习的一个子研究领域，该研究领域的目标在于利用数据、任务、或模型之间的相似性，将在旧领域学习过的知识，迁移应用于新领域中。通俗的来讲，迁移学习就是运用已有的知识来学习新的知识，例如学会了骑自行车的人也能较快的学会骑电动车。较为常用的一种迁移学习方式是利用预训练模型进行微调，即用户基于当前任务的场景从PaddleHub中选择已训练成功的模型进行新任务训练，且该模型曾经使用的数据集与新场景的数据集情况相近，此时仅需要在当前任务场景的训练过程中使用新场景的数据对模型参数进行微调（**Fine-tune**），即可完成训练任务。迁移学习吸引了很多研究者投身其中，因为它能够很好的解决深度学习中的以下几个问题：
* 一些研究领域只有少量标注数据，且数据标注成本较高，不足以训练一个足够鲁棒的神经网络。
* 大规模神经网络的训练依赖于大量的计算资源，这对于一般用户而言难以实现。
* 应对于普适化需求的模型，在特定应用上表现不尽如人意。
为了让开发者更便捷地应用迁移学习，飞桨开源了预训练模型管理工具 PaddleHub。开发者仅仅使用十余行的代码，就能完成迁移学习。本文将为读者全面介绍使用PaddleHub完成迁移学习的方法。
### **前置条件**
在开始迁移学习之前，用户需要做好如下工作：
* 用户已安装PaddleHub。
* 准备好用于迁移学习的数据，用户可以选择使用PaddleHub提供的数据集或者自定义数据集，如果是自定义数据，需要参考“自定义数据集如何Fine-tune”对数据集处理。
* 使用hub install命令安装或更新用于训练的module，以使用ERNIE模型为例，命令格式如下所示。用户可能在之前的任务中已经安装过相关的预训练模型，但是仍然推荐用户在开始训练前执行此步骤，这样可以保证预训练模型是最新版本。
```python
$ hub install ernie==1.2.0
```
### **迁移学习流程**
用户完成迁移学习前需要先编写好用于迁移学习的脚本。用户编写脚本的过程非常简单，仅需要十余行代码即可完成。整个脚本的编写过程，可以分为如下几个步骤：
1. 导入必要的包。
2. 加载预训练模型（Module），即加载PaddleHub提供的预训练模型。
3. 加载数据集（Dataset），用户可以选择使用dataset API加载PaddleHub自带的数据集或者自行编写加载数据集的类来加载自定义数据集。
4. 配置数据读取器（Reader），负责将dataset的数据进行预处理，以特定格式组织并输入给模型进行训练。
5. 选择优化策略（Strategy），优化策略包含了多种预训练参数，例如使用什么学习率变化策略，使用哪种类型的优化器，使用什么类型的正则化等。
6. 设置运行配置（RunConfig），RunConfig包含了一些训练相关的配置，包括是否使用GPU、训练的轮数（Epoch）、训练批次大小（batch_size）等。
7. 组建训练任务（Task），一个迁移学习训练任务中会包含与该任务相关的Program和上面设置好的数据读取器Reader、运行配置等内容。
8. 启动Fine-tune，使用Finetune_and_eval函数完成训练和评估。
### **学会编写迁移学习训练脚本**
PaddleHub提供了Finetune API和预训练模型完成多种不同任务场景的迁移学习，包括图像分类、文本分类、多标签分类、序列标注、检索式问答任务、回归任务、句子语义相似度计算、阅读理解任务等。本文将以文本分类为例介绍迁移学习脚本的编写方法。
#### **1. 导入必要的包。**
```python
import paddlehub as hub
```
#### **2. 加载预训练模型**
使用如下代码加载预训练模型，本例使用ERNIE预训练模型来完成文本分类任务。ERNIE（Enhanced Representation through kNowledge IntEgration）是百度提出的语义表示模型，以Transformer Encoder为网络基本组件，其预训练过程利用了更丰富的语义知识和更多的语义任务，用户可以使用该预训练模型随时逐步引入不同的自定义任务，例如命名实体预测、语篇关系识别、句子顺序预测任务、情感分析等。
```python
module = hub.Module(name="ernie")
```
PaddleHub还提供很多了其它可用于迁移学习的预训练模型, 在PaddleHub的官网上，图像分类、语义模型和情感分析几个目录下的预训练模型都支持迁移学习，用户仅需要将name的取值换成预训练模型名称即可，例如右侧红框中的示例。
![](../../imgs/Howtofinetune1.png)
#### **3. 加载数据集**
在加载好预训练模型后，我们来加载数据集。用于迁移学习的数据集来源可以分为两种，用户自定义数据集和PaddleHub提供的数据集，使用不同类型的数据集加载方式也有所不同。
##### **加载PaddleHub自带数据集**
如果用户使用的是PaddleHub自带数据集，则可以通过PaddleHub的数据集API编写一行代码完成加载数据集的动作。
```python
dataset = hub.dataset.ChnSentiCorp()
```
其中ChnSentiCorp是中文情感分析数据集，其目标是判断一段文本的情感态度。例如文本是“这道菜很好吃”，则对应的标签为“1”，表示正向评价，又例如“房间太小了”，对应标签为“0”，表示负面评价。PaddleHub还提供了其他的文本分类数据集，用户可以自行选择数据集对应的API替换上面代码中dataset的取值，具体信息如下表所示。
|数据集|名称|API|
|:--------:|:--------:|:--------:|
|ChnSentiCorp|中文情感分析数据集|hub.dataset.ChnSentiCorp()|
|LCQMC|哈尔滨工业大学在自然语言处理国际顶会 COLING2018 构建的问答匹配中文数据集，其目标是判断两个问题的语义是否相同。|hub.dataset.LCQMC()|
|NLPCC-DPQA|国际自然语言处理和中文计算会议NLPCC于2016年举办的评测任务数据集，，其目标是选择能够回答问题的答案。|hub.dataset.NLPCC_DPQA()|
|MSRA-NER|微软亚研院发布的数据集，其目标是命名实体识别|hub.dataset.MSRA-NER()|
|Toxic|英文多标签分类数据集|hub.dataset.Toxic()|
|SQUAD|英文阅读理解数据集|hub.dataset.SQUAD()|
|GLUE-CoLA|文本分类任务数据集|hub.dataset.GLUE("CoLA")|
|GLUE-SST-2|情感分析任务数据集|hub.dataset.GLUE("SST-2")|
|GLUE-MNLI|文本推理任务数据集|hub.dataset.GLUE("MNLI_m")|
|GLUE-QQP|句子对分类任务数据集|hub.dataset.GLUE("QQP")|
|GLUE-QNLI|问题推理任务数据集|hub.dataset.GLUE("QNLI")|
|GLUE-STS-B|回归任务数据集|hub.dataset.GLUE("STS-B")|
|GLUE-MRPC|句子对分类任务数据集|hub.dataset.GLUE("MRPC")|
|GLUE-RTE|文本蕴含任务数据集|hub.dataset.GLUE("RTE")|
|XNLI|跨语言自然语言推理数据集|hub.dataset.XNLI(language=zh)|
|ChineseGLUE-TNEWS|今日头条中文新闻（短文本）分类数据集|hub.dataset.TNews()|
|ChineseGLUE-INEWS|互联网情感分析任务数据集|hub.dataset.INews()|
|DRCD|台达阅读理解数据集，属于通用领域繁体中文机器阅读理解数据集|hub.dataset.DRCD()|
|CMRC2018|中文机器阅读理解的跨度提取数据集|hub.dataset.CMRC2018()|
|ChinesGLUE-BQ|智能客服中文问句匹配数据集|hub.dataset.BQ()|
|ChineseGLUE-IFLYTEK|中文长文本分类数据集，该数据集共有1.7万多条关于app应用描述的长文本标注数据|hub.dataset.IFLYTEK()|
|ChineseGLUE-THUCNEWS|中文长文本分类数据集，该数据集共有4万多条中文新闻长文本标注数据，共14个类别|hub.dataset.THUCNEWS()|
|DogCatDataset|由Kaggle提供的数据集，用于图像二分类|hub.dataset.DogCatDataset()|
|Food101|由Kaggle提供的食品图片数据集，含有101种类别|hub.dataset.Food101()|
|Indoor67|由麻省理工学院发布的数据集，其包含67种室内场景，其目标是识别一张室内图片的场景类别。|hub.dataset.Indoor67()|
|Flowers|花卉数据集，数据集有5种类型，包括"roses"，"tulips"，"daisy"，"sunflowers"，"dandelion"|hub.dataset.Flowers()|
|StanfordDogs|斯坦福大学发布的数据集，其包含120个种类的狗，用于做图像分类。|hub.dataset.StanfordDogs()|
##### **加载自定义数据集**
* 加载文本类自定义数据集。用户仅需要继承基类BaseNLPDatast，修改数据集存放地址以及类别即可，具体可以参考如下代码。
```python
from paddlehub.dataset.base_nlp_dataset import BaseNLPDataset
# 构建数据集的类
class DemoDataset(BaseNLPDataset):
def __init__(self):
# 数据集实际路径
self.dataset_dir = "path/to/dataset"
super(DemoDataset, self).__init__(
base_path=self.dataset_dir,
train_file="train.tsv", # 训练集存放地址
dev_file="dev.tsv", # 验证集存放地址
test_file="test.tsv", # 测试集存放地址
# 如果还有预测数据（不需要文本类别label），可以放在predict.tsv
predict_file="predict.tsv",
train_file_with_header=True, # 训练集文件是否有列说明
dev_file_with_header=True, # 验证集文件是否有列说明
test_file_with_header=True, # 测试集文件是否有列说明
predict_file_with_header=True, # 预测集文件是否有列说明
# 数据集类别集合
label_list=["0", "1"])
# 通过创建Dataset对象加载自定义文本数据集
dataset = DemoDataset()
```
然后就可以通过DemoDataset()获取自定义数据集了。进而配合数据预处理器以及预训练模型如ERNIE完成文本类任务。
* 加载图像类自定义数据集。用用户仅需要继承基类BaseCVDatast，修改数据集存放地址即可，具体可以参考如下代码。
```python
from paddlehub.dataset.base_cv_dataset import BaseCVDataset
class DemoDataset(BaseCVDataset):
def __init__(self):
# 数据集存放位置
self.dataset_dir = "/test/data"
super(DemoDataset, self).__init__(
base_path=self.dataset_dir,
train_list_file="train_list.txt", # 训练集存放地址
validate_list_file="validate_list.txt", # 验证集存放地址
test_list_file="test_list.txt", # 测试集存放地址
predict_file="predict_list.txt", # 预测集存放地址
label_list_file="label_list.txt", # 数据集类别文件所在地址
# 如果您的数据集类别较少，可以不用定义label_list.txt，可以在最后设置label_list=["数据集所有类别"]。
)
# 通过创建Dataset对象加载图像类数据集
dataset = DemoDataset()
```
然后就可以通过DemoDataset()获取自定义数据集了。进而配合数据预处理器以及预训练模型完成视觉类的迁移学习任务。
#### **4. 配置数据预处理器**
通过使用PaddleHub的数据预处理器API来读取NLP或CV的数据集数据。
```python
reader = hub.reader.ClassifyReader(
dataset=dataset,
vocab_path=module.get_vocab_path(), # 返回预训练模型对应的词表
max_seq_len=128, # 需要与Step1中context接口传入的序列长度保持一致
sp_model_path=module.get_spm_path(), # 若module为ernie_tiny则返回对应的子词切分模型，否则返回None
word_dict_path=module.get_word_dict_path()) # 若module为ernie_tiny则返回对应的词语切分模型，否则返回None
```
对于不同的任务类型，用户可以选择不同的Reader。
|数据读取器|描述|任务类型|API示例|
|:--------:|:--------:|:--------:|:--------|
|ClassifyReader|适用于Transformer预训练模型(ERNIE/BERT)的数据预处理器。|NLP|reader = hub.reader.ClassifyReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vocab_path=module.get_vocab_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;max_seq_len=128,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;sp_model_path=module.get_spm_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;word_dict_path=module.get_word_dict_path()) |
|LACClassifyReader|以LAC模块为切词器的数据预处理器，适用于Senta、ELMo等需要以词粒度分词的任务。|NLP|reader = hub.reader.LACClassifyReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vocab_path=module.get_vocab_path())|
|SequenceLabelReader|适用于Transformer类模型(ERNIE/BERT)的序列标注预处理器。|NLP|reader = hub.reader.SequenceLabelReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vocab_path=module.get_vocab_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;max_seq_len=128,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;sp_model_path=module.get_spm_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;word_dict_path=module.get_word_dict_path())|
|MultiLabelClassifyReader|适用于Transformer类模型(ERNIE/BERT)的多标签分类预处理器。|NLP|reader = hub.reader.MultiLabelClassifyReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vocab_path=module.get_vocab_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;max_seq_len=128)|
|ReadingComprehensionReader|适用于Transformer类模型(ERNIE/BERT)的阅读理解任务预处理器。|NLP|reader = hub.reader.ReadingComprehensionReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vocab_path=module.get_vocab_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;max_seq_length=384)|
|RegressionReader|适用于回归任务的数据预处理器。|NLP|reader = hub.reader.RegressionReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vocab_path=module.get_vocab_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;max_seq_len=args.max_seq_len)|
|ImageClassificationReader|适用于图像分类数据的预处理器。会修改输入图像的尺寸、进行标准化处理、图像增广处理等操作。|CV|reader = hub.reader.ImageClassificationReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;image_width=module.get_expected_image_width(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;image_height=module.get_expected_image_height(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;images_mean=module.get_pretrained_images_mean(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;images_std=module.get_pretrained_images_std(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset)|
#### **5. 选择优化策略**
在PaddleHub中，Strategy类封装了一系列适用于迁移学习的Fine-tuning策略。Strategy包含了对预训练参数使用什么学习率变化策略，使用哪种类型的优化器，使用什么类型的正则化等。在我们要做的文本分类任务中，我们使用AdamWeightDecayStrategy优化策略。具体可以参考如下代码：
```python
strategy = hub.AdamWeightDecayStrategy(
learning_rate=5e-5, # Fine-tune过程中的最大学习率
weight_decay=0.01, # 模型的正则项参数，默认0.01，如果模型有过拟合倾向，可适当调高这一参数
warmup_proportion=0.1, #如果warmup_proportion>0, 例如0.1, 则学习率会在前10%的steps中线性增长至最高值learning_rate
# 有两种策略可选：
# （1）linear_decay策略学习率会在最高点后以线性方式衰减;
# （2）noam_decay策略学习率会在最高点以多项式形式衰减；
lr_scheduler="linear_decay",
)
```
包括AdamWeightDecayStrategy在内，PaddleHub还提供了多种优化策略的API。
|优化策略|描述|API示例|
|:--------:|:--------|:--------|
|DefaultFinetuneStrategy|默认的优化策略。其对应参数如下：<br>* learning_rate: 全局学习率。默认为1e-4。<br>* optimizer_name: 优化器名称。默认adam。<br>* regularization_coeff: 正则化的λ参数。默认为1e-3。<br>在图像分类任务中推荐使用此优化策略。|strategy = hub.DefaultFinetuneStrategy(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;learning_rate=1e-4,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;optimizer_name="adam",<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;regularization_coeff=1e-3)|
|AdamWeightDecayStrategy|基于Adam优化器的学习率衰减策略。其对应参数如下：<br>* learning_rate: 全局学习率，默认为1e-4。<br>* lr_scheduler: 学习率调度方法，默认为"linear_decay"。<br>* warmup_proportion: warmup所占比重。<br>* weight_decay: 学习率衰减率。<br>* optimizer_name: 优化器名称，默认为adam。<br>在文本分类、阅读理解等任务中推荐使用此优化策略。|strategy = hub.AdamWeightDecayStrategy(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;learning_rate=1e-4,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;lr_scheduler="linear_decay",<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;warmup_proportion=0.0,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;weight_decay=0.01,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;optimizer_name="adam")|
|L2SPFinetuneStrategy|使用L2SP正则作为惩罚因子的Finetune策略。其对应参数如下：<br>* learning_rate: 全局学习率。默认为1e-4。<br>* optimizer_name: 优化器名称。默认adam。<br>* regularization_coeff: 正则化的λ参数。默认为1e-3。|strategy = hub.L2SPFinetuneStrategy(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;learning_rate=1e-4,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;optimizer_name="adam",<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;regularization_coeff=1e-3)|
|ULMFiTStrategy|该策略实现了ULMFiT论文中提出的三种策略：<br> * Slanted triangular learning rates是一种学习率先上升再下降的策略。<br>* Discriminative fine-tuning是一种学习率逐层递减的策略，通过该策略可以减缓底层的更新速度。<br>* Gradual unfreezing是一种逐层解冻的策略，通过该策略可以优先更新上层，再慢慢解冻下层参与更新。<br>其对应参数如下：<br> * learning_rate: 全局学习率。默认为1e-4。<br> * optimizer_name: 优化器名称。默认为adam。<br> * cut_fraction: 设置Slanted triangular learning rates学习率上升的步数在整个训练总步数中的比例。默认为0.1，如果设置为0，则不采用Slanted triangular learning rates。<br> * ratio: 设置Slanted triangular learning rates下降的最小学习率与上升的最大学习率的比例关系，默认为32，表示最小学习率是最大学习率的1/32。<br> * dis_blocks: 设置 Discriminative fine-tuning中的块数。默认为3，如果设置为0，则不采用Discriminative fine-tuning。<br> * factor: 设置Discriminative fine-tuning的衰减率。默认为2.6，表示下一层的学习率是上一层的1/2.6。<br> * frz_blocks: 设置Gradual unfreezing中的块数。块的概念同“dis_blocks”中介绍的概念。|strategy = hub.ULMFiTStrategy(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;learning_rate=1e-4,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;optimizer_name="adam",<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cut_fraction=0.1,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ratio=32,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dis_blocks=3,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;factor=2.6,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;frz_blocks=3)|
#### **6. 设置运行配置。**
在PaddleHub中，用户可以使用Finetune API中的RunConfig配置Task进行Finetune时使用的参数，包括运行的Epoch次数、batch的大小、是否使用GPU训练等。代码示例如下所示。
```python
config = hub.RunConfig(use_cuda=True, num_epoch=3, batch_size=32, strategy=strategy)
```
#### **7. 组建训练任务。**
有了合适的预训练模型，并加载好要迁移的数据集后，我们开始组建一个Task。在PaddleHub中，Task代表了一个Fine-tune的任务。任务中包含了执行该任务相关的Program、数据读取器Reader、运行配置等内容。在这里可以找到文本分类任务对应的Task说明[TextClassifierTask](https://github.com/PaddlePaddle/PaddleHub/blob/release/v1.6/docs/reference/task/text_classify_task.md)。具体实现方案如下：
1. 获取module（PaddleHub的预训练模型）的上下文环境，包括输入和输出的变量，以及Paddle Program（可执行的模型格式）。
2. 从预训练模型的输出变量中找到特征图提取层feature_map，在feature_map后面接入一个全连接层，如下代码中通过hub.TextClassifierTask的pooled_output参数指定。
3. 网络的输入层保持不变，依然从输入层开始，如下代码中通过hub.TextClassifierTask的参数feed_list变量指定。
hub.TextClassifierTask就是通过这两个参数明确我们的截取模型网络的要求，按照这样的配置，我们截取的网络是从输入层一直到特征提取的最后一层“pooled_output”，表示我将使用截出的网络来进行迁移学习训练。
```python
# 获取Module的上下文信息，得到输入、输出以及预训练的Paddle Program副本。
# trainable设置为True时，Module中的参数在Fine-tune时也会随之训练，否则保持不变。
# 其中最大序列长度max_seq_len为可调整的参数，建议值为128，根据任务文本长度不同可以进行修改，但最大不超过512。
# 若序列长度不足，会通过padding方式补到max_seq_len, 若序列长度大于该值，则会以截断方式让序列长度为max_seq_len。
inputs, outputs, program = module.context(trainable=True, max_seq_len=128)
# 返回ERNIE/BERT模型对应的[CLS]向量,可以用于句子或句对的特征表达。
pooled_output = outputs["pooled_output"]
# feed_list的Tensor顺序不可以调整
# 指定ERNIE中的输入tensor的顺序，与ClassifyReader返回的结果一致
feed_list = [
inputs["input_ids"].name,
inputs["position_ids"].name,
inputs["segment_ids"].name,
inputs["input_mask"].name,
]
# 通过输入特征，label与迁移的类别数，可以生成适用于文本分类的迁移任务
cls_task = hub.TextClassifierTask(
data_reader=reader, # 读取数据的reader
feature=pooled_output, # 从预训练提取的特征矩阵
feed_list=feed_list, # 待feed变量的名字列表
num_classes=dataset.num_labels, # 数据集的类别数量
metrics_choices = ["acc"],
config=config) # 运行配置
```
PaddleHub预置了常见任务的Task，每种Task都有特定的应用场景并提供了对应的度量指标，满足用户的不同需求。
|Task类型|描述|任务类型|
|:--------:|:--------:|:--------:|
|ImageClassifierTask|该Task基于输入的特征，添加一个或多个全连接层来创建一个分类任务用于Fine-tune，度量指标为准确率，损失函数为交叉熵Loss。|图像分类任务|
|TextClassifierTask|该Task基于输入的特征，添加一个Dropout层，以及一个或多个全连接层来创建一个文本分类任务用于finetune，度量指标为准确率，损失函数为交叉熵Loss。|文本分类任务|
|SequenceLabelTask|该Task基于输入的特征，添加一个全连接层或者一个全连接层和CRF层来创建一个序列标注任务用于Fine-tune，度量指标为F1，损失函数为交叉熵Loss。|序列标注任务|
|MultiLabelClassifierTask|该Task基于输入的特征，添加一个或多个全连接层来创建一个多标签分类任务用于finetune，度量指标为多个标签的平均AUC，损失函数为多个标签的平均交叉熵。|多标签分类任务|
|RegressionTask|该Task基于输入的特征，添加一个Dropout层，以及一个或多个全连接层来创建一个文本回归任务用于finetune，度量指标为准确率，损失函数为均方差损失函数。|文本回归任务|
|ReadingComprehensionTask|该Task基于输入的特征，添加一个全连接层来创建一个阅读理解任务用于Fine-tune，损失函数为交叉熵Loss。|阅读理解任务|
在设定每个Task前，用户需要提前了解待迁移学习的预训练模型的输入与输出，即对应上面代码中的“feed_list”和“pooled_output”。具体的输入输出代码可以分为如下几类：
* 图像分类模型
```
input_dict, output_dict, program = module.context(trainable=True)
feature_map = output_dict["feature_map"]
feed_list = [input_dict["image"].name]
```
* 自然语言处理模型（不包括word2vec_skipgram、simnet_bow、文本匹配和文本生成几个模型）
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=128)
pooled_output = outputs["pooled_output"]
feed_list = [
inputs["input_ids"].name,
inputs["position_ids"].name,
inputs["segment_ids"].name,
inputs["input_mask"].name,
]
```
* word2vec_skipgram模型
```
inputs, outputs, program = module.context(trainable=True)
word_ids = inputs["word_ids"]
embedding = outputs["word_embs"]
```
* simnet_bow模型
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=args.max_seq_len, num_slots=2)
query = outputs["emb"]
title = outputs['emb_2']
```
* Pairwise文本匹配模型
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=args.max_seq_len, num_slots=3)
query = outputs["emb"]
left = outputs['emb_2']
right = outputs['emb_3']
```
* Pointwise文本匹配
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=args.max_seq_len, num_slots=2)
query = outputs["emb"]
title = outputs['emb_2']
```
* 文本生成模型
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=128)
pooled_output = outputs["pooled_output"]
sequence_output = outputs["sequence_output"]
```
#### **8 启动Fine-tune，使用Finetune_and_eval函数完成训练和评估。**
```python
cls_task.finetune_and_eval()
```
显示信息如下例所示。可以看到训练的评估结果，Loss值和准确率等等。
```
[2020-07-28 21:28:21,658] [ TRAIN] - step 810 / 900: loss=0.05022 acc=0.97813 [step/sec: 4.07]
[2020-07-28 21:28:24,115] [ TRAIN] - step 820 / 900: loss=0.04719 acc=0.98125 [step/sec: 4.07]
[2020-07-28 21:28:26,574] [ TRAIN] - step 830 / 900: loss=0.06895 acc=0.98125 [step/sec: 4.07]
[2020-07-28 21:28:29,035] [ TRAIN] - step 840 / 900: loss=0.07830 acc=0.97813 [step/sec: 4.07]
[2020-07-28 21:28:31,490] [ TRAIN] - step 850 / 900: loss=0.07279 acc=0.97500 [step/sec: 4.08]
[2020-07-28 21:28:33,939] [ TRAIN] - step 860 / 900: loss=0.03220 acc=0.99375 [step/sec: 4.09]
[2020-07-28 21:28:36,388] [ TRAIN] - step 870 / 900: loss=0.05016 acc=0.98750 [step/sec: 4.09]
[2020-07-28 21:28:38,840] [ TRAIN] - step 880 / 900: loss=0.05604 acc=0.98750 [step/sec: 4.08]
[2020-07-28 21:28:41,293] [ TRAIN] - step 890 / 900: loss=0.05622 acc=0.98125 [step/sec: 4.08]
[2020-07-28 21:28:43,748] [ TRAIN] - step 900 / 900: loss=0.06642 acc=0.97813 [step/sec: 4.08]
[2020-07-28 21:28:43,750] [ INFO] - Evaluation on dev dataset start
[2020-07-28 21:28:46,654] [ EVAL] - [dev dataset evaluation result] loss=0.17890 acc=0.94079 [step/sec: 13.23]
[2020-07-28 21:28:46,657] [ INFO] - Evaluation on dev dataset start
[2020-07-28 21:28:49,527] [ EVAL] - [dev dataset evaluation result] loss=0.17890 acc=0.94079 [step/sec: 13.39]
[2020-07-28 21:28:49,529] [ INFO] - Load the best model from ckpt_20200728212416/best_model
[2020-07-28 21:28:50,112] [ INFO] - Evaluation on test dataset start
[2020-07-28 21:28:52,987] [ EVAL] - [test dataset evaluation result] loss=0.14264 acc=0.94819 [step/sec: 13.36]
[2020-07-28 21:28:52,988] [ INFO] - Saving model checkpoint to ckpt_20200728212416/step_900
[2020-07-28 21:28:55,789] [ INFO] - PaddleHub finetune finished.
```
通过Fine-tune完成模型训练后，在对应的ckpt目录（CKPT_DIR）下，PaddleHub会自动保存验证集上效果最好的模型。用户可以参考如下代码进行预测，其中推理出的标签值0表示负向评价，1表示正向评价。。
```python
import numpy as np
# 待预测数据
data = [
["这个宾馆比较陈旧了，特价的房间也很一般。总体来说一般"],
["交通方便；环境很好；服务态度很好 房间较小"],
["19天硬盘就罢工了，算上运来的一周都没用上15天，可就是不能换了。唉，你说这算什么事呀！"]
]
index = 0
run_states = cls_task.predict(data=data)
results = [run_state.run_results for run_state in run_states]
for batch_result in results:
# 预测类别取最大分类概率值
batch_result = np.argmax(batch_result[0], axis=1)
for result in batch_result:
print("%s\tpredict=%s" % (data[index][0], result))
index += 1
```
预测结果如下所示。
```
[2020-07-28 18:06:45,441] [ INFO] - PaddleHub predict start
[2020-07-28 18:06:45,442] [ INFO] - The best model has been loaded
[2020-07-28 18:06:48,406] [ INFO] - PaddleHub predict finished.
这个宾馆比较陈旧了，特价的房间也很一般。总体来说一般 predict=0
交通方便；环境很好；服务态度很好 房间较小 predict=1
19天硬盘就罢工了，算上运来的一周都没用上15天，可就是不能换了。唉，你说这算什么事呀！ predict=0
```
docs/docs_ch/tutorial/how_to_finetune.md.orig 0 → 100644
浏览文件 @ 2b17b1f8
## **迁移学习**
### **概述**
迁移学习 (Transfer Learning) 是属于深度学习的一个子研究领域，该研究领域的目标在于利用数据、任务、或模型之间的相似性，将在旧领域学习过的知识，迁移应用于新领域中。通俗的来讲，迁移学习就是运用已有的知识来学习新的知识，例如学会了骑自行车的人也能较快的学会骑电动车。较为常用的一种迁移学习方式是利用预训练模型进行微调，即用户基于当前任务的场景从PaddleHub中选择已训练成功的模型进行新任务训练，且该模型曾经使用的数据集与新场景的数据集情况相近，此时仅需要在当前任务场景的训练过程中使用新场景的数据对模型参数进行微调（**Fine-tune**），即可完成训练任务。迁移学习吸引了很多研究者投身其中，因为它能够很好的解决深度学习中的以下几个问题：
* 一些研究领域只有少量标注数据，且数据标注成本较高，不足以训练一个足够鲁棒的神经网络。
* 大规模神经网络的训练依赖于大量的计算资源，这对于一般用户而言难以实现。
* 应对于普适化需求的模型，在特定应用上表现不尽如人意。
为了让开发者更便捷地应用迁移学习，飞桨开源了预训练模型管理工具 PaddleHub。开发者仅仅使用十余行的代码，就能完成迁移学习。本文将为读者全面介绍使用PaddleHub完成迁移学习的方法。
### **前置条件**
在开始迁移学习之前，用户需要做好如下工作：
* 用户已安装PaddleHub。
* 准备好用于迁移学习的数据，用户可以选择使用PaddleHub提供的数据集或者自定义数据集，如果是自定义数据，需要参考“自定义数据集如何Fine-tune”对数据集处理。
* 使用hub install命令安装或更新用于训练的module，以使用ERNIE模型为例，命令格式如下所示。用户可能在之前的任务中已经安装过相关的预训练模型，但是仍然推荐用户在开始训练前执行此步骤，这样可以保证预训练模型是最新版本。
```python
$ hub install ernie==1.2.0
```
### **迁移学习流程**
用户完成迁移学习前需要先编写好用于迁移学习的脚本。用户编写脚本的过程非常简单，仅需要十余行代码即可完成。整个脚本的编写过程，可以分为如下几个步骤：
1. 导入必要的包。
2. 加载预训练模型（Module），即加载PaddleHub提供的预训练模型。
3. 加载数据集（Dataset），用户可以选择使用dataset API加载PaddleHub自带的数据集或者自行编写加载数据集的类来加载自定义数据集。
4. 配置数据读取器（Reader），负责将dataset的数据进行预处理，以特定格式组织并输入给模型进行训练。
5. 选择优化策略（Strategy），优化策略包含了多种预训练参数，例如使用什么学习率变化策略，使用哪种类型的优化器，使用什么类型的正则化等。
6. 设置运行配置（RunConfig），RunConfig包含了一些训练相关的配置，包括是否使用GPU、训练的轮数（Epoch）、训练批次大小（batch_size）等。
7. 组建训练任务（Task），一个迁移学习训练任务中会包含与该任务相关的Program和上面设置好的数据读取器Reader、运行配置等内容。
8. 启动Fine-tune，使用Finetune_and_eval函数完成训练和评估。
### **学会编写迁移学习训练脚本**
PaddleHub提供了Finetune API和预训练模型完成多种不同任务场景的迁移学习，包括图像分类、文本分类、多标签分类、序列标注、检索式问答任务、回归任务、句子语义相似度计算、阅读理解任务等。本文将以文本分类为例介绍迁移学习脚本的编写方法。
#### **1. 导入必要的包。**
```python
import paddlehub as hub
```
#### **2. 加载预训练模型**
使用如下代码加载预训练模型，本例使用ERNIE预训练模型来完成文本分类任务。ERNIE（Enhanced Representation through kNowledge IntEgration）是百度提出的语义表示模型，以Transformer Encoder为网络基本组件，其预训练过程利用了更丰富的语义知识和更多的语义任务，用户可以使用该预训练模型随时逐步引入不同的自定义任务，例如命名实体预测、语篇关系识别、句子顺序预测任务、情感分析等。
```python
module = hub.Module(name="ernie")
```
PaddleHub还提供很多了其它可用于迁移学习的预训练模型, 在PaddleHub的官网上，图像分类、语义模型和情感分析几个目录下的预训练模型都支持迁移学习，用户仅需要将name的取值换成预训练模型名称即可，例如右侧红框中的示例。
![](../../imgs/Howtofinetune1.png)
#### **3. 加载数据集**
在加载好预训练模型后，我们来加载数据集。用于迁移学习的数据集来源可以分为两种，用户自定义数据集和PaddleHub提供的数据集，使用不同类型的数据集加载方式也有所不同。
##### **加载PaddleHub自带数据集**
如果用户使用的是PaddleHub自带数据集，则可以通过PaddleHub的数据集API编写一行代码完成加载数据集的动作。
```python
dataset = hub.dataset.ChnSentiCorp()
```
其中ChnSentiCorp是中文情感分析数据集，其目标是判断一段文本的情感态度。例如文本是“这道菜很好吃”，则对应的标签为“1”，表示正向评价，又例如“房间太小了”，对应标签为“0”，表示负面评价。PaddleHub还提供了其他的文本分类数据集，用户可以自行选择数据集对应的API替换上面代码中dataset的取值，具体信息如下表所示。
|数据集|名称|API|
|:--------:|:--------:|:--------:|
|ChnSentiCorp|中文情感分析数据集|hub.dataset.ChnSentiCorp()|
|LCQMC|哈尔滨工业大学在自然语言处理国际顶会 COLING2018 构建的问答匹配中文数据集，其目标是判断两个问题的语义是否相同。|hub.dataset.LCQMC()|
|NLPCC-DPQA|国际自然语言处理和中文计算会议NLPCC于2016年举办的评测任务数据集，，其目标是选择能够回答问题的答案。|hub.dataset.NLPCC_DPQA()|
|MSRA-NER|微软亚研院发布的数据集，其目标是命名实体识别|hub.dataset.MSRA-NER()|
|Toxic|英文多标签分类数据集|hub.dataset.Toxic()|
|SQUAD|英文阅读理解数据集|hub.dataset.SQUAD()|
|GLUE-CoLA|文本分类任务数据集|hub.dataset.GLUE("CoLA")|
|GLUE-SST-2|情感分析任务数据集|hub.dataset.GLUE("SST-2")|
|GLUE-MNLI|文本推理任务数据集|hub.dataset.GLUE("MNLI_m")|
|GLUE-QQP|句子对分类任务数据集|hub.dataset.GLUE("QQP")|
|GLUE-QNLI|问题推理任务数据集|hub.dataset.GLUE("QNLI")|
|GLUE-STS-B|回归任务数据集|hub.dataset.GLUE("STS-B")|
|GLUE-MRPC|句子对分类任务数据集|hub.dataset.GLUE("MRPC")|
|GLUE-RTE|文本蕴含任务数据集|hub.dataset.GLUE("RTE")|
|XNLI|跨语言自然语言推理数据集|hub.dataset.XNLI(language=zh)|
|ChineseGLUE-TNEWS|今日头条中文新闻（短文本）分类数据集|hub.dataset.TNews()|
|ChineseGLUE-INEWS|互联网情感分析任务数据集|hub.dataset.INews()|
|DRCD|台达阅读理解数据集，属于通用领域繁体中文机器阅读理解数据集|hub.dataset.DRCD()|
|CMRC2018|中文机器阅读理解的跨度提取数据集|hub.dataset.CMRC2018()|
|ChinesGLUE-BQ|智能客服中文问句匹配数据集|hub.dataset.BQ()|
|ChineseGLUE-IFLYTEK|中文长文本分类数据集，该数据集共有1.7万多条关于app应用描述的长文本标注数据|hub.dataset.IFLYTEK()|
|ChineseGLUE-THUCNEWS|中文长文本分类数据集，该数据集共有4万多条中文新闻长文本标注数据，共14个类别|hub.dataset.THUCNEWS()|
|DogCatDataset|由Kaggle提供的数据集，用于图像二分类|hub.dataset.DogCatDataset()|
|Food101|由Kaggle提供的食品图片数据集，含有101种类别|hub.dataset.Food101()|
|Indoor67|由麻省理工学院发布的数据集，其包含67种室内场景，其目标是识别一张室内图片的场景类别。|hub.dataset.Indoor67()|
|Flowers|花卉数据集，数据集有5种类型，包括"roses"，"tulips"，"daisy"，"sunflowers"，"dandelion"|hub.dataset.Flowers()|
|StanfordDogs|斯坦福大学发布的数据集，其包含120个种类的狗，用于做图像分类。|hub.dataset.StanfordDogs()|
##### **加载自定义数据集**
* 加载文本类自定义数据集。用户仅需要继承基类BaseNLPDatast，修改数据集存放地址以及类别即可，具体可以参考如下代码。
```python
from paddlehub.dataset.base_nlp_dataset import BaseNLPDataset
# 构建数据集的类
class DemoDataset(BaseNLPDataset):
def __init__(self):
# 数据集实际路径
self.dataset_dir = "path/to/dataset"
super(DemoDataset, self).__init__(
base_path=self.dataset_dir,
train_file="train.tsv", # 训练集存放地址
dev_file="dev.tsv", # 验证集存放地址
test_file="test.tsv", # 测试集存放地址
# 如果还有预测数据（不需要文本类别label），可以放在predict.tsv
predict_file="predict.tsv",
train_file_with_header=True, # 训练集文件是否有列说明
dev_file_with_header=True, # 验证集文件是否有列说明
test_file_with_header=True, # 测试集文件是否有列说明
predict_file_with_header=True, # 预测集文件是否有列说明
# 数据集类别集合
label_list=["0", "1"])
# 通过创建Dataset对象加载自定义文本数据集
dataset = DemoDataset()
```
然后就可以通过DemoDataset()获取自定义数据集了。进而配合数据预处理器以及预训练模型如ERNIE完成文本类任务。
* 加载图像类自定义数据集。用用户仅需要继承基类BaseCVDatast，修改数据集存放地址即可，具体可以参考如下代码。
```python
from paddlehub.dataset.base_cv_dataset import BaseCVDataset
class DemoDataset(BaseCVDataset):
def __init__(self):
# 数据集存放位置
self.dataset_dir = "/test/data"
super(DemoDataset, self).__init__(
base_path=self.dataset_dir,
train_list_file="train_list.txt", # 训练集存放地址
validate_list_file="validate_list.txt", # 验证集存放地址
test_list_file="test_list.txt", # 测试集存放地址
predict_file="predict_list.txt", # 预测集存放地址
label_list_file="label_list.txt", # 数据集类别文件所在地址
# 如果您的数据集类别较少，可以不用定义label_list.txt，可以在最后设置label_list=["数据集所有类别"]。
)
# 通过创建Dataset对象加载图像类数据集
dataset = DemoDataset()
```
然后就可以通过DemoDataset()获取自定义数据集了。进而配合数据预处理器以及预训练模型完成视觉类的迁移学习任务。
#### **4. 配置数据预处理器**
通过使用PaddleHub的数据预处理器API来读取NLP或CV的数据集数据。
```python
reader = hub.reader.ClassifyReader(
dataset=dataset,
vocab_path=module.get_vocab_path(), # 返回预训练模型对应的词表
max_seq_len=128, # 需要与Step1中context接口传入的序列长度保持一致
sp_model_path=module.get_spm_path(), # 若module为ernie_tiny则返回对应的子词切分模型，否则返回None
word_dict_path=module.get_word_dict_path()) # 若module为ernie_tiny则返回对应的词语切分模型，否则返回None
```
对于不同的任务类型，用户可以选择不同的Reader。
|数据读取器|描述|任务类型|API示例|
|:--------:|:--------:|:--------:|:--------|
|ClassifyReader|适用于Transformer预训练模型(ERNIE/BERT)的数据预处理器。|NLP|reader = hub.reader.ClassifyReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vocab_path=module.get_vocab_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;max_seq_len=128,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;sp_model_path=module.get_spm_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;word_dict_path=module.get_word_dict_path()) |
|LACClassifyReader|以LAC模块为切词器的数据预处理器，适用于Senta、ELMo等需要以词粒度分词的任务。|NLP|reader = hub.reader.LACClassifyReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vocab_path=module.get_vocab_path())|
|SequenceLabelReader|适用于Transformer类模型(ERNIE/BERT)的序列标注预处理器。|NLP|reader = hub.reader.SequenceLabelReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vocab_path=module.get_vocab_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;max_seq_len=128,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;sp_model_path=module.get_spm_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;word_dict_path=module.get_word_dict_path())|
|MultiLabelClassifyReader|适用于Transformer类模型(ERNIE/BERT)的多标签分类预处理器。|NLP|reader = hub.reader.MultiLabelClassifyReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vocab_path=module.get_vocab_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;max_seq_len=128)|
|ReadingComprehensionReader|适用于Transformer类模型(ERNIE/BERT)的阅读理解任务预处理器。|NLP|reader = hub.reader.ReadingComprehensionReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vocab_path=module.get_vocab_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;max_seq_length=384)|
|RegressionReader|适用于回归任务的数据预处理器。|NLP|reader = hub.reader.RegressionReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vocab_path=module.get_vocab_path(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;max_seq_len=args.max_seq_len)|
|ImageClassificationReader|适用于图像分类数据的预处理器。会修改输入图像的尺寸、进行标准化处理、图像增广处理等操作。|CV|reader = hub.reader.ImageClassificationReader(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;image_width=module.get_expected_image_width(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;image_height=module.get_expected_image_height(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;images_mean=module.get_pretrained_images_mean(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;images_std=module.get_pretrained_images_std(),<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dataset=dataset)|
#### **5. 选择优化策略**
在PaddleHub中，Strategy类封装了一系列适用于迁移学习的Fine-tuning策略。Strategy包含了对预训练参数使用什么学习率变化策略，使用哪种类型的优化器，使用什么类型的正则化等。在我们要做的文本分类任务中，我们使用AdamWeightDecayStrategy优化策略。具体可以参考如下代码：
```python
strategy = hub.AdamWeightDecayStrategy(
learning_rate=5e-5, # Fine-tune过程中的最大学习率
weight_decay=0.01, # 模型的正则项参数，默认0.01，如果模型有过拟合倾向，可适当调高这一参数
warmup_proportion=0.1, #如果warmup_proportion>0, 例如0.1, 则学习率会在前10%的steps中线性增长至最高值learning_rate
# 有两种策略可选：
# （1）linear_decay策略学习率会在最高点后以线性方式衰减;
# （2）noam_decay策略学习率会在最高点以多项式形式衰减；
lr_scheduler="linear_decay",
)
```
包括AdamWeightDecayStrategy在内，PaddleHub还提供了多种优化策略的API。
|优化策略|描述|API示例|
|:--------:|:--------|:--------|
|DefaultFinetuneStrategy|默认的优化策略。其对应参数如下：<br>* learning_rate: 全局学习率。默认为1e-4。<br>* optimizer_name: 优化器名称。默认adam。<br>* regularization_coeff: 正则化的λ参数。默认为1e-3。<br>在图像分类任务中推荐使用此优化策略。|strategy = hub.DefaultFinetuneStrategy(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;learning_rate=1e-4,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;optimizer_name="adam",<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;regularization_coeff=1e-3)|
|AdamWeightDecayStrategy|基于Adam优化器的学习率衰减策略。其对应参数如下：<br>* learning_rate: 全局学习率，默认为1e-4。<br>* lr_scheduler: 学习率调度方法，默认为"linear_decay"。<br>* warmup_proportion: warmup所占比重。<br>* weight_decay: 学习率衰减率。<br>* optimizer_name: 优化器名称，默认为adam。<br>在文本分类、阅读理解等任务中推荐使用此优化策略。|strategy = hub.AdamWeightDecayStrategy(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;learning_rate=1e-4,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;lr_scheduler="linear_decay",<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;warmup_proportion=0.0,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;weight_decay=0.01,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;optimizer_name="adam")|
|L2SPFinetuneStrategy|使用L2SP正则作为惩罚因子的Finetune策略。其对应参数如下：<br>* learning_rate: 全局学习率。默认为1e-4。<br>* optimizer_name: 优化器名称。默认adam。<br>* regularization_coeff: 正则化的λ参数。默认为1e-3。|strategy = hub.L2SPFinetuneStrategy(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;learning_rate=1e-4,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;optimizer_name="adam",<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;regularization_coeff=1e-3)|
|ULMFiTStrategy|该策略实现了ULMFiT论文中提出的三种策略：<br> * Slanted triangular learning rates是一种学习率先上升再下降的策略。<br>* Discriminative fine-tuning是一种学习率逐层递减的策略，通过该策略可以减缓底层的更新速度。<br>* Gradual unfreezing是一种逐层解冻的策略，通过该策略可以优先更新上层，再慢慢解冻下层参与更新。<br>其对应参数如下：<br> * learning_rate: 全局学习率。默认为1e-4。<br> * optimizer_name: 优化器名称。默认为adam。<br> * cut_fraction: 设置Slanted triangular learning rates学习率上升的步数在整个训练总步数中的比例。默认为0.1，如果设置为0，则不采用Slanted triangular learning rates。<br> * ratio: 设置Slanted triangular learning rates下降的最小学习率与上升的最大学习率的比例关系，默认为32，表示最小学习率是最大学习率的1/32。<br> * dis_blocks: 设置 Discriminative fine-tuning中的块数。默认为3，如果设置为0，则不采用Discriminative fine-tuning。<br> * factor: 设置Discriminative fine-tuning的衰减率。默认为2.6，表示下一层的学习率是上一层的1/2.6。<br> * frz_blocks: 设置Gradual unfreezing中的块数。块的概念同“dis_blocks”中介绍的概念。|strategy = hub.ULMFiTStrategy(<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;learning_rate=1e-4,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;optimizer_name="adam",<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cut_fraction=0.1,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ratio=32,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dis_blocks=3,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;factor=2.6,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;frz_blocks=3)|
#### **6. 设置运行配置。**
在PaddleHub中，用户可以使用Finetune API中的RunConfig配置Task进行Finetune时使用的参数，包括运行的Epoch次数、batch的大小、是否使用GPU训练等。代码示例如下所示。
```python
config = hub.RunConfig(use_cuda=True, num_epoch=3, batch_size=32, strategy=strategy)
```
#### **7. 组建训练任务。**
有了合适的预训练模型，并加载好要迁移的数据集后，我们开始组建一个Task。在PaddleHub中，Task代表了一个Fine-tune的任务。任务中包含了执行该任务相关的Program、数据读取器Reader、运行配置等内容。在这里可以找到文本分类任务对应的Task说明[TextClassifierTask](https://github.com/PaddlePaddle/PaddleHub/blob/release/v1.6/docs/reference/task/text_classify_task.md)。具体实现方案如下：
1. 获取module（PaddleHub的预训练模型）的上下文环境，包括输入和输出的变量，以及Paddle Program（可执行的模型格式）。
2. 从预训练模型的输出变量中找到特征图提取层feature_map，在feature_map后面接入一个全连接层，如下代码中通过hub.TextClassifierTask的pooled_output参数指定。
3. 网络的输入层保持不变，依然从输入层开始，如下代码中通过hub.TextClassifierTask的参数feed_list变量指定。
hub.TextClassifierTask就是通过这两个参数明确我们的截取模型网络的要求，按照这样的配置，我们截取的网络是从输入层一直到特征提取的最后一层“pooled_output”，表示我将使用截出的网络来进行迁移学习训练。
```python
# 获取Module的上下文信息，得到输入、输出以及预训练的Paddle Program副本。
# trainable设置为True时，Module中的参数在Fine-tune时也会随之训练，否则保持不变。
# 其中最大序列长度max_seq_len为可调整的参数，建议值为128，根据任务文本长度不同可以进行修改，但最大不超过512。
# 若序列长度不足，会通过padding方式补到max_seq_len, 若序列长度大于该值，则会以截断方式让序列长度为max_seq_len。
inputs, outputs, program = module.context(trainable=True, max_seq_len=128)
# 返回ERNIE/BERT模型对应的[CLS]向量,可以用于句子或句对的特征表达。
pooled_output = outputs["pooled_output"]
# feed_list的Tensor顺序不可以调整
# 指定ERNIE中的输入tensor的顺序，与ClassifyReader返回的结果一致
feed_list = [
inputs["input_ids"].name,
inputs["position_ids"].name,
inputs["segment_ids"].name,
inputs["input_mask"].name,
]
# 通过输入特征，label与迁移的类别数，可以生成适用于文本分类的迁移任务
cls_task = hub.TextClassifierTask(
data_reader=reader, # 读取数据的reader
feature=pooled_output, # 从预训练提取的特征矩阵
feed_list=feed_list, # 待feed变量的名字列表
num_classes=dataset.num_labels, # 数据集的类别数量
metrics_choices = ["acc"],
config=config) # 运行配置
```
PaddleHub预置了常见任务的Task，每种Task都有特定的应用场景并提供了对应的度量指标，满足用户的不同需求。
|Task类型|描述|任务类型|
|:--------:|:--------:|:--------:|
|ImageClassifierTask|该Task基于输入的特征，添加一个或多个全连接层来创建一个分类任务用于Fine-tune，度量指标为准确率，损失函数为交叉熵Loss。|图像分类任务|
|TextClassifierTask|该Task基于输入的特征，添加一个Dropout层，以及一个或多个全连接层来创建一个文本分类任务用于finetune，度量指标为准确率，损失函数为交叉熵Loss。|文本分类任务|
|SequenceLabelTask|该Task基于输入的特征，添加一个全连接层或者一个全连接层和CRF层来创建一个序列标注任务用于Fine-tune，度量指标为F1，损失函数为交叉熵Loss。|序列标注任务|
|MultiLabelClassifierTask|该Task基于输入的特征，添加一个或多个全连接层来创建一个多标签分类任务用于finetune，度量指标为多个标签的平均AUC，损失函数为多个标签的平均交叉熵。|多标签分类任务|
|RegressionTask|该Task基于输入的特征，添加一个Dropout层，以及一个或多个全连接层来创建一个文本回归任务用于finetune，度量指标为准确率，损失函数为均方差损失函数。|文本回归任务|
|ReadingComprehensionTask|该Task基于输入的特征，添加一个全连接层来创建一个阅读理解任务用于Fine-tune，损失函数为交叉熵Loss。|阅读理解任务|
在设定每个Task前，用户需要提前了解待迁移学习的预训练模型的输入与输出，即对应上面代码中的“feed_list”和“pooled_output”。具体的输入输出代码可以分为如下几类：
* 图像分类模型
```
input_dict, output_dict, program = module.context(trainable=True)
feature_map = output_dict["feature_map"]
feed_list = [input_dict["image"].name]
```
* 自然语言处理模型（不包括word2vec_skipgram、simnet_bow、文本匹配和文本生成几个模型）
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=128)
pooled_output = outputs["pooled_output"]
feed_list = [
inputs["input_ids"].name,
inputs["position_ids"].name,
inputs["segment_ids"].name,
inputs["input_mask"].name,
]
```
* word2vec_skipgram模型
```
inputs, outputs, program = module.context(trainable=True)
word_ids = inputs["word_ids"]
embedding = outputs["word_embs"]
```
* simnet_bow模型
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=args.max_seq_len, num_slots=2)
query = outputs["emb"]
title = outputs['emb_2']
```
* Pairwise文本匹配模型
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=args.max_seq_len, num_slots=3)
query = outputs["emb"]
left = outputs['emb_2']
right = outputs['emb_3']
```
* Pointwise文本匹配
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=args.max_seq_len, num_slots=2)
query = outputs["emb"]
title = outputs['emb_2']
```
* 文本生成模型
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=128)
pooled_output = outputs["pooled_output"]
sequence_output = outputs["sequence_output"]
```
#### **8 启动Fine-tune，使用Finetune_and_eval函数完成训练和评估。**
```python
cls_task.finetune_and_eval()
```
显示信息如下例所示。可以看到训练的评估结果，Loss值和准确率等等。
```
[2020-07-28 21:28:21,658] [ TRAIN] - step 810 / 900: loss=0.05022 acc=0.97813 [step/sec: 4.07]
[2020-07-28 21:28:24,115] [ TRAIN] - step 820 / 900: loss=0.04719 acc=0.98125 [step/sec: 4.07]
[2020-07-28 21:28:26,574] [ TRAIN] - step 830 / 900: loss=0.06895 acc=0.98125 [step/sec: 4.07]
[2020-07-28 21:28:29,035] [ TRAIN] - step 840 / 900: loss=0.07830 acc=0.97813 [step/sec: 4.07]
[2020-07-28 21:28:31,490] [ TRAIN] - step 850 / 900: loss=0.07279 acc=0.97500 [step/sec: 4.08]
[2020-07-28 21:28:33,939] [ TRAIN] - step 860 / 900: loss=0.03220 acc=0.99375 [step/sec: 4.09]
[2020-07-28 21:28:36,388] [ TRAIN] - step 870 / 900: loss=0.05016 acc=0.98750 [step/sec: 4.09]
[2020-07-28 21:28:38,840] [ TRAIN] - step 880 / 900: loss=0.05604 acc=0.98750 [step/sec: 4.08]
[2020-07-28 21:28:41,293] [ TRAIN] - step 890 / 900: loss=0.05622 acc=0.98125 [step/sec: 4.08]
[2020-07-28 21:28:43,748] [ TRAIN] - step 900 / 900: loss=0.06642 acc=0.97813 [step/sec: 4.08]
[2020-07-28 21:28:43,750] [ INFO] - Evaluation on dev dataset start
[2020-07-28 21:28:46,654] [ EVAL] - [dev dataset evaluation result] loss=0.17890 acc=0.94079 [step/sec: 13.23]
[2020-07-28 21:28:46,657] [ INFO] - Evaluation on dev dataset start
[2020-07-28 21:28:49,527] [ EVAL] - [dev dataset evaluation result] loss=0.17890 acc=0.94079 [step/sec: 13.39]
[2020-07-28 21:28:49,529] [ INFO] - Load the best model from ckpt_20200728212416/best_model
[2020-07-28 21:28:50,112] [ INFO] - Evaluation on test dataset start
[2020-07-28 21:28:52,987] [ EVAL] - [test dataset evaluation result] loss=0.14264 acc=0.94819 [step/sec: 13.36]
[2020-07-28 21:28:52,988] [ INFO] - Saving model checkpoint to ckpt_20200728212416/step_900
[2020-07-28 21:28:55,789] [ INFO] - PaddleHub finetune finished.
```
通过Fine-tune完成模型训练后，在对应的ckpt目录（CKPT_DIR）下，PaddleHub会自动保存验证集上效果最好的模型。用户可以参考如下代码进行预测，其中推理出的标签值0表示负向评价，1表示正向评价。。
```python
import numpy as np
# 待预测数据
data = [
["这个宾馆比较陈旧了，特价的房间也很一般。总体来说一般"],
["交通方便；环境很好；服务态度很好 房间较小"],
["19天硬盘就罢工了，算上运来的一周都没用上15天，可就是不能换了。唉，你说这算什么事呀！"]
]
index = 0
run_states = cls_task.predict(data=data)
results = [run_state.run_results for run_state in run_states]
for batch_result in results:
# 预测类别取最大分类概率值
batch_result = np.argmax(batch_result[0], axis=1)
for result in batch_result:
print("%s\tpredict=%s" % (data[index][0], result))
index += 1
```
预测结果如下所示。
```
[2020-07-28 18:06:45,441] [ INFO] - PaddleHub predict start
[2020-07-28 18:06:45,442] [ INFO] - The best model has been loaded
[2020-07-28 18:06:48,406] [ INFO] - PaddleHub predict finished.
这个宾馆比较陈旧了，特价的房间也很一般。总体来说一般 predict=0
交通方便；环境很好；服务态度很好 房间较小 predict=1
19天硬盘就罢工了，算上运来的一周都没用上15天，可就是不能换了。唉，你说这算什么事呀！ predict=0
```
docs/tutorial/how_to_load_data.md → docs/docs_ch/tutorial/how_to_load_data.md
浏览文件 @ 2b17b1f8
...@@ -31,7 +31,7 @@ label_list.txt的格式如下 ...@@ -31,7 +31,7 @@ label_list.txt的格式如下
``` ```
示例： 示例：
以[Flower数据集](../reference/dataset.md)为示例，train_list.txt/test_list.txt/validate_list.txt内容如下示例 以[Flower数据集](../reference/datasets.md)为示例，train_list.txt/test_list.txt/validate_list.txt内容如下示例
``` ```
roses/8050213579_48e1e7109f.jpg 0 roses/8050213579_48e1e7109f.jpg 0
sunflowers/45045003_30bbd0a142_m.jpg 3 sunflowers/45045003_30bbd0a142_m.jpg 3
......
docs/tutorial/serving.md → docs/docs_ch/tutorial/serving.md
浏览文件 @ 2b17b1f8
...@@ -99,7 +99,7 @@ $ hub serving start --config config.json ...@@ -99,7 +99,7 @@ $ hub serving start --config config.json
### Step3：利用PaddleHub Serving进行个性化开发 ### Step3：利用PaddleHub Serving进行个性化开发
使用PaddleHub Serving进行模型服务部署后，可以利用得到的接口进行开发，如对外提供web服务，或接入到应用程序中，以降低客户端预测压力，提高性能，下面展示了一个web页面demo: 使用PaddleHub Serving进行模型服务部署后，可以利用得到的接口进行开发，如对外提供web服务，或接入到应用程序中，以降低客户端预测压力，提高性能，下面展示了一个web页面demo:
![](../imgs/web_demo.png) ![](../../imgs/web_demo.png)
### Step4：关闭serving ### Step4：关闭serving
...@@ -147,7 +147,7 @@ $ hub serving start -c serving_config.json ...@@ -147,7 +147,7 @@ $ hub serving start -c serving_config.json
``` ```
启动成功界面如图： 启动成功界面如图：
![](../imgs/start_serving_lac.png) ![](../../imgs/start_serving_lac.png)
这样我们就在8866端口成功部署了lac的在线分词服务。 这样我们就在8866端口成功部署了lac的在线分词服务。
*此处warning为Flask提示，不影响使用* *此处warning为Flask提示，不影响使用*
...@@ -227,13 +227,11 @@ $ PaddleHub Serving will stop. ...@@ -227,13 +227,11 @@ $ PaddleHub Serving will stop.
## Demo——其他模型的一键部署服务 ## Demo——其他模型的一键部署服务
* [中文词法分析-基于lac](../../demo/serving/module_serving/lexical_analysis_lac) * [中文词法分析-基于lac](../../../demo/serving/module_serving/lexical_analysis_lac)
&emsp;&emsp;该示例展示了利用lac完成中文文本分词服务化部署和在线预测，获取文本的分词结果，并可通过用户自定义词典干预分词结果。 &emsp;&emsp;该示例展示了利用lac完成中文文本分词服务化部署和在线预测，获取文本的分词结果，并可通过用户自定义词典干预分词结果。
* [人脸检测-基于pyramidbox_lite_server_mask](../../demo/serving/module_serving/object_detection_pyramidbox_lite_server_mask) * [人脸检测-基于pyramidbox_lite_server_mask](../../../demo/serving/module_serving/object_detection_pyramidbox_lite_server_mask)
&emsp;&emsp;该示例展示了利用pyramidbox_lite_server_mask完成人脸口罩检测，检测人脸位置以及戴口罩的置信度。 &emsp;&emsp;该示例展示了利用pyramidbox_lite_server_mask完成人脸口罩检测，检测人脸位置以及戴口罩的置信度。
## Bert Service
除了预训练模型一键服务部署功能之外，PaddleHub Serving还具有`Bert Service`功能，支持ernie_tiny、bert等模型快速部署，对外提供可靠的在线embedding服务，具体信息请参见[Bert Service](./bert_service.md)。
docs/docs_en/contribution/contri_index_en.rst 0 → 100644
浏览文件 @ 2b17b1f8
Contribution Code
==================
Contributors, especially code and pre-training model contributors, are very welcome at PaddleHub. The following document describes how to contribute code and pre-training models.
... toctree::
:maxdepth: 1
Contribution code <contri_pr.md>
Contribute to the pre-training model <contri_pretrained_model>.
docs/docs_en/contribution/contri_pr_en.md 0 → 100644
浏览文件 @ 2b17b1f8
# Contribution Code
PaddleHub welcomes contributors.
First of all, feel free to submit a question or pull request if there is something you are unsure about. No one will complain about it. We appreciate contributions of any kind, and don't want to block them with a bunch of rules.
This document includes all the key points to keep in mind when making contributions. This will speed up the process of merging code and solving problems.
Click Overview for an initial overview.
Here are some simple guidelines for making contributions.
## Submit a Question
When you encounter a problem with PaddleHub, you can provide feedback by submitting [issue](https://github.com/PaddlePaddle/PaddleHub/issues).
When asking your question, specify the following:
* Fill in the details of the problem according to the problem template so that the reviewer can find the cause of the problem.
* Problem scenarios (as detailed as possible to reproduce the problem):
* Error and log messages.
* Other details that may be useful.
## Submit new feature suggestions/bug fixing
* When adapting a usage scenario, there is always a need for new features. You can either join the discussion of new features, or submit a Pull-Request for new features directly.
* Fork PaddleHub (https://github.com/PaddlePaddle/PaddleHub) under your own github account. After fork, use the git tools (add, commit, pull, push) to submit the PR. Then you can submit the pull-request.
To do the PR, follow the following steps:
### Step 1: Clone the PaddleHub remote repository in your own directory to your local.
```
https://github.com/USERNAME/PaddleHub
```
### Step 2: Switch to the remote branch develop.
```
git checkout develop
```
### Step 3: Create a local branch new-feature based on remote branch develop.
```
git checkout -b new-feature
```
### Step 4: Use the pre-commit hook.
PaddleHub developers use the pre-commit tool to manage Git pre-commit hooks. It helps us to format the source Python and automatically check some basic things before committing (for example, only one EOL per file, and not adding large files in Git).
The pre-commit test is a part of the unit tests in Travis-CI. PRs that do not meet the hooks cannot be committed to Paddle. First install and run it in the current directory.
```shell
➜ pip install pre-commit
➜ pre-commit install
```
### Step 5: Develop your requirements on the new-feature branch and commit your changes.
```
git commit -m "add new feature"
```
### Step 6: Before you are ready to launch a Pull Request, you need to synchronize the latest codes from the original repository (https://github.com/PaddlePaddle/PaddleHub).
Check the name of the current remote repository via git remote.
```shell
➜ git remote
origin
➜ git remote -v
origin https://github.com/USERNAME/PaddleHub (fetch)
origin https://github.com/USERNAME/PaddleHub (push)
```
Here, origin is PaddleHub under your own username. Next, create a remote host of the original PaddleHub repository and name it upstream.
```shell
➜ git remote add upstream https://github.com/PaddlePaddle/PaddleHub
➜ git remote
origin
upstream
```
Get the latest code for upstream and update the current branch.
```shell
➜ git fetch upstream
➜ git pull upstream develop
```
### Step 7: Push a local branch new-feature to your own PaddleHub repository.
```
➜ git push origin new-feature
```
So, the new-feature branch of your PaddleHub repository contains your latest changes, click "pull request" above to push the request.
If the reviewer gives you feedback that you need to continue fixing the codes, you can start over from step 5 so that all commits are displayed in the same pull request.
## Code Style and Naming Conventions
* PaddleHub follows the Python code naming convention of [PEP8](https://www.python.org/dev/peps/pep-0008/). Try to follow this specification when submitting pull requests. You can use the `flake8` or `pylint` hint tool to help you follow this specification.
## Document
Documents are generated using [sphinx](http://sphinx-doc.org/). Files support the [Markdown](https://guides.github.com/features/mastering-markdown/) and [reStructuredText](http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html) formats. All documents are in the [docs/](../../) directory.
* Before submitting document changes, please Generate documents locally: `cd docs/ && make clean && make html` and then, all generated pages can be found in `docs/_build/html` directory. Please carefully analyze Each WARNING in generated logs, which is very likely or Empty connections or other problems.
* Try to use Relative Path when you need links.
docs/docs_en/contribution/contri_pretrained_model_en.md 0 → 100644
浏览文件 @ 2b17b1f8
# How to Write a PaddleHub Module and Go Live
## I. Preparation
### Basic Model Information
We are going to write a PaddleHub Module with the following basic information about the module:
```yaml
name: senta_test
version: 1.0.0
summary: This is a PaddleHub Module. Just for test.
author: anonymous
author_email:
type: nlp/sentiment_analysis
```
**This sample code can be referred to as [senta\_module\_sample](../../demo/senta_module_sample/senta_test)**
The Module has an interface sentiment\_classify, which is used to receive incoming text and give it a sentiment preference (positive/negative). It supports python interface calls and command line calls.
```python
import paddlehub as hub
senta_test = hub.Module(name="senta_test")
senta_test.sentiment_classify(texts=["这部电影太差劲了"])
```
```cmd
hub run senta_test --input_text 这部电影太差劲了
```
<br/>
### Strategy
For the sake of simplicity of the sample codes, we use a very simple sentiment strategy. When the input text has the word specified in the vocabulary list, the text tendency is judged to be negative; otherwise it is positive.
<br/>
## II. Create Module
### Step 1: Create the necessary directories and files.
Create a senta\_test directory. Then, create module.py, processor.py, and vocab.list in the senta\_test directory, respectively.
| File Name | Purpose |
| ------------ | ------------------------------------------------------------ |
| module.py | It is the main module that provides the implementation codes of Module. |
| processor.py | It is the helper module that provides a way to load the vocabulary list. |
| vocab.list | It stores the vocabulary. |
```cmd
➜ tree senta_test
senta_test/
├── vocab.list
├── module.py
└── processor.py
```
### Step 2: Implement the helper module processor.
Implement a load\_vocab interface in processor.py to read the vocabulary list.
```python
def load_vocab(vocab_path):
with open(vocab_path) as file:
return file.read().split()
```
### Step 3: Write Module processing codes.
The module.py file is the place where the Module entry code is located. We need to implement prediction logic on it.
#### Step 3\_1. Reference the necessary header files
```python
import argparse
import os
import paddlehub as hub
from paddlehub.module.module import runnable, moduleinfo
from senta_test.processor import load_vocab
```
**NOTE:** When referencing a module in Module, you need to enter the full path, for example, senta\_test. processor.
#### Step 3\_2. Define the SentaTest class.
Module.py needs to have a class that inherits hub. Module, and this class is responsible for implementing the prediction logic and filling in basic information with using moduleinfo. When the hub. Module(name="senta\_test") is used to load Module, PaddleHub automatically creates an object of SentaTest and return it.
```python
@moduleinfo(
name="senta_test",
version="1.0.0",
summary="This is a PaddleHub Module. Just for test.",
author="anonymous",
author_email="",
type="nlp/sentiment_analysis",
)
class SentaTest:
...
```
#### Step 3\_3. Perform necessary initialization.
```python
def __init__(self):
# add arg parser
self.parser = argparse.ArgumentParser(
description="Run the senta_test module.",
prog='hub run senta_test',
usage='%(prog)s',
add_help=True)
self.parser.add_argument(
'--input_text', type=str, default=None, help="text to predict")
# load word dict
vocab_path = os.path.join(self.directory, "vocab.list")
self.vocab = load_vocab(vocab_path)
```
`注意`: The execution class object has a built-in directory attribute by default. You can directly get the path of the Module.
#### Step 3\_4: Refine the prediction logic.
```python
def sentiment_classify(self, texts):
results = []
for text in texts:
sentiment = "positive"
for word in self.vocab:
if word in text:
sentiment = "negative"
break
results.append({"text":text, "sentiment":sentiment})
return results
```
#### Step 3\_5. Support the command-line invoke.
If you want the module to support command-line invoke, you need to provide a runnable modified interface that parses the incoming data, makes prediction, and returns the results.
If you don't want to provide command-line prediction, you can leave the interface alone and PaddleHub automatically finds out that the module does not support command-line methods and gives a hint when PaddleHub executes in command lines.
```python
@runnable
def run_cmd(self, argvs):
args = self.parser.parse_args(argvs)
texts = [args.input_text]
return self.sentiment_classify(texts)
```
#### step 3\_6. Support the serving invoke.
If you want the module to support the PaddleHub Serving deployment prediction service, you need to provide a serving-modified interface that parses the incoming data, makes prediction, and returns the results.
If you do not want to provide the PaddleHub Serving deployment prediction service, you do not need to add the serving modification.
```python
@serving
def sentiment_classify(self, texts):
results = []
for text in texts:
sentiment = "positive"
for word in self.vocab:
if word in text:
sentiment = "negative"
break
results.append({"text":text, "sentiment":sentiment})
return results
```
### Complete Code
* [module.py](../../../modules/demo/senta_test/module.py)
* [processor.py](../../../modules/demo/senta_test/processor.py)
<br/>
## III. Install and test Module.
After writing a module, we can test it in the following ways:
### Call Method 1
Install the Module into the local machine, and then load it through Hub.Module(name=...)
```shell
➜ hub install senta_test
```
```python
import paddlehub as hub
senta_test = hub.Module(name="senta_test")
senta_test.sentiment_classify(texts=["这部电影太差劲了"])
```
### Call Method 2
Load directly through Hub.Module(directory=...)
```python
import paddlehub as hub
senta_test = hub.Module(directory="senta_test/")
senta_test.sentiment_classify(texts=["这部电影太差劲了"])
```
### Call Method 3
Load SentaTest object directly by adding senta\_test as a path to the environment variable.
```shell
➜ export PYTHONPATH=senta_test:$PYTHONPATH
```
```python
from senta_test.module import SentaTest
SentaTest.sentiment_classify(texts=["这部电影太差劲了"])
```
### Call Method 4
Install the Module on the local machine and run it through hub run.
```shell
➜ hub install senta_test
➜ hub run senta_test --input_text "这部电影太差劲了"
```
## IV. Release Module
After completing the development and testing of the module, if you want to share the model with others, you can release the model in the following ways.
### Method 1: Upload the Module to the PaddleHub website.
https://www.paddlepaddle.org.cn/hub
We will complete the review of the module and give feedback in the shortest possible time. After passing the review and going online, the module will be displayed on the PaddleHub website, and users can load it like any other official modules.
### Method 2: Upload the Module to the remote code hosting platform.
PaddleHub also supports loading Modules directly to the remote code hosting platforms. The steps are as follows:
#### Step 1: Create a new repository.
To create a new Git repository on the code hosting platform, add the codes of the module we wrote earlier. To make it easier to manage different modules, we create a modules directory and put senta\_test in the modules directory.
#### Step 2: Add a new configuration file`hubconf.py`.
In the root directory, add a new configuration `hubconf.py` file, which references a class modified by `moduleinfo` as follows:
```python
from modules.senta_test.module import SentaTest
```
*The structure of the file at this point is as follows:*
```
hubconf.py
modules
├── senta_test/
├── vocab.list
├── module.py
└── processor.py
```
#### Step 3: Complete the commit and push to the remote repository.
#### Step 4: Load Module in the remote repository locally.
To facilitate the experience, we have stored the SentaTest codes on GitHub and Gitee. So you can directly experience the effect in the following ways:
```python
import paddlehub as hub
senta_test = hub.Module(name='senta_test', source='https://github.com/nepeplwu/myhub.git')
# senta_test = hub.Module(name='senta_test', source='https://gitee.com/nepeplwu/myhub.git')
print(senta_test.sentiment_classify(texts=["这部电影太差劲了"]))
```
docs/docs_en/figures_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## Detailed Features
<a name="Various Pre-training Models"></a>
### 1\. Various Pre-training Models
#### 1.1. Image
| | **Examples of Boutique Models** |
| -------------------- | :----------------------------------------------------------- |
| Image Classification | [Dish Identification](https://www.paddlepaddle.org.cn/hubdetail?name=resnet50_vd_dishes&en_category=ImageClassification), [Animal Identification](https://www.paddlepaddle.org.cn/hubdetail?name=resnet50_vd_animals&en_category=ImageClassification), [Animal Identification](https://www.paddlepaddle.org.cn/hubdetail?name=resnet50_vd_animals&en_category=ImageClassification), [-->More](../modules/image/classification/README.md) |
| Object Detection | [Universal Detection](https://www.paddlepaddle.org.cn/hubdetail?name=yolov3_darknet53_coco2017&en_category=ObjectDetection), [Pedestrian Detection](https://www.paddlepaddle.org.cn/hubdetail?name=yolov3_darknet53_pedestrian&en_category=ObjectDetection), [Vehicle Detection](https://www.paddlepaddle.org.cn/hubdetail?name=yolov3_darknet53_vehicles&en_category=ObjectDetection), [-->More](../modules/image/object_detection/README.md) |
| Face Detection | [Face Detection](https://www.paddlepaddle.org.cn/hubdetail?name=pyramidbox_lite_server&en_category=FaceDetection), [Mask Detection](https://www.paddlepaddle.org.cn/hubdetail?name=pyramidbox_lite_server_mask&en_category=FaceDetection), [-->More](../modules/image/face_detection/README.md) |
| Image Segmentation | [Portrait Segmentation](https://www.paddlepaddle.org.cn/hubdetail?name=deeplabv3p_xception65_humanseg&en_category=ImageSegmentation), [Body Analysis](https://www.paddlepaddle.org.cn/hubdetail?name=ace2p&en_category=ImageSegmentation), [Pneumonia CT Imaging Analysis](https://www.paddlepaddle.org.cn/hubdetail?name=Pneumonia_CT_LKM_PP&en_category=ImageSegmentation), [-->More](../modules/image/semantic_segmentation/README.md) |
| Key Point Detection | [Body Key Points](https://www.paddlepaddle.org.cn/hubdetail?name=human_pose_estimation_resnet50_mpii&en_category=KeyPointDetection), [Face Key Points](https://www.paddlepaddle.org.cn/hubdetail?name=face_landmark_localization&en_category=KeyPointDetection), [Hands Key Points](https://www.paddlepaddle.org.cn/hubdetail?name=hand_pose_localization&en_category=KeyPointDetection), [-->More](./modules/image/keypoint_detection/README.md) |
| Text Recognition | [Ultra Lightweight Chinese \& English OCR Text Recognition](https://www.paddlepaddle.org.cn/hubdetail?name=chinese_ocr_db_crnn_mobile&en_category=TextRecognition), [-->More](../modules/image/text_recognition/README.md) |
| Image Generation | [Style Migration](https://www.paddlepaddle.org.cn/hubdetail?name=stylepro_artistic&en_category=GANs), [Street View Cartoon](https://www.paddlepaddle.org.cn/hubdetail?name=animegan_v2_hayao_99&en_category=GANs), [-->More](../modules/image/Image_gan/README.md) |
| Image Editing | [Super Resolution](https://www.paddlepaddle.org.cn/hubdetail?name=realsr&en_category=ImageEditing), [B\&W Color](https://www.paddlepaddle.org.cn/hubdetail?name=deoldify&en_category=ImageEditing), [-->More](../modules/image/Image_editing/README.md) |
#### 1.2 Text
| | **Examples of Boutique Models** |
| ------------------ | :----------------------------------------------------------- |
| Word Analysis | [Linguistic Analysis](https://www.paddlepaddle.org.cn/hubdetail?name=lac&en_category=LexicalAnalysis), [Syntactic Analysis](https://www.paddlepaddle.org.cn/hubdetail?name=ddparser&en_category=SyntacticAnalysis), [-->More](../modules/text/lexical_analysis/README.md) |
| Sentiment Analysis | [Emotion Judgment](https://www.paddlepaddle.org.cn/hubdetail?name=lac&en_category=LexicalAnalysis), [Emotion Analysis](https://www.paddlepaddle.org.cn/hubdetail?name=emotion_detection_textcnn&en_category=SentimentAnalysis), [-->More](../modules/text/sentiment_analysis/README.md) |
| Text Review | [Porn Review](https://www.paddlepaddle.org.cn/hubdetail?name=porn_detection_gru&en_category=TextCensorship), [-->More](../modules/text/text_review/README.md) |
| Text Generation | [Poetic Couplet Generation](https://www.paddlepaddle.org.cn/hubdetail?name=ernie_tiny_couplet&en_category=TextGeneration), [Love Letters Generation](https://www.paddlepaddle.org.cn/hubdetail?name=ernie_gen_poetry&en_category=TextGeneration), [Popular Love Letters](https://www.paddlepaddle.org.cn/hubdetail?name=ernie_gen_lover_words&en_category=TextGeneration), [-->More](../modules/text/text_generation/README.md) |
| Semantic Models | [ERNIE](https://www.paddlepaddle.org.cn/hubdetail?name=ERNIE&en_category=SemanticModel), [Text Similarity](https://www.paddlepaddle.org.cn/hubdetail?name=simnet_bow&en_category=SemanticModel), [-->More](../modules/text/language_model/README.md) |
#### 1.3. Speech
| | **Examples of Boutique Models** |
| -------------- | :-------------------------------------------------------- |
| Text-to-speech | [Text-to-speech](https://www.paddlepaddle.org.cn/hubdetail?name=deepvoice3_ljspeech&en_category=TextToSpeech), [-->More](../modules/audio/README.md) |
#### 1.4. Video
| | **Examples of Boutique Models** |
| -------------------- | :----------------------------------------------------------- |
| Video Classification | [ Video Classification](https://www.paddlepaddle.org.cn/hublist?filter=en_category&value=VideoClassification), [-->More](../modules/video/README.md) |
<a name="One-key Model Prediction"></a>
### 2\. One-key Model Prediction
* For example, if you use the lightweight Chinese OCR model chinese\_ocr\_db\_crnn\_mobile for text recognition, you can quickly recognize the text in an image with pressing one key.
```shell
$ pip install paddlehub
$ wget https://paddlehub.bj.bcebos.com/model/image/ocr/test_ocr.jpg
$ hub run chinese_ocr_db_crnn_mobile --input_path test_ocr.jpg --visualization=True
```
* The prediction results images are stored in the ocr\_result folder under the current path, as shown in the following figure.
<p align="center">
<img src="./imgs/ocr_res.jpg" width='70%' align="middle"
</p>
* Use the lexical analysis model LAC for word segmentation.
```shell
$ hub run lac --input_text "现在，慕尼黑再保险公司不仅是此类行动的倡议者，更是将其大量气候数据整合进保险产品中，并与公众共享大量天气信息，参与到新能源领域的保障中。"
[{
'word': ['现在', '，', '慕尼黑再保险公司', '不仅', '是', '此类', '行动', '的', '倡议者', '，', '更是', '将', '其', '大量', '气候', '数据', '整合', '进', '保险', '产品', '中', '，', '并', '与', '公众', '共享', '大量', '天气', '信息', '，', '参与', '到', '新能源', '领域', '的', '保障', '中', '。'],
'tag': ['TIME', 'w', 'ORG', 'c', 'v', 'r', 'n', 'u', 'n', 'w', 'd', 'p', 'r', 'a', 'n', 'n', 'v', 'v', 'n', 'n', 'f', 'w', 'c', 'p', 'n', 'v', 'a', 'n', 'n', 'w', 'v', 'v', 'n', 'n', 'u', 'vn', 'f', 'w']
}]
```
In addition to one-line code prediction, PaddleHub also supports the use of API to revoke the model. For details, refer to the detailed documentation of each model.
<a name="One-Key Model to Service"></a>
### 3\. One-Key to deploy Models as Services
PaddleHub provides convenient model-to-service capability to deploy HTTP services for models with one simple command. The LAC lexical analysis service can quickly start with the following commands:
```shell
$ hub serving start -m chinese_ocr_db_crnn_mobile
```
For more instructions on using Model Serving, See PaddleHub Model One-Key Model Serving Deployment.
<a name="Transfer Learning within ten lines of Codes"></a>
### 4\. Transfer Learning within Ten Lines of Codes
With the Fine-tune API, deep learning models can be migrated and learned in computer vision scenarios with a small number of codes.
* The [Demo Examples](../demo) provides rich codes for using Fine-tune API, including [Image Classification](../demo/image_classification), [Image Coloring](../demo/colorization), [Style Migration](../demo/style_transfer), and other scenario model migration examples.
<p align="center">
<img src="../../imgs/paddlehub_finetune.gif" align="middle"
</p>
<p align='center'>
Transfer Learning within Ten Lines of Codes
</p>
* For a quick online experience, click [PaddleHub Tutorial Collection](https://aistudio.baidu.com/aistudio/projectdetail/231146) to use the GPU computing power provided by AI Studio platform for a quick attempt.
docs/docs_en/installation_en.md 0 → 100644
浏览文件 @ 2b17b1f8
# Installation
## Environment Dependence
* Python>=3.6
* PaddlePaddle>=2.0.0rc
* Operating System: Windows/Mac/Linux
## Installation Command
Before installing the PaddleHub, install the PaddlePaddle deep learning framework first. For more installation instructions, refer to [PaddleQuickInstall](https://github.com/PaddlePaddle/PaddleHub)
```shell
pip install paddlehub==2.0.0b
```
In addition to the above dependences, PaddleHub's pre-training models and pre-set datasets need to be downloaded through connecting to the server. Make sure that the computer can access the network. You can run PaddleHub offline if the relevant datasets and pre-set models are already available locally.
**NOTE:** Make sure that the computer can access the external network when the PaddleHub is used to download datasets and pre-training models. You can use `server_check()` to check the connection status between the local and remote PaddleHub-Server in the following methods:
```python
import paddlehub
paddlehub.server_check()
# If OK, reply "Request Hub-Server successfully".
# If not OK, reply "Hub-Server unsuccessfully".
```
docs/docs_en/quick_experience/cmd_quick_run_en.md 0 → 100644
浏览文件 @ 2b17b1f8
# Using PaddleHub through Command Line Execution
The codes/commands on this page can run online on [AIStudio](https://aistudio.baidu.com/aistudio/projectdetail/643120). It is similar to the notebook environment, which can be accessed through a browser, without environment preparation. This is a quick and easy experiences for developers.
PaddleHub is designed to provide the command line tool for model management and usage. It also provides a method of completing predictions by executing the PaddleHub model in the command line. For example, the tasks of portrait segmentation and text word segmentation in the previous sections can also be implemented through command line execution.
### Before experience, install the PaddleHub.
```shell
# Lastest version
$ pip install paddlehub --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple
```
### Portrait Cutout
```shell
# Download picture
$ wget https://paddlehub.bj.bcebos.com/resources/test_image.jpg
# Inference with command line
$ hub run deeplabv3p_xception65_humanseg --input_path test_image.jpg --visualization=True --output_dir="humanseg_output"
```
--2020-07-22 12:19:52-- https://paddlehub.bj.bcebos.com/resources/test_image.jpg
Resolving paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)... 182.61.200.195, 182.61.200.229
Connecting to paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)|182.61.200.195|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 967120 (944K) [image/jpeg]
Saving to: ‘test_image.jpg’
test_image.jpg 100%[===================>] 944.45K 6.13MB/s in 0.2s
2020-07-22 12:19:53 (6.13 MB/s) - ‘test_image.jpg’ saved [967120/967120]
[{'save_path': 'humanseg_output/test_image.png', 'data': array([[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]], dtype=float32)}]
![png](../../imgs/humanseg_test_res.png)
### Chinese word segmentation
```shell
# Inference with command line
$ hub run lac --input_text "今天是个好日子"
```
Install Module lac
Downloading lac
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpjcskpj8x/lac
[==================================================] 100.00%
Successfully installed lac-2.1.1
[{'word': ['今天', '是', '个', '好日子'], 'tag': ['TIME', 'v', 'q', 'n']}]
The above command contains four parts:
- Hub: indicates the PaddleHub command.
- Run: invokes run to execute the model prediction.
- deeplabv3p\_xception65\_humanseg and lac: Indicate the algorithm model to be executed.
- --input\_path/-input\_text: Indicates the input data for the model, with different input methods for images and text.
In addition, the command line `visualization=True` indicates the visual output of the results, and `output_dir="humanseg_output"` indicates the directory where the prediction results are saved. You can access this path to view the output images.
Let's look at an example of OCR and mask detection.
### OCR
```shell
# Download picture
$ wget https://paddlehub.bj.bcebos.com/model/image/ocr/test_ocr.jpg
# requirements
$ pip install shapely
$ pip install pyclipper
# Inference with command line
$ hub run chinese_ocr_db_crnn_mobile --input_path test_ocr.jpg --visualization=True --output_dir='ocr_result'
```
--2020-07-22 15:00:50-- https://paddlehub.bj.bcebos.com/model/image/ocr/test_ocr.jpg
Resolving paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)... 182.61.200.195, 182.61.200.229
Connecting to paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)|182.61.200.195|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 48680 (48K) [image/jpeg]
Saving to: ‘test_ocr.jpg’
test_ocr.jpg 100%[===================>] 47.54K --.-KB/s in 0.02s
2020-07-22 15:00:51 (2.88 MB/s) - ‘test_ocr.jpg’ saved [48680/48680]
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Requirement already satisfied: shapely in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (1.7.0)
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Requirement already satisfied: pyclipper in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (1.2.0)
[{'save_path': 'ocr_result/ndarray_1595401261.294494.jpg', 'data': [{'text': '纯臻营养护发素', 'confidence': 0.9438689351081848, 'text_box_position': [[24, 36], [304, 34], [304, 72], [24, 74]]}, {'text': '产品信息/参数', 'confidence': 0.9843138456344604, 'text_box_position': [[24, 80], [172, 80], [172, 104], [24, 104]]}, {'text': '（45元/每公斤，100公斤起订）', 'confidence': 0.9210420250892639, 'text_box_position': [[24, 109], [333, 109], [333, 136], [24, 136]]}, {'text': '每瓶22元，1000瓶起订）', 'confidence': 0.9685984253883362, 'text_box_position': [[22, 139], [283, 139], [283, 166], [22, 166]]}, {'text': '【品牌】', 'confidence': 0.9527574181556702, 'text_box_position': [[22, 174], [85, 174], [85, 198], [22, 198]]}, {'text': '：代加工方式/OEMODM', 'confidence': 0.9442129135131836, 'text_box_position': [[90, 176], [301, 176], [301, 196], [90, 196]]}, {'text': '【品名】', 'confidence': 0.8793742060661316, 'text_box_position': [[23, 205], [85, 205], [85, 229], [23, 229]]}, {'text': '：纯臻营养护发素', 'confidence': 0.9230973124504089, 'text_box_position': [[95, 204], [235, 206], [235, 229], [95, 227]]}, {'text': '【产品编号】', 'confidence': 0.9311650395393372, 'text_box_position': [[24, 238], [120, 238], [120, 260], [24, 260]]}, {'text': 'J：YM-X-3011', 'confidence': 0.8866629004478455, 'text_box_position': [[110, 239], [239, 239], [239, 256], [110, 256]]}, {'text': 'ODMOEM', 'confidence': 0.9916308522224426, 'text_box_position': [[414, 233], [430, 233], [430, 304], [414, 304]]}, {'text': '【净含量】：220ml', 'confidence': 0.8709315657615662, 'text_box_position': [[23, 268], [181, 268], [181, 292], [23, 292]]}, {'text': '【适用人群】', 'confidence': 0.9589888453483582, 'text_box_position': [[24, 301], [118, 301], [118, 321], [24, 321]]}, {'text': '：适合所有肤质', 'confidence': 0.935418963432312, 'text_box_position': [[131, 300], [254, 300], [254, 323], [131, 323]]}, {'text': '【主要成分】', 'confidence': 0.9366627335548401, 'text_box_position': [[24, 332], [117, 332], [117, 353], [24, 353]]}, {'text': '鲸蜡硬脂醇', 'confidence': 0.9033458828926086, 'text_box_position': [[138, 331], [235, 331], [235, 351], [138, 351]]}, {'text': '燕麦B-葡聚', 'confidence': 0.8497812747955322, 'text_box_position': [[248, 332], [345, 332], [345, 352], [248, 352]]}, {'text': '椰油酰胺丙基甜菜碱、', 'confidence': 0.8935506939888, 'text_box_position': [[54, 363], [232, 363], [232, 383], [54, 383]]}, {'text': '糖、', 'confidence': 0.8750994205474854, 'text_box_position': [[25, 364], [62, 364], [62, 383], [25, 383]]}, {'text': '泛酯', 'confidence': 0.5581164956092834, 'text_box_position': [[244, 363], [281, 363], [281, 382], [244, 382]]}, {'text': '（成品包材）', 'confidence': 0.9566792845726013, 'text_box_position': [[368, 367], [475, 367], [475, 388], [368, 388]]}, {'text': '【主要功能】', 'confidence': 0.9493741393089294, 'text_box_position': [[24, 395], [119, 395], [119, 416], [24, 416]]}, {'text': '：可紧致头发磷层', 'confidence': 0.9692543745040894, 'text_box_position': [[128, 397], [273, 397], [273, 414], [128, 414]]}, {'text': '美，从而达到', 'confidence': 0.8662520051002502, 'text_box_position': [[265, 395], [361, 395], [361, 415], [265, 415]]}, {'text': '即时持久改善头发光泽的效果，给干燥的头', 'confidence': 0.9690631031990051, 'text_box_position': [[25, 425], [372, 425], [372, 448], [25, 448]]}, {'text': '发足够的滋养', 'confidence': 0.8946213126182556, 'text_box_position': [[26, 457], [136, 457], [136, 477], [26, 477]]}]}]
```shell
# check the result
```
![png](../../imgs/ocr_res.jpg)
### Mask Detection
```shell
# Download picture
$ wget https://paddlehub.bj.bcebos.com/resources/test_mask_detection.jpg
# Inference with command line
$ hub run pyramidbox_lite_mobile_mask --input_path test_mask_detection.jpg --visualization=True --output_dir='detection_result'
```
--2020-07-22 15:08:11-- https://paddlehub.bj.bcebos.com/resources/test_mask_detection.jpg
Resolving paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)... 182.61.200.229, 182.61.200.195
Connecting to paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)|182.61.200.229|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 299133 (292K) [image/jpeg]
Saving to: ‘test_mask_detection.jpg’
test_mask_detection 100%[===================>] 292.12K --.-KB/s in 0.06s
2020-07-22 15:08:11 (4.55 MB/s) - ‘test_mask_detection.jpg’ saved [299133/299133]
Install Module pyramidbox_lite_mobile_mask
Downloading pyramidbox_lite_mobile_mask
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmp8oes9jid/pyramidbox_lite_mobile_mask
[==================================================] 100.00%
Successfully installed pyramidbox_lite_mobile_mask-1.3.0
Downloading pyramidbox_lite_mobile
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpvhjhlr10/pyramidbox_lite_mobile
[==================================================] 100.00%
[{'data': [{'label': 'MASK', 'confidence': 0.9992434978485107, 'top': 181, 'bottom': 440, 'left': 457, 'right': 654}, {'label': 'MASK', 'confidence': 0.9224318265914917, 'top': 340, 'bottom': 578, 'left': 945, 'right': 1125}, {'label': 'NO MASK', 'confidence': 0.9996706247329712, 'top': 292, 'bottom': 500, 'left': 1166, 'right': 1323}], 'path': 'test_mask_detection.jpg'}]
```shell
# check the result
```
![png](../../imgs/test_mask_detection_result.jpg)
### This is the introduction to the PaddleHub command line tool.
The command line tool of PaddleHub is developed with introducing package management concepts such as Anaconda and PIP. It can be used to search, download, install, upgrade, and predict models quickly and conveniently. The following is an overview of the 12 commands supported by PaddleHub. For details, see Command Line Reference :
* install: Installs the module locally, by default, in {HUB\_HOME}/.paddlehub/modules directory.
* uninstall: Uninstalls the local module.
* show: Views the properties of locally installed module or the properties of a module identified in the specified directory, including its name, version, description, author and other information.
* download: Downloads the module provided by PaddleHub of Baidu.
* search: Searches for matching Modules on the server by keywords. When you want to find a Module of a specific model, run search command to get fast results. For example, hub search ssd: the command runs to search all Modules that contain ssd. The command supports regular expressions, for example, hub search \^s.\*: the command runs to search all resources beginning with s.
* list: Lists locally installed module.
* run: Executes the predictions of the module.
* version: displays PaddleHub version information.
* help: displays help information.
* clear: PaddleHub generates some cached data during operation, which is stored in ${HUB\_HOME}/.paddlehub/cache by default. Users can clear the cache by running the clear command.
* autofinetune: Automatically adjusts the hyper-parameters of Fine-tune tasks. For details, see [PaddleHub AutoDL Finetuner](https://github.com/PaddlePaddle/PaddleHub/blob/release/v1.5/docs/tutorial/autofinetune.md) Tutorial.
* config: Views and sets Paddlehub-related settings, including settings of server address and log level.
* serving: Deploys Module Prediction Services in one key. For details, see PaddleHub Serving One-Key Service Deployment.
## Summary
According to the PaddleHub product concept, the model is the software. It is executed through Python API or command line to allow you to quickly experience or integrate the pre-training models with the Paddle characteristics. In addition, when users want to optimize pre-training models with small amounts of data, PaddleHub also supports migration learning. Through the Fine-tune API and a variety of built-in optimization strategies, only a small number of codes is needed to complete the fine-tuning of pre-training models. You can learn more about this later in the chapter on Transfer Learning.
> It should be noted that not all modules support prediction through command line (for example, BERT/ERNIE Transformer class models, which generally require fine-tuning with tasks), and not all modules can be used for fine-tuning (for example, it is not recommended to use the fine-tune of lexical analysis LAC models). It is recommended to read the Introduction to the pre-training model to understand the usage scenarios.
docs/docs_en/quick_experience/more_demos_en.md 0 → 100644
浏览文件 @ 2b17b1f8
# More Experience Demos in PaddleHub
## PaddleHub Official Demo Collection \[Offical].
The official demo is available online in AI Studio. It is similar to notebook environments, which can be accessed through a browser. You do not need to prepare an environment, and it is very convenient for developers to experience quickly. It will be updated continuously, and we recommend to bookmark it. [https://aistudio.baidu.com/aistudio/personalcenter/thirdview/79927](https://aistudio.baidu.com/aistudio/personalcenter/thirdview/79927)
## PaddleHub Developer Fun Practice \[ThirdParty]
The following are some of the interesting and practical works created by developers based on PaddleHub in previous PaddleHub courses or events, which are all included in AI Studio and can run online.
1. [Cloth Scissors Rock \[Face Recognition to Switch Local Window] ](https://aistudio.baidu.com/aistudio/projectdetail/507630)
2. [Fish in the Autumn Water \[yesok dance background cutout conversion \& anime style migration] ](http://aistudio.baidu.com/aistudio/projectdetail/517066)
3. [Ninetailskim \[play retro windows pinball on people's faces] ](https://aistudio.baidu.com/aistudio/projectdetail/518861)
4. [Ura\_\_ \[Monitor Mask, Voice Alerts, Background Recording] ](https://aistudio.baidu.com/aistudio/projectdetail/506931)
5. [Nine Alchemists \[Shadow stream's Green Frog Cai Xukun, Face Recognition + Headgear + Portrait Segmentation ](https://aistudio.baidu.com/aistudio/projectdetail/505168)
6. [Seven-year term \[style migration and local deployment] ](https://aistudio.baidu.com/aistudio/projectdetail/520453)
7. [Fanas Invincible \[Lipstick Color Test Project]](https://aistudio.baidu.com/aistudio/projectdetail/516520)
8. [skywalk163 \[statistics of source code word frequency and word cloud \& portrait display with paddlehub] ](https://aistudio.baidu.com/aistudio/projectdetail/519841)
9. [AIStudio261428 \[Face Recognition + Cartoon Emoticons]](https://aistudio.baidu.com/aistudio/projectdetail/519616)
10. [Tudou Sprouts \[Invite Cartoon Characters to Visit on Children's Day]](https://aistudio.baidu.com/aistudio/projectdetail/520925)
11. [Panda Feeling \[changing masks] ](https://aistudio.baidu.com/aistudio/projectdetail/520996)
12. [Kly1997 \[One-key Travel + Sunglasses]](https://aistudio.baidu.com/aistudio/projectdetail/518117)
13. [Loneliness\_Memo \[Crossing over to the painting] ](https://aistudio.baidu.com/aistudio/projectdetail/516332)
14. [isse7 \[Creative Projects: Style "Ghostface" Change]](https://aistudio.baidu.com/aistudio/projectdetail/515307)
15. [Pda \[Face Fun Change] ](https://aistudio.baidu.com/aistudio/projectdetail/516306)
16. [Kgkzhiwen \[My New Clothes]](https://aistudio.baidu.com/aistudio/projectdetail/516663)
17. [Ah Ah Ah Good Good Study \[Automatic face shape adjustment]](https://aistudio.baidu.com/aistudio/projectdetail/513640)
18. [Tfboy \[ID Photo Replacement](https://aistudio.baidu.com/aistudio/projectdetail/509443)
19. [Leigangblog \[I am a star face]](https://aistudio.baidu.com/aistudio/projectdetail/505537)
20. [wpb3dm \[Fashion Model Dressup] wpb3dm](https://aistudio.baidu.com/aistudio/projectdetail/519349)
21. [lsvine\_bai \[Girlfriend turns into mysterious blonde goddess in seconds]](https://aistudio.baidu.com/aistudio/projectdetail/521784)
22. [Lemonadeqk \[Easy Stargazing]](https://aistudio.baidu.com/aistudio/projectdetail/520488)
23. [XM1436gr \[AI for Cartoon Face using PaddleHub Keypoint Detection] ](https://aistudio.baidu.com/aistudio/projectdetail/514547)
24. [Wantzai \[Everyone is a round-eyed cute boy] ](https://aistudio.baidu.com/aistudio/projectdetail/519222)
25. [Arrowarcher \[AI One Key Hair chane] ](https://aistudio.baidu.com/aistudio/projectdetail/508270)
26. [WHY197598 \[Fundamentals of Shifting Objects] ](https://aistudio.baidu.com/aistudio/projectdetail/517961)
27. [SIGNED BY JINGYI \[Fatigue detection based on paddlehub face key point detection]](https://aistudio.baidu.com/aistudio/projectdetail/506024)
28. [thunder95 \[PaddleHub Eyes Emotion Poll] ](https://aistudio.baidu.com/aistudio/projectdetail/514205)
29. [Windy Moon C \[Grave Bouncing Graduation Photo]](https://aistudio.baidu.com/aistudio/projectdetail/511253)
30. [Ruyi\_Egg \[left looks like Chow Yun-Fat, right looks like Andy Lau] ](https://aistudio.baidu.com/aistudio/projectdetail/507231)
docs/docs_en/quick_experience/python_use_hub_en.md 0 → 100644
浏览文件 @ 2b17b1f8
# Using PaddleHub through Python Code Execution
The codes/commands on this page can run online on [AIStudio](https://aistudio.baidu.com/aistudio/projectdetail/635335). It is similar to the notebook environment, which can be accessed through a browser, without environment preparation. This is a quick and easy experiences for developers.
## PaddleHub example for computer vision tasks
Taking the computer vision task as an example, we choose a test image test.jpg to implement the following four functions:
* Portrait Cutout ([deeplabv3p\_xception65\_humanseg](https://www.paddlepaddle.org.cn/hubdetail?name=deeplabv3p_xception65_humanseg&en_category=ImageSegmentation))
* Body Part Segmentation ([ace2p](https://www.paddlepaddle.org.cn/hubdetail?name=ace2p&en_category=ImageSegmentation))
* Face Detection ([ultra\_light\_fast\_generic\_face\_detector\_1mb\_640](https://www.paddlepaddle.org.cn/hubdetail?name=ultra_light_fast_generic_face_detector_1mb_640&en_category=FaceDetection))
* Key Point Detection ([human\_pose\_estimation\_resnet50\_mpii](https://www.paddlepaddle.org.cn/hubdetail?name=human_pose_estimation_resnet50_mpii&en_category=KeyPointDetection))
> Note: If you need to find out which pre-training models can be executed from PaddleHub, get the model name (for example, deeplabv3p\_xception65\_humanseg, by which the model will be executed in subsequent codes). For details, see official website document. In the documents, it is easy to search because models are classified according to model categories, with providing a detailed description of the models.
### Before experience, install the PaddleHub.
```shell
# install the latest version
$ pip install paddlehub --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple
```
### Original Image Display
```shell
# Download a picture
$ wget https://paddlehub.bj.bcebos.com/resources/test_image.jpg
```
--2020-07-22 12:22:19-- https://paddlehub.bj.bcebos.com/resources/test_image.jpg
Resolving paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)... 182.61.200.195, 182.61.200.229
Connecting to paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)|182.61.200.195|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 967120 (944K) [image/jpeg]
Saving to: ‘test_image.jpg.1’
test_image.jpg.1 100%[===================>] 944.45K 5.51MB/s in 0.2s
2020-07-22 12:22:19 (5.51 MB/s) - ‘test_image.jpg.1’ saved [967120/967120]
![png](../../imgs/humanseg_test.png)
### Portrait Cutout
PaddleHub adopts the model-based software design concept. All pre-training models are similar to Python packages, with the concept of version. You can install, upgrade, and remove the model conveniently by running the commands `hub install` and `hub uninstall`:
> By default, you can download the model of the latest version by running the following command. If you want to specify the version, you can add the version number after the `==1.1.1` command.
```shell
#intall pre-trained models
$ hub install deeplabv3p_xception65_humanseg
```
Downloading deeplabv3p_xception65_humanseg
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpo32jeve0/deeplabv3p_xception65_humanseg
[==================================================] 100.00%
Successfully installed deeplabv3p_xception65_humanseg-1.1.1
```python
# import paddlehub
import paddlehub as hub
module = hub.Module(name="deeplabv3p_xception65_humanseg")
res = module.segmentation(paths = ["./test_image.jpg"], visualization=True, output_dir='humanseg_output')
```
[32m[2020-07-22 12:22:49,474] [ INFO] - Installing deeplabv3p_xception65_humanseg module [0m
​
Downloading deeplabv3p_xception65_humanseg
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpzrrl1duq/deeplabv3p_xception65_humanseg
[==================================================] 100.00%
​
[32m[2020-07-22 12:23:11,811] [ INFO] - Successfully installed deeplabv3p_xception65_humanseg-1.1.1 [0m
![png](../../imgs/output_8_3.png)
As you can see, the execution of PaddleHub with Python codes requires only three lines of codes:
```
import paddlehub as hub
module = hub.Module(name="deeplabv3p_xception65_humanseg")
res = module.segmentation(paths = ["./test.jpg"], visualization=True, output_dir='humanseg_output')
```
* Model names are specified through the `hub.Module` API.
* `module.segmentation` is used to execute the image segmentation prediction tasks. Different prediction APIs are designed for different types of tasks. For example, the face detection task uses the `face_detection` function. It is recommended to view the corresponding model introduction document before the pre-training model is executed.
* The prediction results are saved in `output_dir='humanseg_output'` directory, and you can view the output images in this directory.
For the implementation of other tasks, refer to this pattern. Let's see how the next few tasks are implemented.
### Body Part Segmentation
```shell
$ hub install ace2p
```
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
Downloading ace2p
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpfsovt3f8/ace2p
[==================================================] 100.00%
Successfully installed ace2p-1.1.0
```python
import paddlehub as hub
module = hub.Module(name="ace2p")
res = module.segmentation(paths = ["./test_image.jpg"], visualization=True, output_dir='ace2p_output')
```
[32m[2020-07-22 12:23:58,027] [ INFO] - Installing ace2p module [0m
​
Downloading ace2p
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmptrogpj6j/ace2p
[==================================================] 100.00%
​
[32m[2020-07-22 12:24:22,575] [ INFO] - Successfully installed ace2p-1.1.0 [0m
![png](../../imgs/output_12_3.png)
### Face Detection
```shell
#install pre-trained model
$ hub install ultra_light_fast_generic_face_detector_1mb_640
```
Downloading ultra_light_fast_generic_face_detector_1mb_640
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpz82xnmy6/ultra_light_fast_generic_face_detector_1mb_640
[==================================================] 100.00%
Successfully installed ultra_light_fast_generic_face_detector_1mb_640-1.1.2
```python
import paddlehub as hub
module = hub.Module(name="ultra_light_fast_generic_face_detector_1mb_640")
res = module.face_detection(paths = ["./test_image.jpg"], visualization=True, output_dir='face_detection_output')
```
[32m[2020-07-22 12:25:12,948] [ INFO] - Installing ultra_light_fast_generic_face_detector_1mb_640 module [0m
​
Downloading ultra_light_fast_generic_face_detector_1mb_640
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpw44mo56p/ultra_light_fast_generic_face_detector_1mb_640
[==================================================] 100.00%
​
[32m[2020-07-22 12:25:14,698] [ INFO] - Successfully installed ultra_light_fast_generic_face_detector_1mb_640-1.1.2
![png](../../imgs/output_15_3.png)
### Key Point Detection
```shell
$ hub install human_pose_estimation_resnet50_mpii
```
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
Downloading human_pose_estimation_resnet50_mpii
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpn_ppwkzq/human_pose_estimation_resnet50_mpii
[======== ] 17.99%
```python
import paddlehub as hub
module = hub.Module(name="human_pose_estimation_resnet50_mpii")
res = module.keypoint_detection(paths = ["./test_image.jpg"], visualization=True, output_dir='keypoint_output')
```
[32m[2020-07-23 11:27:33,989] [ INFO] - Installing human_pose_estimation_resnet50_mpii module [0m
[32m[2020-07-23 11:27:33,992] [ INFO] - Module human_pose_estimation_resnet50_mpii already installed in /home/aistudio/.paddlehub/modules/human_pose_estimation_resnet50_mpii [0m
​
image saved in keypoint_output/test_imagetime=1595474855.jpg
![png](../../imgs/output_18_2.png)
## PaddleHub Example for Natural Language Processing Tasks
Let's look at two more examples of natural language processing tasks: Chinese word segmentation and sentiment classification tasks.
* Chinese word segmentation ([lac](https://www.paddlepaddle.org.cn/hubdetail?name=lac&en_category=LexicalAnalysis))
* Sentiment analysis ([senta\_bilstm](https://www.paddlepaddle.org.cn/hubdetail?name=senta_bilstm&en_category=SentimentAnalysis))
### Chinese word segmentation
```shell
$ hub install lac
```
2020-07-22 10:03:09,866-INFO: font search path ['/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/ttf', '/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/afm', '/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/pdfcorefonts']
2020-07-22 10:03:10,208-INFO: generated new fontManager
Downloading lac
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmp8ukaz690/lac
[==================================================] 100.00%
Successfully installed lac-2.1.1
```python
import paddlehub as hub
lac = hub.Module(name="lac")
test_text = ["1996年，曾经是微软员工的加布·纽维尔和麦克·哈灵顿一同创建了Valve软件公司。他们在1996年下半年从id software取得了雷神之锤引擎的使用许可，用来开发半条命系列。"]
res = lac.lexical_analysis(texts = test_text)
print("The resuls are: ", res)
```
[32m[2020-07-22 10:03:18,439] [ INFO] - Installing lac module[0m
[32m[2020-07-22 10:03:18,531] [ INFO] - Module lac already installed in /home/aistudio/.paddlehub/modules/lac [0m
​
The resuls are: [{'word': ['1996年', '，', '曾经', '是', '微软', '员工', '的', '加布·纽维尔', '和', '麦克·哈灵顿', '一同', '创建', '了', 'Valve软件公司', '。', '他们', '在', '1996年下半年', '从', 'id', ' ', 'software', '取得', '了', '雷神之锤', '引擎', '的', '使用', '许可', '，', '用来', '开发', '半条命', '系列', '。'], 'tag': ['TIME', 'w', 'd', 'v', 'ORG', 'n', 'u', 'PER', 'c', 'PER', 'd', 'v', 'u', 'ORG', 'w', 'r', 'p', 'TIME', 'p', 'nz', 'w', 'n', 'v', 'u', 'n', 'n', 'u', 'vn', 'vn', 'w', 'v', 'v', 'n', 'n', 'w']}]
As you can see, compared to computer vision tasks, there are differences between the input and output interfaces (where you need to enter text, executed as the function parameters), because it depends on the task types. For details, see the API description for the corresponding pre-training model.
### Sentiment classification
```shell
$ hub install senta_bilstm
```
Module senta_bilstm-1.1.0 already installed in /home/aistudio/.paddlehub/modules/senta_bilstm
```python
import paddlehub as hub
senta = hub.Module(name="senta_bilstm")
test_text = ["味道不错，确实不算太辣，适合不能吃辣的人。就在长江边上，抬头就能看到长江的风景。鸭肠、黄鳝都比较新鲜。"]
res = senta.sentiment_classify(texts = test_text)
print("情感分析结果：", res)
```
[32m[2020-07-22 10:34:06,922] [ INFO] - Installing senta_bilstm module [0m
[32m[2020-07-22 10:34:06,984] [ INFO] - Module senta_bilstm already installed in /home/aistudio/.paddlehub/modules/senta_bilstm
[32m[2020-07-22 10:34:08,937] [ INFO] - Installing lac module[0m
[32m[2020-07-22 10:34:08,939] [ INFO] - Module lac already installed in /home/aistudio/.paddlehub/modules/lac [0m
​
情感分析结果： [{'text': '味道不错，确实不算太辣，适合不能吃辣的人。就在长江边上，抬头就能看到长江的风景。鸭肠、黄鳝都比较新鲜。', 'sentiment_label': 1, 'sentiment_key': 'positive', 'positive_probs': 0.9771, 'negative_probs': 0.0229}]
## Summary
PaddleHub provides a rich set of pre-training models, including image classification, semantic model, video classification, image generation, image segmentation, text review, key point detection and other mainstream models. These can be executed quickly and easily with only 3 lines of Python codes, with the instant output of prediction results. You can try it out by selecting some models from the Pre-training Model List.
docs/docs_en/quick_experience/quick_index_en.rst 0 → 100644
浏览文件 @ 2b17b1f8
Quick Experience
==================
The PaddleHub is used in two modes: Python code execution and command line execution.
The command line mode requires only one line of command. It is perfect for quickly experiencing the effects of the pre-training models provided by PaddleHub. The Python code mode requires only three lines of codes. If you need to use your own data Fine-tune to generate models, you can use this method.
Example of command line:
::
$ hub run chinese_ocr_db_crnn_mobile --input_path test_ocr.jpg
Example of Python Codes:
::
import paddlehub as hub
ocr = hub.Module(name="chinese_ocr_db_crnn_mobile")
result = ocr.recognize_text(paths = "./test_image.jpg"], visualization=True, output_dir='ocr_output')
Example of results:
... image:: ... /imgs/ocr_res.jpg
This section provides a quick way to experience these two methods, so that you can get started quickly. It also provides a rich experience Demo, covering multiple scenarios. For more detailed explanations of the API interface parameters and command line parameters, refer to Python API Interface and Command Line.
... toctree::
:maxdepth: 1
Command line execution: PaddleHub<cmd_quick_run>
Python code execution: PaddleHub<python_use_hub>
More Experience Demos of PaddleHub: <more_demos>
docs/docs_en/tutorial/cmdintro_en.md 0 → 100644
浏览文件 @ 2b17b1f8
# PaddleHub Command Line Tool
PaddleHub provides the command line tool for the management and use of pre-training models.
PaddleHub supports the change of the path of storing the pre-training models.
* If `${HUB_HOME}`Environment Variable is set, the pre-training models and configuration files are stored in the path specified by `${HUB_HOME}`.
* If the `${HUB_HOME}`Environment variable is not set, these files are stored in the path specified by `$HOME`.
The following 11 commands are currently supported on the command line.
## `hub install`
Installs Module locally. By default, it is installed in the `${HUB_HOME}/.paddlehub/modules` directory. When a module is installed locally, users can operate the Module through other commands (e.g., use the Module for prediction), or use the python API provided by PaddleHub to apply the Module to their own tasks to achieve migration learning.
## `hub uninstall`
Uninstalls local Modules.
## `hub show`
Views properties of locally installed modules or properties of modules in a specified directory, including name, version, description, author and other information.
## `hub download`
Downloads the Module provided by PaddleHub.
## `hub search`
Search the matching Module on the server by keywords. When you want to find the Module of a specific model, you can run the search command to quickly get the result. For example, the `hub search ssd` command runs to search for all Modules containing the word ssd. The command supports regular expressions, for example, `hub search ^s.*` runs to search all resources beginning with s.
`Note` If you want to search all Modules, the running of `hub search *` does not work. Because the shell expands its own wildcard, replacing \* with the filename in the current directory. For global search, users can type `hub search` directly.
## `hub list`
Lists the locally installed Modules
## `hub run`
Executes the prediction of Module. It should be noted that not all models support prediction (and likewise, not all models support migration learning). For more details on the run command, see `About Prediction` below.
## `hub help`
Displays help information.
## `hub version`
Displays the PaddleHub version information.
## `hub clear`
PaddleHub generates some cached data in the operation, which is stored in ${HUB\_HOME}/.paddlehub/cache directory by default. Users can clear the cache by running the clear command.
## `hub config`
Views and configures paddlehub-related information, including server address and log level.
`Example`
* `hub config`: Displays the current paddlehub settings.
* `hub config reset`: Restores current paddlehub settings to default settings
* `hub config server==[address]`: Sets the current paddlehub-server address to \[address], and paddlehub client gets model information from this address.
* `hub config log.level==[level]`: Sets the current log level to \[level]. Options are CRITICAL, ERROR, WARNING, EVAL, TRAIN, INFO, DEBUG, from left to right, from high priority to low priority.
* `hub config log.enable==True|False`: Sets whether the current log is available.
## `hub serving`
Deploys Module prediction service in one key. For details, see PaddleHub Serving One-Key Service Deployment.
**NOTE:**
In PaddleHub, Module represents a executable pre-training model`可执行的预训练模型`. A Module can support direct command-line prediction, or with the PaddleHub Finetune API. It implements the migration learning through a small number of codes. Not all Modules support command line prediction (for example, for BERT/ERNIE Transformer model, finetune is performed generally with a task). Not all Modules can be used for fine-tune (for example, for the LAC lexical analysis model, we do not recommend users to use finetune).
PaddleHub tries to simplify the cost of understanding when using command line predictions. Generally, the predictions are classified into two categories: NLP and CV.
## NLP Class Tasks
You can input data specified by --input\_text or --input\_file. Take the Baidu LAC model (Chinese lexical analysis) as an example. You can use the following two commands to analyze the single-line texts and multi-line texts.
```shell
# Single-line of sentence
$ hub run lac --input_text "It is a nice day today！"
```
```shell
# multi-line of sentences
$ hub run lac --input_file test.txt
```
The sample format of test.txt is as follows (each line is a sentence requiring lexical analysis):
```
It is a nice day today！
It is going to rain outside.
……more……
```
## CV Class Tasks
Input data is specified by `--input_path` or `--input_file`. Take the SSD model (single-stage object detection) as an example. Single-image and multi-image predictions can be performed by running the following two commands:
```shell
# Single Picture
$ hub run resnet_v2_50_imagenet --input_path test.jpg
```
```shell
# List of Pictures
$ hub run resnet_v2_50_imagenet --input_file test.txt
```
The format of test.txt is
```
cat.jpg
dog.jpg
person.jpg
……more……
```
docs/docs_en/tutorial/how_to_finetune_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **Transfer Learning**
### **Overview**
Transfer Learning is a subfield of deep learning that aims to use similarities in data, tasks, or models to transfer knowledge learned in old fields to new fields. In other words, transfer learning refers to the use of existing knowledge to learn new knowledge. For example, people who have learned to ride a bicycle can learn to ride an electric bike faster. A common method of transfer learning is to perform fine-tune of a pre-training model. That is, the user selects a successfully trained model from PaddleHub for a new task based on the current task scenario, and the dataset used by the model is similar to the dataset of the new scenario. In this case, you only needs to perform fine-tune of the parameters of the model (**Fine-tune**) during the training of the current task scenario using the data of the new scenario. Transfer learning has attracted many researchers because it is a good solution to the following problems in deep learning:
* In some research areas, there are only a small amount of annotated data, and the cost of data annotation is high, which is not enough to train a sufficiently robust neural network.
* The training of large-scale neural networks relies on large computational resources, which is difficult to implement for a common user.
* Models that address generalized needs do not perform as well as expected for specific applications.
In order to make it easier for developers to apply transfer learning, Paddle has open-sourced PaddleHub – a pre-training model management tool. With just ten lines of codes, developers can complete the transfer learning process. This section describes comprehensive transfer learning by using the PaddleHub.
### **Prerequisites**
Before starting transfer learning, users need to do the following:
* The user has installed PaddleHub.
* To prepare the data for transfer learning, users can choose to use the dataset provided by PaddleHub or a custom dataset. For details of the custom data, refer to the "Custom dataset Fine-tune" to process the dataset.
* Run the hub install command to install or update the modules used for training. For example, for the ERNIE model, the command format is as follows: You may have already installed the relevant pre-training model in a previous task. However, it is still recommended that you perform this step before you start training, so that you can ensure that the pre-training model is the latest version.
```python
$ hub install ernie==1.2.0
```
### **Transfer Learning Process**
Before you can complete the transfer learning process, you need to write scripts for transfer learning. The scripting process is very simple, requiring only about ten lines of codes. The entire scripting process has the following steps:
1. Import the necessary packages.
2. Load a pre-training model, that is, load the pre-training model provided by PaddleHub.
3. Load Dataset. You can choose to load the dataset that comes with PaddleHub by using the dataset API or write your own class to load the dataset to load the custom dataset.
4. Configure the reader to pre-process the data in dataset. Organize in a specific format and input to the model for training.
5. Choose an optimization strategy that includes various pre-training parameters, such as, what learning rate variation strategy, what type of optimizer, and what type of regularization.
6. Set the RunConfig. It contains a number of training-related configurations, including whether to use the GPU, the number of training rounds (Epoch), training batch size (batch\_size), and so on.
7. Create a training task. A transfer learning training task contains the task-related Program, Reader and RunConfig set above.
8. Start the Fine-tune. Use the Fine-tune\_and\_eval function to complete the training and evaluation.
### **Learn to Write Transfer Learning Training Scripts**
PaddleHub provides a Finetune API and a pre-training model to perform transfer learning for a variety of different task scenarios, including image classification, text classification, multi-label classification, sequence annotation, retrieval quiz task, regression task, sentence semantic similarity computation, reading comprehension task, and so on. This section describes how to write transfer learning scripts by taking the text classification as an example.
#### **1\. Import the necessary packages.**
```python
import paddlehub as hub
```
#### **2\. Load the pre-training model.**
The following code is used to load the pre-training model. In this case, the ERNIE pre-training model is used to complete the text classification task. The Enhanced Representation through Knowledge Integration (ERNIE) is a semantic representation model proposed by Baidu, with Transformer Encoder as the basic component. The pre-training process uses richer semantic knowledge and more semantic tasks. Users can use the pre-training model to gradually introduce different custom tasks at any time, such as, naming entity prediction, semantic paragraph relationship recognition, sentence order prediction task, sentiment analysis, and so on.
```python
module = hub.Module(name="ernie")
```
PaddleHub also provides many other pre-training models for transfer learning. On the PaddleHub website, the pre-training models under Image Classification, Semantic Model, and Sentiment Analysis all support transfer learning. Users only need to replace the name value with the name of the pre-training model, for example, the red box on the right.
![](../imgs/Howtofinetune1.png)
#### **3\. Load the dataset.**
After loading the pre-training model, we load the dataset. The sources of datasets used for transfer learning can be divided into customized datasets and datasets provided by PaddleHub. Loading methods vary with datasets.
##### **Load PaddleHub's own datasets.**
If you are using PaddleHub's own dataset, you can use PaddleHub's dataset API to write a single line of code to load the dataset.
```python
dataset = hub.dataset.ChnSentiCorp()
```
ChnSentiCorp is a Chinese sentiment analysis dataset, and aims to determine the sentiment attitude of a paragraph of texts. For example, if the text is "The food is delicious", the corresponding label is "1", indicating a positive evaluation; or if the text is "The room is too small", the corresponding label is "0", indicating a negative evaluation. PaddleHub also provides other text classification datasets. You can choose the corresponding API to replace the values of the dataset in the above code, as listed in the table below.
| Dataset | Name | API |
| :------------------: | :----------------------------------------------------------: | :---------------------------: |
| ChnSentiCorp | Chinese Sentiment Analysis Dataset | hub.dataset.ChnSentiCorp() |
| LCQMC | A question-and-answer matching Chinese dataset constructed by Harbin Institute of Technology at the International Summit on Natural Language Processing COLING2018, with the goal of determining whether the semantics of two questions are the same. | hub.dataset.LCQMC() |
| NLPCC-DPQA | The goal of the evaluation task dataset held by the International Natural Language Processing and Chinese Computing Conference NLPCC in 2016 is to select answers that can answer the questions. | hub.dataset.NLPCC_DPQA() |
| MSRA-NER | A dataset released by Microsoft Asian Research Institute that aims to identify named entities | hub.dataset.MSRA-NER() |
| Toxic | English multi-labeled taxonomic datasets | hub.dataset.Toxic() |
| SQUAD | English reading comprehension dataset | hub.dataset.SQUAD() |
| GLUE-CoLA | Text Classification Task Dataset | hub.dataset.GLUE("CoLA") |
| GLUE-SST-2 | Sentiment Analysis Task Dataset | hub.dataset.GLUE("SST-2") |
| GLUE-MNLI | Text Reasoning Task Dataset | hub.dataset.GLUE("MNLI_m") |
| GLUE-QQP | Sentence pairs Classification Task Dataset | hub.dataset.GLUE("QQP") |
| GLUE-QNLI | Problem Inference Task Dataset | hub.dataset.GLUE("QNLI") |
| GLUE-STS-B | Regression Task Dataset | hub.dataset.GLUE("STS-B") |
| GLUE-MRPC | Sentence pairs Classification Task Dataset | hub.dataset.GLUE("MRPC") |
| GLUE-RTE | Text implication task dataset | hub.dataset.GLUE("RTE") |
| XNLI | Cross-language natural language inference dataset | hub.dataset.XNLI(language=zh) |
| ChineseGLUE-TNEWS | Today's Headlines Chinese News (Short text) Classified Dataset | hub.dataset.TNews() |
| ChineseGLUE-INEWS | Internet Sentiment Analysis Task Dataset | hub.dataset.INews() |
| DRCD | Delta Reading Comprehension Dataset, a General Domain Traditional Chinese Character Machine Reading Comprehension Dataset | hub.dataset.DRCD() |
| CMRC2018 | Span Extraction Dataset for Chinese Machine Reading Comprehension | hub.dataset.CMRC2018() |
| ChinesGLUE-BQ | Intelligent Customer Service Chinese Question Matching Dataset | hub.dataset.BQ() |
| ChineseGLUE-IFLYTEK | Chinese long-text classification dataset with over 17,000 text markups on app | hub.dataset.IFLYTEK() |
| ChineseGLUE-THUCNEWS | Chinese long text classification dataset, with more than 40,000 Chinese news long text markup data, with 14 categories in total. | hub.dataset.THUCNEWS() |
| DogCatDataset | Dataset provided by Kaggle for image binning. | hub.dataset.DogCatDataset() |
| Food101 | Food photo dataset provided by Kaggle, containing 101 categories | hub.dataset.Food101() |
| Indoor67 | A dataset of 67 indoor scenes released by MIT with the goal of identifying the scene category of an indoor image. | hub.dataset.Indoor67() |
| Flowers | Flower dataset. There are 5 types of datasets, including "roses", "tulips", "daisy", "sunflowers", and "dandelion". | hub.dataset.Flowers() |
| StanfordDogs | Dataset released by Stanford University, containing 120 species of dogs for image classification. | hub.dataset.StanfordDogs() |
##### **Load custom datasets**
* Load the text class custom dataset. The user only needs to inherit the base class BaseNLPDatast and modify the dataset address and category. Refer to the following codes.
```python
from paddlehub.dataset.base_nlp_dataset import BaseNLPDataset
# Define Class
class DemoDataset(BaseNLPDataset):
def __init__(self):
# path of dateset
self.dataset_dir = "path/to/dataset"
super(DemoDataset, self).__init__(
base_path=self.dataset_dir,
train_file="train.tsv", # training dataset
dev_file="dev.tsv", # dev dataset
test_file="test.tsv", # test dataset
predict_file="predict.tsv",
train_file_with_header=True,
dev_file_with_header=True,
test_file_with_header=True,
predict_file_with_header=True,
# Dataset Label
label_list=["0", "1"])
# Create Dataset Object
dataset = DemoDataset()
```
Then you can get the custom dataset by DemoDataset(). With the data preprocessor and pre-training models such as ERNIE, you can complete the text class task.
* Load the image class custom dataset. The user only needs to inherit the base class BaseCVDatast and modify the dataset storage address. Refer to the following codes.
```python
from paddlehub.dataset.base_cv_dataset import BaseCVDataset
class DemoDataset(BaseCVDataset):
def __init__(self):
# Path of dataset
self.dataset_dir = "/test/data"
super(DemoDataset, self).__init__(
base_path=self.dataset_dir,
train_list_file="train_list.txt",
validate_list_file="validate_list.txt",
test_list_file="test_list.txt",
predict_file="predict_list.txt",
label_list_file="label_list.txt",
)
dataset = DemoDataset()
```
Then you can get the custom dataset by DemoDataset(). With the data preprocessor and pre-training models, you can complete the vision class transfer learning task.
#### **4\. Configure the Data Preprocessor.**
Read data of NLP or CV dataset by using PaddleHub's Data Preprocessor API.
```python
reader = hub.reader.ClassifyReader(
dataset=dataset,
vocab_path=module.get_vocab_path(),
max_seq_len=128,
sp_model_path=module.get_spm_path(),
word_dict_path=module.get_word_dict_path())
```
For different task types, users can select different Readers.
| Data Readers | Description | Task Type | API Examples |
| :------------------------: | :----------------------------------------------------------: | :-------: | :----------------------------------------------------------- |
| ClassifyReader | It is the data preprocessor suitable for the Transformer pre-training model (ERNIE/BERT). | NLP | reader = hub.reader.ClassifyReader( dataset=dataset, vocab_path=module.get_vocab_path(), max_seq_len=128, sp_model_path=module.get_spm_path(), word_dict_path=module.get_word_dict_path()) |
| LACClassifyReader | A data preprocessor using the LAC module as a word cutter, suitable for tasks such as Senta and ELMo that require granularity-based word segmentation. | NLP | reader = hub.reader.LACClassifyReader( dataset=dataset, vocab_path=module.get_vocab_path()) |
| SequenceLabelReader | Sequence annotation preprocessor for the Transformer class model (ERNIE/BERT). | NLP | reader = hub.reader.SequenceLabelReader( dataset=dataset, vocab_path=module.get_vocab_path(), max_seq_len=128, sp_model_path=module.get_spm_path(), word_dict_path=module.get_word_dict_path()) |
| MultiLabelClassifyReader | Multi-label classification preprocessor for the Transformer class model (ERNIE/BERT). | NLP | reader = hub.reader.MultiLabelClassifyReader( dataset=dataset, vocab_path=module.get_vocab_path(), max_seq_len=128) |
| ReadingComprehensionReader | Reading comprehension task preprocessor for the Transformer class model (ERNIE/BERT). | NLP | reader = hub.reader.ReadingComprehensionReader( dataset=dataset, vocab_path=module.get_vocab_path(), max_seq_length=384) |
| RegressionReader | A data preprocessor suitable for regression tasks. | NLP | reader = hub.reader.RegressionReader( dataset=dataset, vocab_path=module.get_vocab_path(), max_seq_len=args.max_seq_len) |
| ImageClassificationReader | Preprocessor suitable for image classification data. Modify the size of the input image. Perform standardization, image broadening, and other operations. | CV | reader = hub.reader.ImageClassificationReader( image_width=module.get_expected_image_width(), image_height=module.get_expected_image_height(), images_mean=module.get_pretrained_images_mean(), images_std=module.get_pretrained_images_std(), dataset=dataset) |
#### **5\. Choose an Optimization Strategy.**
In PaddleHub, the Strategy class encapsulates a set of fine-tuning strategies for transfer learning. Strategies include what learning rate variation strategy for pre-training parameters, what type of optimizer, what type of regularization, and so on. In the text classification task, we use AdamWeightDecayStrategy optimization strategy. Refer to the following code.
```python
strategy = hub.AdamWeightDecayStrategy(
learning_rate=5e-5, # Fine-tune max learning rate
weight_decay=0.01, # defaule 0.01
warmup_proportion=0.1, #warmup_proportion>0, for example: 0.1
lr_scheduler="linear_decay",
)
```
PaddleHub also provides a variety of APIs for various optimization strategies, in addition to AdamWeightDecayStrategy.
| Optimization Strategies | Description | API Examples |
| :---------------------: | :----------------------------------------------------------- | :----------------------------------------------------------- |
| DefaultFinetuneStrategy | The default optimization strategy. The corresponding parameters are as follows: <br/>\* learning\_rate: The global learning rate. The default value is 1e-4. <br/>\* optimizer\_name: Optimizer\_name: The default is adam. <br/>\* regularization\_coeff: The regularization λ parameter. The default value is 1e-3. <br/> This optimization strategy is recommended for the image classification task. | strategy = hub.DefaultFinetuneStrategy( learning_rate=1e-4, optimizer_name="adam", regularization_coeff=1e-3) |
| AdamWeightDecayStrategy | A learning rate decay strategy based on Adam optimizer. The corresponding parameters are as follows: <br/>\* learning\_rate: Global learning rate. The default value is 1e-4. <br/>\* lr\_scheduler: Learning rate scheduling method. The default value is "linear\_decay". <br/>\* warmup\_proportion: Warmup proportion. <br/>\* weight\_decay: Learning rate decay rate. <br/>\* optimizer\_name: Optimizer name. The default value is adam.<br/> This optimization strategy is recommended for text classification, reading comprehension, and other tasks. | strategy = hub.AdamWeightDecayStrategy( learning_rate=1e-4, lr_scheduler="linear_decay", warmup_proportion=0.0, weight_decay=0.01, optimizer_name="adam") |
| L2SPFinetuneStrategy | Finetune strategy using the L2SP regular as the penalty factor. The corresponding parameters are as follows: <br/>\* learning\_rate: The global learning rate. The default value is 1e-4. <br/>\* optimizer\_name: Optimizer\_name: The default is adam. <br/>\* regularization\_coeff: The regularization λ parameter. The default value is 1e-3. | strategy = hub.L2SPFinetuneStrategy( learning_rate=1e-4, optimizer_name="adam", regularization_coeff=1e-3) |
| ULMFiTStrategy | The strategy implements the three strategies proposed in the ULMFiT paper: <br/> \* Slanted triangular learning rates: a strategy of learning to rise and then fall. <br/>\* Discriminative fine-tuning: a strategy of decreasing learning rates layer by layer. It can slow down the underlying update rate. <br/>\* Gradual unfreezing: a layer-by-layer unfreezing strategy. It gives a priority to updating the upper layer and then slowly unfreezes the lower layer, to participate in the update. <br/> The corresponding parameters are as follows: <br/> \* learning\_rate: The global learning rate. The default value is 1e-4. <br/> \* optimizer\_name: Optimizer\_name: The default value is adam. <br/> \* cut\_fraction: Sets the ratio of number of steps of learning rate increase of “Slanted triangular learning rates” to the total number of training steps. The default value is 0.1. If it is set to 0, “Slanted triangular learning rates” is not used. <br/> \* ratio: Sets the ratio of the decrease minimum learning rate to the increase maximum learning rate of “Slanted triangular learning rates”. The default value is 32, indicating that the minimum learning rate is 1/32 of the maximum learning rate. <br/> \* dis\_blocks: Sets the number of blocks in the Discriminative fine-tuning. The default value is 3. If it is set to 0, the Discriminative fine-tuning is not used. <br/> \* factor: Sets the decay rate of the Discriminative fine-tuning. The default value is 2.6, indicating that the learning rate of the next layer is 1/2.6 of the upper layer. <br/> \* frz\_blocks: Sets the number of blocks in the gradual unfreezing. The concept of block is the same as that in "dis\_blocks". | strategy = hub.ULMFiTStrategy( learning_rate=1e-4, optimizer_name="adam", cut_fraction=0.1, ratio=32, dis_blocks=3, factor=2.6, frz_blocks=3) |
#### **6\. Set the RunConfig.**
In PaddleHub, users can use RunConfig in the Finetune API to configure the parameters when a Task is in Fine-tune, including Epochs, batch size, and whether to use GPU for training. The example codes are as follows:
```python
config = hub.RunConfig(use_cuda=True, num_epoch=3, batch_size=32, strategy=strategy)
```
#### **7\. Assemble the Training Task.**
Once we have a suitable pre-training model and load the dataset to be transferred, we start to assemble a Task. In PaddleHub, the Task represents a fine-tuned task. The Task contains the information related to the execution of the task, such as Program, Reader, and RunConfig. The task description corresponding to the [TextClassifierTask](https://github.com/PaddlePaddle/PaddleHub/blob/release/v1.6/docs/reference/task/text_classify_task.md) can be found here. The specific implementation solution is as follows:
1. Get the context of the module (PaddleHub's pre-training model), including the input and output variables, and Paddle Program (executable model format).
2. Find the feature\_map of the extraction layer from the output variables of the pre-training model, and access a full connection layer behind the feature\_map. In the following codes, it is specified by the pooled\_output parameter of the hub.TextClassifierTask.
3. The input layer of the network remains unchanged, still starting from the input layer, as specified in the following code via the feed\_list variable of the hub.TextClassifierTask. The hub.TextClassifierTask specifies our requirements for intercepting the model network with these two parameters. According to this configuration, the network we intercept is from the input layer to the last layer of feature extraction "pooled\_output", indicating that we will use the intercepted Network for transfer learning training.
```python
inputs, outputs, program = module.context(trainable=True, max_seq_len=128)
pooled_output = outputs["pooled_output"]
feed_list = [
inputs["input_ids"].name,
inputs["position_ids"].name,
inputs["segment_ids"].name,
inputs["input_mask"].name,
]
cls_task = hub.TextClassifierTask(
data_reader=reader,
feature=pooled_output,
feed_list=feed_list,
num_classes=dataset.num_labels,
metrics_choices = ["acc"],
config=config)
```
PaddleHub comes with pre-built Tasks for common tasks. Each task has a specific application scenario and corresponding metrics to meet the different user requirements.
| Task Type | Description | Task Type |
| :----------------------: | :----------------------------------------------------------: | :-----------------------------: |
| ImageClassifierTask | The Task is added with one or more full connection layers to create a classification task for Fine-tune based on the input features. The metric is the accuracy, and the loss function is the cross-entropy Loss. | Image classification task |
| TextClassifierTask | The Task is added with a Dropout layer and one or more full connection layers to create a text classification task for fine-tune based on the input features. The metric is accuracy, and the loss function is cross-entropy Loss. | Text classification task |
| SequenceLabelTask | The Task is added with one full connection layer or a full connection layer and CRF layer to create a sequence label task for Fine-tune based on the input features. The metric is F1, and the loss function is the cross-entropy Loss. | Sequence Annotation Task |
| MultiLabelClassifierTask | The Task is added with one or more full connection layers to create a multi-label classification task for fine-tune based on the input features. The metric is average AUC of multiple labels, and the loss function is the average cross-entropy of multiple labels. | Multi-label classification task |
| RegressionTask | The Task is added with a Dropout layer and one or more full connection layers to create a text regression task for fine-tune based on the input features. The metric is accuracy, and the loss function is the mean variance loss function. | Text Regression Task |
| ReadingComprehensionTask | The Task is added with one full connection layer to create a reading comprehension task for fine-tune based on the input features. The loss function is the cross-entropy Loss. | Reading Comprehension Task |
Before setting each Task, the user needs to know the input and output of the pre-training model of the transfer learning, that is, "feed\_list" and "pooled\_output" in the above codes. The specific input and output codes can be divided into the following:
* Image classification model
```
input_dict, output_dict, program = module.context(trainable=True)
feature_map = output_dict["feature_map"]
feed_list = [input_dict["image"].name]
```
* Natural language processing model (excluding word2vec\_skipgram, simnet\_bow, text matching, and text generation models).
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=128)
pooled_output = outputs["pooled_output"]
feed_list = [
inputs["input_ids"].name,
inputs["position_ids"].name,
inputs["segment_ids"].name,
inputs["input_mask"].name,
]
```
* word2vec\_skipgram model
```
inputs, outputs, program = module.context(trainable=True)
word_ids = inputs["word_ids"]
embedding = outputs["word_embs"]
```
* simnet\_bow model
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=args.max_seq_len, num_slots=2)
query = outputs["emb"]
title = outputs['emb_2']
```
* Pairwise Text Matching Model
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=args.max_seq_len, num_slots=3)
query = outputs["emb"]
left = outputs['emb_2']
right = outputs['emb_3']
```
* Pointwise Text Matching
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=args.max_seq_len, num_slots=2)
query = outputs["emb"]
title = outputs['emb_2']
```
* Text Generation Model
```
inputs, outputs, program = module.context(trainable=True, max_seq_len=128)
pooled_output = outputs["pooled_output"]
sequence_output = outputs["sequence_output"]
```
#### **8\. Start the Fine-tune. Use the Finetune\_and\_eval Function to Complete the Training and Evaluation.**
```python
cls_task.finetune_and_eval()
```
The information is displayed as follows: You can see the evaluation results of the training, Loss value, accuracy, and so on.
```
[2020-07-28 21:28:21,658] [ TRAIN] - step 810 / 900: loss=0.05022 acc=0.97813 [step/sec: 4.07]
[2020-07-28 21:28:24,115] [ TRAIN] - step 820 / 900: loss=0.04719 acc=0.98125 [step/sec: 4.07]
[2020-07-28 21:28:26,574] [ TRAIN] - step 830 / 900: loss=0.06895 acc=0.98125 [step/sec: 4.07]
[2020-07-28 21:28:29,035] [ TRAIN] - step 840 / 900: loss=0.07830 acc=0.97813 [step/sec: 4.07]
[2020-07-28 21:28:31,490] [ TRAIN] - step 850 / 900: loss=0.07279 acc=0.97500 [step/sec: 4.08]
[2020-07-28 21:28:33,939] [ TRAIN] - step 860 / 900: loss=0.03220 acc=0.99375 [step/sec: 4.09]
[2020-07-28 21:28:36,388] [ TRAIN] - step 870 / 900: loss=0.05016 acc=0.98750 [step/sec: 4.09]
[2020-07-28 21:28:38,840] [ TRAIN] - step 880 / 900: loss=0.05604 acc=0.98750 [step/sec: 4.08]
[2020-07-28 21:28:41,293] [ TRAIN] - step 890 / 900: loss=0.05622 acc=0.98125 [step/sec: 4.08]
[2020-07-28 21:28:43,748] [ TRAIN] - step 900 / 900: loss=0.06642 acc=0.97813 [step/sec: 4.08]
[2020-07-28 21:28:43,750] [ INFO] - Evaluation on dev dataset start
[2020-07-28 21:28:46,654] [ EVAL] - [dev dataset evaluation result] loss=0.17890 acc=0.94079 [step/sec: 13.23]
[2020-07-28 21:28:46,657] [ INFO] - Evaluation on dev dataset start
[2020-07-28 21:28:49,527] [ EVAL] - [dev dataset evaluation result] loss=0.17890 acc=0.94079 [step/sec: 13.39]
[2020-07-28 21:28:49,529] [ INFO] - Load the best model from ckpt_20200728212416/best_model
[2020-07-28 21:28:50,112] [ INFO] - Evaluation on test dataset start
[2020-07-28 21:28:52,987] [ EVAL] - [test dataset evaluation result] loss=0.14264 acc=0.94819 [step/sec: 13.36]
[2020-07-28 21:28:52,988] [ INFO] - Saving model checkpoint to ckpt_20200728212416/step_900
[2020-07-28 21:28:55,789] [ INFO] - PaddleHub finetune finished.
```
After completing the model training with Fine-tune, PaddleHub will automatically save the best model in the validation set in the corresponding ckpt directory (CKPT\_DIR). The user can make predictions with reference to the following codes, where the inferred label value 0 indicates a negative evaluation, and 1 indicates a positive evaluation.
```python
import numpy as np
# 待预测数据
data = [
["这个宾馆比较陈旧了，特价的房间也很一般。总体来说一般"],
["交通方便；环境很好；服务态度很好 房间较小"],
["19天硬盘就罢工了，算上运来的一周都没用上15天，可就是不能换了。唉，你说这算什么事呀！"]
]
index = 0
run_states = cls_task.predict(data=data)
results = [run_state.run_results for run_state in run_states]
for batch_result in results:
# 预测类别取最大分类概率值
batch_result = np.argmax(batch_result[0], axis=1)
for result in batch_result:
print("%s\tpredict=%s" % (data[index][0], result))
index += 1
```
The prediction results are as follows:
```
[2020-07-28 18:06:45,441] [ INFO] - PaddleHub predict start
[2020-07-28 18:06:45,442] [ INFO] - The best model has been loaded
[2020-07-28 18:06:48,406] [ INFO] - PaddleHub predict finished.
这个宾馆比较陈旧了，特价的房间也很一般。总体来说一般 predict=0
交通方便；环境很好；服务态度很好 房间较小 predict=1
19天硬盘就罢工了，算上运来的一周都没用上15天，可就是不能换了。唉，你说这算什么事呀！ predict=0
```
docs/docs_en/tutorial/how_to_load_data_en.md 0 → 100644
浏览文件 @ 2b17b1f8
# Customized Data
In the training of a new task, it is a time-consuming process starting from zero, without producing the desired results probably. You can use the pre-training model provided by PaddleHub for fine-tune of a specific task. You just need to perform pre-processing of the customized data accordingly, and then input the pre-training model to get the corresponding results. Refer to the following for setting up the structure of the dataset.
## I. Image Classification Dataset
When migrating a classification task using custom data with PaddleHub, you need to slice the dataset into training set, validation set, and test set.
### Data Preparation
Three text files are needed to record the corresponding image paths and labels, plus a label file to record the name of the label.
```
├─data:
  ├─train_list.txt：
  ├─test_list.txt：
  ├─validate_list.txt：
├─label_list.txt：
  └─...
```
The format of the data list file for the training/validation/test set is as follows:
```
Path-1 label-1
Path-2 label-2
...
```
The format of label\_list.txt is:
```
Classification 1
Classification 2
...
```
Example: Take [Flower Dataset](../reference/dataset.md) as an example, train\_list.txt/test\_list.txt/validate\_list.txt:
```
roses/8050213579_48e1e7109f.jpg 0
sunflowers/45045003_30bbd0a142_m.jpg 3
daisy/3415180846_d7b5cced14_m.jpg 2
```
label\_list.txt reads as follows.
```
roses
tulips
daisy
sunflowers
dandelion
```
### Dataset Loading
For the preparation code of dataset, see [flowers.py](../../paddlehub/datasets/flowers.py). `hub.datasets.Flowers()` It automatically downloads the dataset from the network and unzip it into the `$HOME/.paddlehub/dataset` directory. Specific usage:
```python
from paddlehub.datasets import Flowers
flowers = Flowers(transforms)
flowers_validate = Flowers(transforms, mode='val')
```
* `transforms`: Data pre-processing mode.
* `mode`: Select data mode. Options are `train`, `test`, `val`. Default is `train`.
## II. Image Coloring Dataset
When migrating a coloring task using custom data with PaddleHub, you need to slice the dataset into training set and test set.
### Data Preparation
You need to divide the color images for coloring training and testing into training set data and test set data.
```
├─data:
  ├─train：
|-folder1
|-folder2
|-……
|-pic1
|-pic2
|-……
  ├─test：
|-folder1
|-folder2
|-……
|-pic1
|-pic2
|-……
  └─……
```
Example: PaddleHub provides users with a dataset for coloring `Canvas dataset. It consists of 1193 images in Monet style and 400 images in Van Gogh style. Take [Canvas Dataset](../reference/datasets.md) as an example, the contents of the train folder are as follows:
```
├─train：
|-monet
|-pic1
|-pic2
|-……
|-vango
|-pic1
|-pic2
|-……
```
### Dataset Loading
For dataset preparation codes, refer to [canvas.py](../../paddlehub/datasets/canvas.py). `hub.datasets.Canvas()` It automatically downloads the dataset from the network and unzip it into the `$HOME/.paddlehub/dataset` directory. Specific usage:
```python
from paddlehub.datasets import Canvas
color_set = Canvas(transforms, mode='train')
```
* `transforms`: Data pre-processing mode.
* `mode`: Select data mode. Options are `train`, `test`. The default is `train`.
## III. Style Migration Dataset
When using custom data for style migration tasks with PaddleHub, you need to slice the dataset into training set and test set.
### Data Preparation
You need to split the color images for style migration into training set and test set data.
```
├─data:
  ├─train：
|-folder1
|-folder2
|-...
|-pic1
|-pic2
|-...
  ├─test：
|-folder1
|-folder2
|-...
|-pic1
|-pic1
|-...
|- 21styles
｜-pic1
｜-pic1
  └─...
```
Example: PaddleHub provides users with data sets for style migration `MiniCOCO dataset`. Training set data and test set data is from COCO2014, in which there are 2001 images in the training set and 200 images in the test set. There are 21 images with different styles in `21styles` folder. Users can change the images of different styles as needed. Take [MiniCOCO Dataset](../reference/datasets.md) as an example. The content of train folder is as follows:
```
├─train：
|-train
|-pic1
|-pic2
|-……
|-test
|-pic1
|-pic2
|-……
|-21styles
|-pic1
|-pic2
|-……
```
### Dataset Loading
For the preparation codes of the dataset, refer to [minicoco.py](../../paddlehub/datasets/minicoco.py). `hub.datasets.MiniCOCO()` It automatically downloads the dataset from the network and unzip it into the `$HOME/.paddlehub/dataset` directory. Specific usage:
```python
from paddlehub.datasets import MiniCOCO
ccolor_set = MiniCOCO(transforms, mode='train')
```
* `transforms`: Data pre-processing mode.
* `mode`: Select data mode. Options are `train`, `test`. The default is `train`.
docs/docs_en/tutorial/serving_en.md 0 → 100644
浏览文件 @ 2b17b1f8
# PaddleHub Serving ： One-Key Deploy Models as Services
## Introduction
### Background
PaddleHub enables the rapid model prediction. Developers are often faced with the need to migrate the local prediction process to go online. PaddleHub's ability to rapidly deploy model prediction services is required, whether to open public service ports or to build the prediction services on a LAN. In this background, the PaddleHub Serving, the model one-key service deployment tool, emerges. Developers can quickly launch a model prediction online service with using a single line of command, without having to concern about the network framework selection and implementation.
### One-Key Service Deployment
PaddleHub Serving is a one-key model service deployment tool based on PaddleHub. It can easily start a model prediction online service through a simple Hub command line tool. The front-end completes the processing of network requests through Flask and Gunicorn. The background directly invokes the PaddleHub prediction interface and supports multi-process to improve the concurrency by using multi-core to ensure the performance of the prediction service.
### Supported Models
Currently, PaddleHub Serving supports the service deployment for all direct prediction PaddleHub models, including `lac`, `senta_bilstm` and other NLP models, as well as `yolov3_darknet53_coco2017`, `vgg16_imagenet` and other CV models. For more models, refer to [PaddleHub Supported Models List](https://paddlepaddle.org.cn/hublist). In the future, developers will also be able to use the models from PaddleHub Fine-tune API for rapid service deployment.
## Usage
### Step 1: Start the server deployment.
PaddleHub Serving starts in two ways: Starting in the command line, and Starting by using the configuration file.
#### Start in command line
Command
```shell
$ hub serving start --modules [Module1==Version1, Module2==Version2, ...] \
--port XXXX \
--use_gpu \
--use_multiprocess \
--workers \
--gpu \
```
Parameter:
| Parameter | Purpose |
| ------------------ | ------------------------------------------------------------ |
| --modules/-m | PaddleHub Serving pre-installed model. It is listed as multiple Module==Version key value pairs <br/> |
| --port/-p | Service port. By default, it is 8866. |
| --use_gpu | paddlepaddle-gpu must be installed when the GPU is used for prediction. |
| --use_multiprocess | Whether to enable concurrent mode. By default, it is single process. This is recommended for multi-core CPUs <br/>*`Only single process is supported in Windows system`* |
| --workers | The number of concurrent tasks specified in the concurrent mode. By default, it is `2*cpu_count-1`, where `cpu_count` is the number of CPU cores. |
| --gpu | Specify the card number of the GPU to be used. For example, `1,2` means using VGA card 1 and VGA card 2. By default, only VGA card 0 is used. |
**NOTE:** --use\_gpu should not be used together with --use\_multiprocess.
#### Start by using the configuration file
Command
```shell
$ hub serving start --config config.json
```
`config.json`The format is as follows:
```json
{
"modules_info": {
"yolov3_darknet53_coco2017": {
"init_args": {
"version": "1.0.0"
},
"predict_args": {
"batch_size": 1,
"use_gpu": false
}
},
"lac": {
"init_args": {
"version": "1.1.0"
},
"predict_args": {
"batch_size": 1,
"use_gpu": false
}
}
},
"port": 8866,
"use_multiprocess": false,
"workers": 2,
"gpu": "0,1,2"
}
```
Parameter:
### Step 2: Access the server
After deploying the server model prediction service using PaddleHub Serving, you can access the prediction interface on the client to get the results. The interface url format is:
`http://127.0.0.1:8866/predict/<MODULE>`
where `<MODULE>` is the model name.
The prediction results can be obtained by sending a POST request. Below we will show a concrete demo to illustrate the deployment and usage process using the PaddleHub Serving.
### Step 3: Use the PaddleHub Serving to perform the personalization development.
After deploying the model service using the PaddleHub Serving, you can use the obtained interface for development, such as, providing external web services, or accessing the application to reduce the prediction pressure on clients and improve performance. A web page demo is displayed as follows:
![](../../imgs/web_demo.png)
### Step 4: Shut down the serving.
Run the shutdown command to shut down the serving.
```shell
$ hub serving stop --port XXXX
```
Parameter:
| Parameter | Purpose |
| --------- | ------------------------------------------------------------ |
| --port/-p | Specify the service port to be closed. By default, it is 8866. |
## Demo – Deploy an Online Lac Word Segmentation Service
### Step 1: Deploy the lac online service.
Now, we want to deploy a lac online service to get the word segmentation results of texts through an interface.
First, choose either start-up methods:
```shell
$ hub serving start -m lac
```
OR
```shell
$ hub serving start -c serving_config.json
```
where `serving_config.json` is as follows.
```json
{
"modules_info": {
"lac": {
"init_args": {
"version": "1.1.0"
},
"predict_args": {
"batch_size": 1,
"use_gpu": false
}
}
},
"port": 8866,
"use_multiprocess": false,
"workers": 2
}
```
The successful startup interface is displayed in the following figure:
![](../../imgs/start_serving_lac.png)
We have successfully deployed lac's online word segmentation service on port 8866. *The warning here is a Flask prompt, without affecting the usage.*
### Step 2: Access the lac prediction interface.
After the service is deployed, we can test it with the texts `今天是个好日子` and `天气预报说今天要下雨`.
The client codes are as follows:
```python
# coding: utf8
import requests
import json
if __name__ == "__main__":
text = ["今天是个好日子", "天气预报说今天要下雨"]
data = {"texts": text, "batch_size": 1}
url = "http://127.0.0.1:8866/predict/lac"
headers = {"Content-Type": "application/json"}
r = requests.post(url=url, headers=headers, data=json.dumps(data))
# Print results
print(json.dumps(r.json(), indent=4, ensure_ascii=False))
```
Run and get results.
```python
{
"msg": "",
"results": [
{
"tag": [
"TIME", "v", "q", "n"
],
"word": [
"今天", "是", "个", "好日子"
]
},
{
"tag": [
"n", "v", "TIME", "v", "v"
],
"word": [
"天气预报", "说", "今天", "要", "下雨"
]
}
],
"status": "0"
}
```
### Step 3: Stop the serving the service
Since we are using the default service port 8866 at startup, the corresponding shutdown command is:
```shell
$ hub serving stop --port 8866
```
Or if you do not specify a shutdown port, the default port is 8866.
```shell
$ hub serving stop
```
Wait for serving to clear the service. The system prompts:
```shell
$ PaddleHub Serving will stop.
```
The serving service has been stopped.
For specific information and codes of this demo, see [LAC Serving](../../demo/serving/module_serving/lexical_analysis_lac). In addition, some other One-key Service Deployment demos are shown below.
## More Demos
* [Chinese lexical analysis - based on lac.](../../demo/serving/module_serving/lexical_analysis_lac)
  This example demonstrates the deployment and online prediction of Chinese text segmentation services using lac, to get the text word-segmentation results, and intervene in the word segmentation results with a user-defined dictionary.
* [Face Detection - based on pyramidbox\_lite\_server\_mask](../../demo/serving/module_serving/object_detection_pyramidbox_lite_server_mask)
  This example shows the face mask detection by using pyramidbox\_lite\_server\_mask, to detect the location of the face and the confidence level of the mask.
modules/audio/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the official documentation on WEB -> [Text-to-speech](https://www.paddlepaddle.org.cn/hublist)**
### OCR
Text-to-speech (TTS) task can realize the conversion of text into speech, which has been widely used in a variety of speech interactive devices.
- Recommended Models
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Text-to-speech transformer\_tts\_ljspeech](https://www.paddlepaddle.org.cn/hubdetail?name=transformer_tts_ljspeech&en_category=TextToSpeech) | TansformerTTS is a fusion of Transformer and Tacotron2 with satisfactory results. It is an English TTS model and supports the prediction only. |
| [Text-to-speech fastspeech\_ljspeech](https://www.paddlepaddle.org.cn/hubdetail?name=fastspeech_ljspeech&en_category=TextToSpeech) | FastSpeech is based on the attention diagonal line extracted from the teacher model in the encoder-decoder structure to make pronunciation duration prediction. It is an English TTS model and supports the prediction only. |
| [Text-to-speech deepvoice3\_ljspeech](https://www.paddlepaddle.org.cn/hubdetail?name=deepvoice3_ljspeech&en_category=TextToSpeech) | Deep Voice 3 is an end-to-end TTS model released by Baidu Research Institute in 2017 (paper accepted in ICLR 2018). It is a seq2seq model based on the convolutional neural network and attention mechanism. It is an English TTS model and supports the prediction only. |
modules/image/Image_editing/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For a better user experience, refer to the WEB official document -> [Image Editing](https://www.paddlepaddle.org.cn/hublist)**
### Image Editing
Image editing refers to the further editing and adjustment of image pixels on the basis of the input images, to output a new target image. Specific application scenarios include super-resolution, black and white picture coloring, old photo restoration, and so on.
- Recommended Models
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Super-resolution](https://www.paddlepaddle.org.cn/hubdetail?name=realsr&en_category=ImageEditing) | It can be used for the super-resolution models of images and videos, with four times of resolution of input images and videos. |
| [B\&W Image Coloring](https://www.paddlepaddle.org.cn/hubdetail?name=deoldify&en_category=ImageEditing) | deoldify is a color rendering model for images and videos that allows you to restore original color to black and white photos and videos. |
| [Old Photo Restoration](https://www.paddlepaddle.org.cn/hubdetail?name=photo_restoration&en_category=ImageEditing) | It is a model for the restoration of old photos. It consists of two parts: coloring and superdivision. |
modules/image/Image_gan/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the Web official document -> [Image Generation](https://www.paddlepaddle.org.cn/hublist)**
### Image Generation
Image generation is the generation of a target image based on an input vector. The input vector here can be a random noise or a user-specified conditional vector. Specific application scenarios include style migration, image cartoons, and so on.
- Recommended Models
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Art Style Migration](https://www.paddlepaddle.org.cn/hubdetail?name=stylepro_artistic&en_category=GANs) | Convert a given image into an arbitrary artistic style. Ensure that high fidelity model reproduction of semantic detail information of content images is consistent with the style information of style images. |
| [Image Animation - Makoto Shinkai](https://www.paddlepaddle.org.cn/hubdetail?name=animegan_v2_shinkai_53&en_category=GANs) | AnimeGAN V2 image style conversion model. It is a model that converts the input image into a Makoto Shinkai style. |
| [Image Animation - Hayao Miyazaki](https://www.paddlepaddle.org.cn/hubdetail?name=animegan_v2_hayao_64&en_category=GANs) | AnimeGAN V2 image style conversion model. It is a model that converts the input image into a Miyazaki anime style. |
| [Image Animation - Satoshi Kon Paprika](https://www.paddlepaddle.org.cn/hubdetail?name=animegan_v2_paprika_97&en_category=GANs) | AnimeGAN V2 image style conversion model. It is a model that converts the input image into an Satoshi Kon Paprika style anime style. |
modules/image/classification/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the official web documentation -> [Image Classification](https://www.paddlepaddle.org.cn/hublist)**
### Image Classification
Image classification distinguishes between different categories of images based on the semantic information. This is an important basic issue in computer vision and is the basis for object detection, image segmentation, object tracking, behavior analysis, face recognition and other high-level vision tasks. It is widely applied in many fields. For example, face recognition and intelligent video analysis in the security field, traffic scene recognition in the transportation field, content-based image retrieval and automatic classification of photo albums in the Internet field, and image recognition in the medicine field.
**Note:** If you are an experienced developer, feel free to use as required. If you are a newcomer, choose Resnet50 first at the server side and MobileNetV3 for mobile.
- Recommended Models
| | **Model Name** | **Model Features** |
| -------------------- | :----------------------------------------------------------- | ------------------------------------------------------------ |
| Image Classification | [Dish Identification](https://www.paddlepaddle.org.cn/hubdetail?name=resnet50_vd_dishes&en_category=ImageClassification) | Private dataset training. It supports the category identification of 8416 dishes. It is suitable for further fine-tuning in the dish orientation. |
| Image Classification | [Animal Identification](https://www.paddlepaddle.org.cn/hubdetail?name=resnet50_vd_animals&en_category=ImageClassification) | Private dataset training. It supports the category identification of 7978 kinds of animals. It is suitable for further fine-tuning in the animal orientation. |
| Image Classification | [Wildlife product identification](https://www.paddlepaddle.org.cn/hubdetail?name=resnet50_vd_wildanimals&en_category=ImageClassification) | Supports identification of the ten tags 'Ivory product', 'Ivory', 'Elephant', 'Tiger Skin', 'Tiger', 'Tiger Tusk/Claw/Bone', 'Pangolin squama', 'Pangolin', 'Pangolin Paw', and 'Other'. |
- More Models
| **Model Name** | **Model Introduction** |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [AlexNet](https://www.paddlepaddle.org.cn/hubdetail?name=alexnet_imagenet&en_category=ImageClassification) | It is the first successful application of ReLU, Dropout and LRN in CNN, with using GPU for acceleration. |
| [VGG19](https://www.paddlepaddle.org.cn/hubdetail?name=vgg19_imagenet&en_category=ImageClassification) | Use 3x3 small convolutional cores based on AlexNet to increase network depth with good generalization capabilities |
| [GoogLeNet](https://github.com/PaddlePaddle/models/tree/release/1.7/PaddleCV/image_classification) | Increased network depth and width without increasing computational load for superior performance |
| [ResNet50](https://www.paddlepaddle.org.cn/hubdetail?name=resnet_v2_50_imagenet&en_category=ImageClassification) | Residual Network. Introduce a new residual structure that solves the problem of decreasing accuracy as the network deepens. |
| [Inceptionv4](https://www.paddlepaddle.org.cn/hubdetail?name=inception_v4_imagenet&en_category=ImageClassification) | Combine the Inception module with the Residual Connection greatly. This accelerates training and performance increases through ResNet's architecture. |
| [MobileNetV2](https://www.paddlepaddle.org.cn/hubdetail?name=mobilenet_v2_imagenet&en_category=ImageClassification) | Fine-tuning of the MobileNet structure. Perform the skip learning connections directly on the bottleneck layer of thinner. There is no ReLu nonlinear processing of the bottleneck layer, with a good results. |
| [se_resnext50](https://www.paddlepaddle.org.cn/hubdetail?name=se_resnext50_32x4d_imagenet&en_category=ImageClassification) | Add the SE (Sequeeze-and-Excitation) module to ResNeXt to improve recognition accuracy. Achieve the first place in the ILSVRC 2017 classification program. |
| [ShuffleNetV2](https://www.paddlepaddle.org.cn/hubdetail?name=shufflenet_v2_imagenet&en_category=ImageClassification) | ECCV2018, lightweight CNN network. It offers a good balance between speed and accuracy. It is more accurate than ShuffleNet and MobileNetv2 at the same level of complexity. It is more suitable for mobile and unmanned vehicle fields. |
| [efficientNetb7](https://www.paddlepaddle.org.cn/hubdetail?name=efficientnetb7_imagenet&en_category=ImageClassification) | Simultaneous resolution for models through the scaling of number of channels and depth. SOTA's accuracy can be achieved with very few parameters. |
| [xception71](https://www.paddlepaddle.org.cn/hubdetail?name=xception71_imagenet&en_category=ImageClassification) | Improvements to inceptiv-v3. Replace regular convolution with deeply separable convolution. It reduces the number of parameters and increases the accuracy. |
| [dpn107](https://www.paddlepaddle.org.cn/hubdetail?name=dpn107_imagenet&en_category=ImageClassification) | A fusion of densenet and resnext features. |
| [DarkNet53](https://www.paddlepaddle.org.cn/hubdetail?name=darknet53_imagenet&en_category=ImageClassification) | The detection framework yolov3 uses backbone. It has good performance for both classification and detection tasks. |
| [DenseNet161](https://www.paddlepaddle.org.cn/hubdetail?name=densenet161_imagenet&en_category=ImageClassification) | A densely connected network structure is proposed, which is more conducive to the flow of information. |
| [ResNeXt152_vd](https://www.paddlepaddle.org.cn/hubdetail?name=resnext152_64x4d_imagenet&en_category=ImageClassification) | The concept of cardinatity is proposed as an additional measure of model complexity, effectively improving model accuracy. |
modules/image/face_detection/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the Web official documents -> [Face Detection](https://www.paddlepaddle.org.cn/hublist)**
### Face Detection
Face detection belongs to an important branch of the object detection. It has become one of the most important tasks in target detection in recent years in the security market, face recognition, and face security.
- Recommended Models
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Face Detection](https://www.paddlepaddle.org.cn/hubdetail?name=pyramidbox_lite_server&en_category=FaceDetection) | Baidu self-developed model. It is the WIDER Face dataset ►Champion Model◄ in March 2018. |
| [Ultra Lightweight Face Detection](https://www.paddlepaddle.org.cn/hubdetail?name=ultra_light_fast_generic_face_detector_1mb_640&en_category=FaceDetection) | Real-time ultra-lightweight general-purpose face detection model designed for edge computing devices or low computational power devices (e.g., ARM inference). It can perform real-time face detection inference in general-purpose scenarios in low computational power devices, such as ARM. |
| [Mask face detection and recognition.](https://www.paddlepaddle.org.cn/hubdetail?name=pyramidbox_lite_server_mask&en_category=FaceDetection) | It is the First open source mask face detection and recognition model in the industry, with attracting widespread attention. |
modules/image/keypoint_detection/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the Web official document -> [Key Points Detection](https://www.paddlepaddle.org.cn/hublist)**
#### Key Point Detection
Pose Estimation refers to the detection of key points in the human body, such as joints, facial features, and so on. It describes information about the human skeleton through these key points. Pose estimation is essential for describing human posture and predicting human behaviors. It is the basis for many computer vision tasks, such as motion classification, abnormal behavior detection, and automated driving.
- Recommended Models
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Single Person - Pose Estimation](https://www.paddlepaddle.org.cn/hubdetail?name=human_pose_estimation_resnet50_mpii&en_category=KeyPointDetection) | It can be used in behavior recognition, person tracking, gait recognition and other related fields. Specific applications include intelligent video surveillance, patient monitoring systems, human-computer interaction, virtual reality, human animation, smart home, intelligent security, athlete training, and so on. |
| [Multi Person - Pose Estimation](https://www.paddlepaddle.org.cn/hubdetail?name=openpose_body_estimation&en_category=KeyPointDetection) | It can be used in behavior recognition, person tracking, gait recognition and other related fields. Specific applications include intelligent video surveillance, patient monitoring systems, human-computer interaction, virtual reality, human animation, smart home, intelligent security, athlete training, and so on. |
| [Facial Key Points Detection](https://www.paddlepaddle.org.cn/hubdetail?name=face_landmark_localization&en_category=KeyPointDetection) | It can be used for face recognition, expression analysis, 3D face reconstruction, 3D animation and other face-related issues. It supports multiple face detection in the same picture. |
| [Hand Key Point Detection](https://www.paddlepaddle.org.cn/hubdetail?name=hand_pose_localization&en_category=KeyPointDetection) | It can be used for gesture recognition. In combination with Pose Estimation, it can be used for abnormal behavior detection and many other scenarios. |
modules/image/object_detection/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to Web official document -> [Object Detection](https://www.paddlepaddle.org.cn/hublist)**
### Object Detection
The goal of an object detection task is to allow a computer to find the locations of all objects when an image or a video frame is given. In addition, the specific category of each object is specified. For the computer, what it "sees" are the digits encoded. It is difficult for the computer to understand the high level semantic concept - whether it is a person or an object in the image or video frame. It is even more difficult to locate where the object exists in the image.
- Recommended Models
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [YOLOv3](https://www.paddlepaddle.org.cn/hubdetail?name=yolov3_darknet53_coco2017&en_category=ObjectDetection) | Compared with the original author, the implementation accuracy is increased by 5.9 absolute percents, and the performance is extremely optimized. |
| [Pedestrian detection](https://www.paddlepaddle.org.cn/hubdetail?name=yolov3_darknet53_pedestrian&en_category=ObjectDetection) | Baidu's self-developed model. It is the massive private dataset training, and can be applied to intelligent video surveillance, human behavior analysis, passenger flow statistics system, intelligent transportation and other fields. |
| [Vehicle inspection](https://www.paddlepaddle.org.cn/hubdetail?name=yolov3_darknet53_vehicles&en_category=ObjectDetection) | Baidu's self-developed model. It supports recognition of car, truck, bus, motorbike, tricycle and other vehicle types. |
modules/image/semantic_segmentation/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the Web official document -> [Image Segmentation](https://www.paddlepaddle.org.cn/hublist)**
### Image Segmentation
Image semantic segmentation, as the name implies, is the grouping/segmentation of image pixels according to the semantic meanings. Image semantics refers to the understanding of the image content, for example, depict what is done in which place by which object. Segmentation refers to the labeling of each pixel in an image as to which category it belongs. In recent years, it has been used to segment street scenes to avoid pedestrians and vehicles in unmanned vehicle driving technology, and to assist in diagnosis in medical image analysis.
- Selected Model
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Human Image Segmentation](https://www.paddlepaddle.org.cn/hubdetail?name=deeplabv3p_xception65_humanseg&en_category=ImageSegmentation) | Baidu Self-Built DataSet Training. It has the excellent effect of human Image Segmentation. |
| [Human Body Analysis](https://www.paddlepaddle.org.cn/hubdetail?name=ace2p&en_category=ImageSegmentation) | CVPR2019 LIP Challenge Triple Crown. It is a necessity for a human body analysis task. |
| [Pneumonia CT Imaging Analysis](https://www.paddlepaddle.org.cn/hubdetail?name=Pneumonia_CT_LKM_PP&en_category=ImageSegmentation) | Help LinkingMed Open source First Case in the industry Pneumonia CT Imaging Analysis Model
modules/image/text_recognition/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the web official document -> [OCR](https://www.paddlepaddle.org.cn/hublist)**
### OCR
Optical Character Recognition (OCR) is one of the important tasks of computer vision, mainly used for the extraction of text information from images. It has the important industrial practice significance.
- Recommended Models
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Ultra Lightweight - Chinese \& English OCR](https://www.paddlepaddle.org.cn/hubdetail?name=chinese_ocr_db_crnn_mobile&en_category=TextRecognition) | It has the industry's smallest open source, with 8.1M ultra-lightweight Chinese and English recognition model. Support Chinese and English recognition. In addition, it supports tilt, vertical and other multi-directional OCR. It is highly recommended. |
| [High Precision - Chinese \& English OCR](https://www.paddlepaddle.org.cn/hubdetail?name=chinese_ocr_db_crnn_mobile&en_category=TextRecognition) | It has the industry's best open source effect, with 155M high-precision Chinese and English recognition model. Support Chinese and English recognition. In addition, it supports tilt, vertical and other multi-directional OCR. It is highly recommended. |
| [German - Ultra lightweight OCR](https://www.paddlepaddle.org.cn/hubdetail?name=german_ocr_db_crnn_mobile&en_category=TextRecognition) | German OCR, ultra-lightweight |
| [French - Ultra Lightweight OCR](https://www.paddlepaddle.org.cn/hubdetail?name=french_ocr_db_crnn_mobile&en_category=TextRecognition) | French OCR, ultra lightweight |
| [Japanese - Ultra Lightweight OCR](https://www.paddlepaddle.org.cn/hubdetail?name=japan_ocr_db_crnn_mobile&en_category=TextRecognition) | Japanese OCR, ultra lightweight |
| [Korean - Ultra Lightweight OCR](https://www.paddlepaddle.org.cn/hubdetail?name=korean_ocr_db_crnn_mobile&en_category=TextRecognition) | Korean OCR, ultra lightweight |
modules/text/language_model/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the Web official document -> [Language Model](https://www.paddlepaddle.org.cn/hublist)**
### Language Model
- Recommended Model
| Model Name | Module Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Word embedding model](https://www.paddlepaddle.org.cn/hubdetail?name=word2vec_skipgram&en_category=SemanticModel) | In the massive Baidu search dataset, the Chinese character pre-training word embedding is obtained through pre-training. It supports Fine-tune. The vocabulary list size of Word2vec's pre-training dataset is 1700249. The word embedding dimension is 128. |
| [Text similarity](https://www.paddlepaddle.org.cn/hubdetail?name=simnet_bow&en_category=SemanticModel) | Based on the two texts entered by a user, the score of the text similarity is calculated. |
| [ERNIE](https://www.paddlepaddle.org.cn/hubdetail?name=ERNIE&en_category=SemanticModel) |Based on Chinese corpus self-developed models such as encyclopedia, information, and forum dialogue data, it can be used for tasks such as text classification, sequence annotation, and reading comprehension.
modules/text/lexical_analysis/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the Web official document -> [Lexical Analysis](https://www.paddlepaddle.org.cn/hublist)**
### Lexical Analysis
- Recommended Models
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Lexical Analysis – LAC](https://www.paddlepaddle.org.cn/hubdetail?name=lac&en_category=LexicalAnalysis) | Baidu's self-developed lexical analysis model. It is able to complete the tasks of Chinese word segmentation, lexical labeling and proper name recognition roughly. It is evaluated on Baidu's own dataset. LAC effect: Precision=88.0%, Recall=88.7%, F1-Score=88.4%. |
| [Word Segmentation Network - jieba](https://www.paddlepaddle.org.cn/hubdetail?name=jieba_paddle&en_category=LexicalAnalysis) | This module is a word segmentation network (bi-directional GRU) built by jieba using the PaddlePaddle deep learning framework. It also supports the traditional word segmentation methods of jieba, such as exact mode, full mode, search engine mode and other word segmentation modes. |
modules/text/sentiment_analysis/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the Web official document -> [Sentiment Analysis](https://www.paddlepaddle.org.cn/hublist)**
### Sentiment Analysis
Sentiment Classification (Senta) can automatically determine the sentiment polarity category of Chinese texts with subjective descriptions and give the corresponding confidence level. This can help enterprises understand users' consumption habits, analyze hot topics, and monitor public opinion at crisis, and provide favorable decision support for enterprises.
- Recommended Models
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Sentiment Analysis - LSTM](https://www.paddlepaddle.org.cn/hubdetail?name=senta_lstm&en_category=SentimentAnalysis) | Implementation of sentiment tendency analysis LSTM |
| [Sentiment Analysis - GRU](https://www.paddlepaddle.org.cn/hubdetail?name=senta_gru&en_category=SentimentAnalysis) | Implementation of sentiment tendency analysis GRU |
| [Conversation Sentiment Recognition](https://www.paddlepaddle.org.cn/hubdetail?name=emotion_detection_textcnn&en_category=SentimentAnalysis) | For user texts in an intelligent conversation scene, it automatically determines the sentiment category of the texts and assigns a corresponding confidence level. The sentiment type is classified as positive, negative, or neutral. |
modules/text/syntactic_analysis/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the Web official document -> [Sentiment Analysis](https://www.paddlepaddle.org.cn/hublist)**
### Sentiment Analysis
Sentiment Classification (Senta) can automatically determine the sentiment polarity category of Chinese texts with subjective descriptions and give the corresponding confidence level. This can help enterprises understand users' consumption habits, analyze hot topics, and monitor public opinion at crisis, and provide favorable decision support for enterprises.
- Recommended Models
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Syntactic Analysis - DDParser](https://www.paddlepaddle.org.cn/hubdetail?name=ddparser&en_category=SyntacticAnalysis) | Baidu Dependency Parser (DDParser) is a Chinese dependency syntactic analysis tool developed by Baidu NLP based on large-scale annotation data and deep learning platform Paddle. This can help users directly obtain related word pairs and long-distance dependent word pairs in input texts. |
modules/text/text_generation/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the Web official document -> [Text Generation]https://www.paddlepaddle.org.cn/hublist)**
### Text Generation
- Recommended Models
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [AI Poem Writing](https://www.paddlepaddle.org.cn/hubdetail?name=ernie_gen_poetry&en_category=TextGeneration) | Generate the next lines of poetry automatically according to the input of the previous line. |
| [AI Love Word](https://www.paddlepaddle.org.cn/hubdetail?name=ernie_gen_lover_words&en_category=TextGeneration) | Generate the next lines of poetry automatically according to the input of the previous line. |
| [Generative Dialogues](https://www.paddlepaddle.org.cn/hubdetail?name=plato2_en_large&en_category=TextGeneration) | Interactive multi-round question and answer. It currently supports only English. |
| [Poem Writing by Watching an Image](https://www.paddlepaddle.org.cn/hubdetail?name=reading_pictures_writing_poems&en_category=TextGeneration) | The model can automatically generate an ancient poem according to an images. |
modules/text/text_review/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the Web official document -> [Text Review](https://www.paddlepaddle.org.cn/hublist)**
### Text Review
- Recommended Models
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Porn recognition - LSTM](https://www.paddlepaddle.org.cn/hubdetail?name=porn_detection_lstm&en_category=TextCensorship) | The porn detection model can automatically determine whether the text is pornographic or not and give the corresponding confidence level, and identify porn descriptions, vulgar dating, and obscene literature in the text. The core uses the LSTM network structure and performs the word segmentation according to the word granularity. |
| [Porn recognition - GRU](https://www.paddlepaddle.org.cn/hubdetail?name=porn_detection_gru&en_category=TextCensorship) | The porn detection model can automatically determine whether the text is pornographic or not and give the corresponding confidence level, and identify porn descriptions, vulgar dating, and obscene literature in the text. The core uses the GRU network structure and performs the word segmentation according to the word granularity. |
| [Porn recognition - CNNN](https://www.paddlepaddle.org.cn/hubdetail?name=porn_detection_cnn&en_category=TextCensorship) | The porn detection model can automatically determine whether the text is pornographic or not and give the corresponding confidence level, and identify porn descriptions, vulgar dating, and obscene literature in the text. The core uses the CNN network structure and performs the word segmentation according to the word granularity. |
modules/video/README_en.md 0 → 100644
浏览文件 @ 2b17b1f8
## **For better user experience, refer to the Web official document -> [Video Classification](https://www.paddlepaddle.org.cn/hublist)**
### Video Classification
Video data contains lots of information such as voice and image. For the video task, the voice and images are processed, and the contextual information is extracted from time series of video frames. The video classification model is suitable for quick tagging of various short videos.
- Recommended Models
| Model Name | Model Introduction |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| [Video Classification](https://www.paddlepaddle.org.cn/hubdetail?name=videotag_tsn_lstm&en_category=VideoClassification) | Based on the pre-trained video classification model for tens of millions of short videos, it supports more than 3000 short video tags, achieving 89.9% Top-1 accuracy in the actual business with good generalization ability.
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或