README.md 6.7 KB
Newer Older
1
English | [简体中文](README_ch.md)
W
WenmuZhou 已提交
2

3 4 5 6 7 8 9 10 11 12 13 14 15
- [1. Introduction](#1-introduction)
- [2. Update log](#2-update-log)
- [3. Features](#3-features)
- [4. Results](#4-results)
  - [4.1 Layout analysis and table recognition](#41-layout-analysis-and-table-recognition)
  - [4.2 DOC-VQA](#42-doc-vqa)
- [5. Quick start](#5-quick-start)
- [6. PP-Structure System](#6-pp-structure-system)
  - [6.1 Layout analysis and table recognition](#61-layout-analysis-and-table-recognition)
    - [6.1.1 Layout analysis](#611-layout-analysis)
    - [6.1.2 Table recognition](#612-table-recognition)
  - [6.2 DOC-VQA](#62-doc-vqa)
- [7. Model List](#7-model-list)
文幕地方's avatar
文幕地方 已提交
16 17 18
  - [7.1 Layout analysis model](#71-layout-analysis-model)
  - [7.2 OCR and table recognition model](#72-ocr-and-table-recognition-model)
  - [7.3 DOC-VQA model](#73-doc-vqa-model)
W
opt doc  
WenmuZhou 已提交
19

文幕地方's avatar
update  
文幕地方 已提交
20
## 1. Introduction
21

文幕地方's avatar
update  
文幕地方 已提交
22
PP-Structure is an OCR toolkit that can be used for document analysis and processing with complex structures, designed to help developers better complete document understanding tasks
23

文幕地方's avatar
update  
文幕地方 已提交
24
## 2. Update log
文幕地方's avatar
文幕地方 已提交
25
* 2022.02.12 DOC-VQA add LayoutLMv2 model。
文幕地方's avatar
update  
文幕地方 已提交
26
* 2021.12.07 add [DOC-VQA SER and RE tasks](vqa/README.md)
27

文幕地方's avatar
update  
文幕地方 已提交
28
## 3. Features
29

文幕地方's avatar
update  
文幕地方 已提交
30
The main features of PP-Structure are as follows:
31

文幕地方's avatar
update  
文幕地方 已提交
32 33 34 35 36 37
- Support the layout analysis of documents, divide the documents into 5 types of areas **text, title, table, image and list** (conjunction with Layout-Parser)
- Support to extract the texts from the text, title, picture and list areas (used in conjunction with PP-OCR)
- Support to extract excel files from the table areas
- Support python whl package and command line usage, easy to use
- Support custom training for layout analysis and table structure tasks
- Support Document Visual Question Answering (DOC-VQA) tasks: Semantic Entity Recognition (SER) and Relation Extraction (RE)
38

文幕地方's avatar
update  
文幕地方 已提交
39
## 4. Results
40

文幕地方's avatar
update  
文幕地方 已提交
41
### 4.1 Layout analysis and table recognition
W
WenmuZhou 已提交
42

文幕地方's avatar
update  
文幕地方 已提交
43
<img src="../doc/table/ppstructure.GIF" width="100%"/>
44

文幕地方's avatar
update  
文幕地方 已提交
45
The figure shows the pipeline of layout analysis + table recognition. The image is first divided into four areas of image, text, title and table by layout analysis, and then OCR detection and recognition is performed on the three areas of image, text and title, and the table is performed table recognition, where the image will also be stored for use.
46

文幕地方's avatar
update  
文幕地方 已提交
47
### 4.2 DOC-VQA
W
WenmuZhou 已提交
48

文幕地方's avatar
update  
文幕地方 已提交
49
* SER
50 51
*
![](../doc/vqa/result_ser/zh_val_0_ser.jpg) | ![](../doc/vqa/result_ser/zh_val_42_ser.jpg)
文幕地方's avatar
update  
文幕地方 已提交
52
---|---
W
WenmuZhou 已提交
53

文幕地方's avatar
update  
文幕地方 已提交
54
Different colored boxes in the figure represent different categories. For xfun dataset, there are three categories: query, answer and header:
W
opt doc  
WenmuZhou 已提交
55

文幕地方's avatar
update  
文幕地方 已提交
56 57 58
* Dark purple: header
* Light purple: query
* Army green: answer
W
WenmuZhou 已提交
59

文幕地方's avatar
update  
文幕地方 已提交
60
The corresponding category and OCR recognition results are also marked at the top left of the OCR detection box.
W
WenmuZhou 已提交
61 62


文幕地方's avatar
update  
文幕地方 已提交
63
* RE
W
WenmuZhou 已提交
64

65
![](../doc/vqa/result_re/zh_val_21_re.jpg) | ![](../doc/vqa/result_re/zh_val_40_re.jpg)
文幕地方's avatar
update  
文幕地方 已提交
66
---|---
W
WenmuZhou 已提交
67 68


文幕地方's avatar
update  
文幕地方 已提交
69
In the figure, the red box represents the question, the blue box represents the answer, and the question and answer are connected by green lines. The corresponding category and OCR recognition results are also marked at the top left of the OCR detection box.
W
WenmuZhou 已提交
70

文幕地方's avatar
update  
文幕地方 已提交
71
## 5. Quick start
W
WenmuZhou 已提交
72

文幕地方's avatar
update  
文幕地方 已提交
73
Start from [Quick Installation](./docs/quickstart.md)
W
opt doc  
WenmuZhou 已提交
74

文幕地方's avatar
update  
文幕地方 已提交
75
## 6. PP-Structure System
W
opt doc  
WenmuZhou 已提交
76

文幕地方's avatar
update  
文幕地方 已提交
77
### 6.1 Layout analysis and table recognition
W
opt doc  
WenmuZhou 已提交
78

文幕地方's avatar
update  
文幕地方 已提交
79
![pipeline](../doc/table/pipeline.jpg)
W
WenmuZhou 已提交
80

文幕地方's avatar
update  
文幕地方 已提交
81
In PP-Structure, the image will be divided into 5 types of areas **text, title, image list and table**. For the first 4 types of areas, directly use PP-OCR system to complete the text detection and recognition. For the table area, after the table structuring process, the table in image is converted into an Excel file with the same table style.
W
opt doc  
WenmuZhou 已提交
82

文幕地方's avatar
update  
文幕地方 已提交
83
#### 6.1.1 Layout analysis
W
opt doc  
WenmuZhou 已提交
84

85
Layout analysis classifies image by region, including the use of Python scripts of layout analysis tools, extraction of designated category detection boxes, performance indicators, and custom training layout analysis models. For details, please refer to [document](layout/README.md).
W
opt doc  
WenmuZhou 已提交
86

文幕地方's avatar
update  
文幕地方 已提交
87
#### 6.1.2 Table recognition
W
opt doc  
WenmuZhou 已提交
88

文幕地方's avatar
update  
文幕地方 已提交
89
Table recognition converts table images into excel documents, which include the detection and recognition of table text and the prediction of table structure and cell coordinates. For detailed instructions, please refer to [document](table/README.md)
W
opt doc  
WenmuZhou 已提交
90

文幕地方's avatar
update  
文幕地方 已提交
91
### 6.2 DOC-VQA
W
WenmuZhou 已提交
92

文幕地方's avatar
update  
文幕地方 已提交
93
Document Visual Question Answering (DOC-VQA) if a type of Visual Question Answering (VQA), which includes Semantic Entity Recognition (SER) and Relation Extraction (RE) tasks. Based on SER task, text recognition and classification in images can be completed. Based on THE RE task, we can extract the relation of the text content in the image, such as judge the problem pair. For details, please refer to [document](vqa/README.md)
W
WenmuZhou 已提交
94

文幕地方's avatar
update  
文幕地方 已提交
95
## 7. Model List
96

文幕地方's avatar
文幕地方 已提交
97
PP-Structure Series Model List (Updating)
98

文幕地方's avatar
文幕地方 已提交
99
### 7.1 Layout analysis model
100 101 102

|model name|description|download|
| --- | --- | --- |
文幕地方's avatar
update  
文幕地方 已提交
103
| ppyolov2_r50vd_dcn_365e_publaynet | The layout analysis model trained on the PubLayNet dataset can divide image into 5 types of areas **text, title, table, picture, and list** | [PubLayNet](https://paddle-model-ecology.bj.bcebos.com/model/layout-parser/ppyolov2_r50vd_dcn_365e_publaynet.tar) |
104

文幕地方's avatar
文幕地方 已提交
105
### 7.2 OCR and table recognition model
106 107 108

|model name|description|model size|download|
| --- | --- | --- | --- |
109 110
|ch_PP-OCRv2_det_slim|[New] Slim quantization with distillation lightweight model, supporting Chinese, English, multilingual text detection| 3M |[inference model](https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_slim_quant_infer.tar)|
|ch_PP-OCRv2_rec_slim|[New] Slim qunatization with distillation lightweight model, supporting Chinese, English, multilingual text recognition| 9M |[inference model](https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_slim_quant_infer.tar) / [trained model](https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_slim_quant_train.tar) |
文幕地方's avatar
文幕地方 已提交
111 112 113
|en_ppocr_mobile_v2.0_table_structure|Table structure prediction of English table scene trained on PubLayNet dataset| 18.6M |[inference model](https://paddleocr.bj.bcebos.com/dygraph_v2.0/table/en_ppocr_mobile_v2.0_table_structure_infer.tar) / [trained model](https://paddleocr.bj.bcebos.com/dygraph_v2.1/table/en_ppocr_mobile_v2.0_table_structure_train.tar) |

### 7.3 DOC-VQA model
114

文幕地方's avatar
update  
文幕地方 已提交
115 116
|model name|description|model size|download|
| --- | --- | --- | --- |
文幕地方's avatar
文幕地方 已提交
117
|ser_LayoutXLM_xfun_zhd|SER model trained on xfun Chinese dataset based on LayoutXLM|1.4G|[inference model coming soon]() / [trained model](https://paddleocr.bj.bcebos.com/pplayout/ser_LayoutXLM_xfun_zh.tar) |
文幕地方's avatar
文幕地方 已提交
118
|re_LayoutXLM_xfun_zh|RE model trained on xfun Chinese dataset based on LayoutXLM|1.4G|[inference model coming soon]() / [trained model](https://paddleocr.bj.bcebos.com/pplayout/re_LayoutXLM_xfun_zh.tar) |
L
LDOUBLEV 已提交
119

A
andyjpaddle 已提交
120
If you need to use other models, you can download the model in [PPOCR model_list](../doc/doc_en/models_list_en.md) and  [PPStructure model_list](./docs/models_list.md)
反馈
建议
客服 返回
顶部