diff --git a/doc/doc_ch/style_text_rec.md b/doc/doc_ch/style_text_rec.md index 52d104468c26059f96454fd3877202fff22f87e4..9562040b930721a5019ea659d63ce5b6ffe8474b 100644 --- a/doc/doc_ch/style_text_rec.md +++ b/doc/doc_ch/style_text_rec.md @@ -1,24 +1,35 @@ -### 快速上手 +## Style Text Rec -Style-Text是对百度自研文本编辑算法《Editing Text in the Wild》中提出的SRNet网络的改进，不同于常用的GAN的方法只选择一个分支，该工具将文本合成任务分解为三个子模块，文本风格迁移模块、背景抽取模块和前背景融合模块，来提升合成数据的效果。下图显示了一些示例结果。 +### 目录 +[TOC] -此外，在实际铭牌文本识别场景和韩语文本识别场景，验证了该合成工具的有效性，具体如下。 +### 工具简介 +
+ +
+Style-Text是对百度自研文本编辑算法《Editing Text in the Wild》中提出的SRNet网络的改进，不同于常用的GAN的方法只选择一个分支，该工具将文本合成任务分解为三个子模块，文本风格迁移模块、背景抽取模块和前背景融合模块，来提升合成数据的效果。下图显示了一些示例结果。 +
+ + +
-#### 环境配置 +此外，在实际铭牌文本识别场景和韩语文本识别场景，验证了该合成工具的有效性。 -1. 参考[快速安装](./installation.md)，安装PaddlePaddle并准备环境。强烈建议您使用python3环境。 +### 环境配置 + +1. 参考[快速安装](./installation.md)，安装PaddleOCR。强烈建议您使用python3环境。 2. 进入`style_text_rec`目录，下载模型，并解压： ```bash -cd tools/style_text_rec -wget /path/to/style_text_models.zip +cd style_text_rec +wget /https://paddleocr.bj.bcebos.com/dygraph_v2.0/style_text/style_text_models.zip unzip style_text_models.zip ``` -您可以在 [此处](https://paddleocr.bj.bcebos.com/dygraph_v2.0/style_text/style_text_models.zip) 下载模型文件。如果您选择了其他下载位置，请在`configs/config.yml`中修改模型文件的地址，修改时需要同时修改这三个配置： +如果您将模型保存再其他位置，请在`configs/config.yml`中修改模型文件的地址，修改时需要同时修改这三个配置： ``` bg_generator: @@ -31,17 +42,19 @@ fusion_generator: pretrain: style_text_models/fusion_generator ``` - - -#### 合成单张图片 +### 快速上手 1. 运行tools/synth_image，生成示例图片： ```python -python -m tools.synth_image -c configs/config.yml +python3 -m tools.synth_image -c configs/config.yml ``` -1. 运行后，会生成`fake_busion.jpg`，即为最终结果。除此之外，程序还会生成并保存中间结果： +1. 运行后，会生成`fake_busion.jpg`，即为最终结果。 +
+ +
+除此之外，程序还会生成并保存中间结果： * `fake_bg.jpg`：为风格参考图去掉文字后的背景； * `fake_text.jpg`：是用提供的字符串，仿照风格参考图中文字的风格，生成在灰色背景上的文字图片。 @@ -54,26 +67,6 @@ python -m tools.synth_image -c configs/config.yml ### 高级使用 -#### 组件介绍 - -`Style Text Rec`主要包含以下组件： - -* `style_samplers`：风格图片采样器，负责返回风格图片。目前我们提供了`DatasetSampler`，可以从一个有标注的数据集中采样。 - -* `corpus_generators`：语料生成器，负责生成语料。目前提供了两种语料成生成器： - * `EnNumCorpus`：根据给定的长度生成随机字符串，字符可能是大小写英文字母、数字和空格。 - * `FileCorpus`：读取指定的文本文件，并随机返回其中的单词. - -* `text_drawers`：标准字体图片生成器，负责根据输入的语料，生成标准字体的图片。注意，使用该组件时，一定要根据语料修改对应的语言信息，否则可能会书写失败。 - -* `predictors`：预测器，根据给定的风格图片和标准字体图片，调用深度学习模型，生成新的数据。`predictor`是整个算法的核心模块。 - -* `writers`：文件输出器，负责将合成的图片与标签文件写入硬盘。 - -* `synthesisers`：合成器，负责调用各个模块，完成数据合成。 - -### 合成数据集 - 在开始合成数据集前，需要准备一些素材。 首先，需要风格图片作为合成图片的参考依据，这些数据可以是用作训练OCR识别模型的数据集。本例中使用带有标注文件的数据集作为风格图片. @@ -84,7 +77,7 @@ python -m tools.synth_image -c configs/config.yml * `image_home`：风格图片目录； * `label_file`：风格图片路径列表文件，如果所用数据集有label，则label_file为label文件路径； * `with_label`：标志`label_file`是否为label文件。 - + * `CorpusGenerator`： * `method`：语料生成方法，目前有`FileCorpus`和`EnNumCorpus`可选。如果使用`EnNumCorpus`，则不需要填写其他配置，否则需要修改`corpus_file`和`language`； * `language`：语料的语种； @@ -99,11 +92,52 @@ python -m tools.synth_image -c configs/config.yml 3. 如果您想使用并行方式来快速合成数据，可以通过启动多个进程，在启动时需要指定不同的`tag`（`-t`），如下所示: ```bash - python -m tools.synth_dataset -t 0 -c configs/dataset_config.yml - python -m tools.synth_dataset -t 1 -c configs/dataset_config.yml + python3 -m tools.synth_dataset -t 0 -c configs/dataset_config.yml + python3 -m tools.synth_dataset -t 1 -c configs/dataset_config.yml ``` -### 使用合成数据集进行OCR识别训练 - -在完成上述操作后，即可得到用于OCR识别的合成数据集，接下来请参考[OCR识别文档](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_ch/recognition.md#%E5%90%AF%E5%8A%A8%E8%AE%AD%E7%BB%83)，完成训练。 \ No newline at end of file +### 应用示例 + +在完成上述操作后，即可得到用于OCR识别的合成数据集，接下来请参考[OCR识别文档](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_ch/recognition.md#%E5%90%AF%E5%8A%A8%E8%AE%AD%E7%BB%83)，完成训练。 + +### 项目结构 + +. +|-- arch +| |-- base_module.py +| |-- decoder.py +| |-- encoder.py +| |-- spectral_norm.py +| `-- style_text_rec.py +|-- configs +| |-- config.yml +| `-- dataset_config.yml +|-- engine +| |-- corpus_generators.py +| |-- predictors.py +| |-- style_samplers.py +| |-- synthesisers.py +| |-- text_drawers.py +| `-- writers.py +|-- examples +| |-- corpus +| | `-- example.txt +| |-- image_list.txt +| `-- style_images +| |-- 1.jpg +| `-- 2.jpg +|-- fonts +| |-- ch_standard.ttf +| |-- en_standard.ttf +| `-- ko_standard.ttf +|-- tools +| |-- __init__.py +| |-- synth_dataset.py +| `-- synth_image.py +`-- utils + |-- config.py + |-- load_params.py + |-- logging.py + |-- math_functions.py + `-- sys_funcs.py \ No newline at end of file diff --git a/doc/imgs_style_text/1.png b/doc/imgs_style_text/1.png new file mode 100644 index 0000000000000000000000000000000000000000..8f7574ba2f723ac82241fec6dc52828713a5d293 Binary files /dev/null and b/doc/imgs_style_text/1.png differ diff --git a/doc/imgs_style_text/2.png b/doc/imgs_style_text/2.png new file mode 100644 index 0000000000000000000000000000000000000000..ce9bf4712a551b9d9d27eae00f9c7b9b5845d8b3 Binary files /dev/null and b/doc/imgs_style_text/2.png differ diff --git a/doc/imgs_style_text/3.png b/doc/imgs_style_text/3.png new file mode 100644 index 0000000000000000000000000000000000000000..0fb73a31f58c1c476cf84f3c507f0af6523385f4 Binary files /dev/null and b/doc/imgs_style_text/3.png differ diff --git a/doc/imgs_style_text/4.jpg b/doc/imgs_style_text/4.jpg new file mode 100644 index 0000000000000000000000000000000000000000..5fda9548632b63e55b42315dca4a5b9cec2a353c Binary files /dev/null and b/doc/imgs_style_text/4.jpg differ diff --git a/style_text_rec/__init__.py b/style_text_rec/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/style_text_rec/arch/__init__.py b/style_text_rec/arch/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/tools/style_text_rec/arch/base_module.py b/style_text_rec/arch/base_module.py similarity index 100% rename from tools/style_text_rec/arch/base_module.py rename to style_text_rec/arch/base_module.py diff --git a/tools/style_text_rec/arch/decoder.py b/style_text_rec/arch/decoder.py similarity index 100% rename from tools/style_text_rec/arch/decoder.py rename to style_text_rec/arch/decoder.py diff --git a/tools/style_text_rec/arch/encoder.py b/style_text_rec/arch/encoder.py similarity index 100% rename from tools/style_text_rec/arch/encoder.py rename to style_text_rec/arch/encoder.py diff --git a/tools/style_text_rec/arch/spectral_norm.py b/style_text_rec/arch/spectral_norm.py similarity index 100% rename from tools/style_text_rec/arch/spectral_norm.py rename to style_text_rec/arch/spectral_norm.py diff --git a/tools/style_text_rec/arch/style_text_rec.py b/style_text_rec/arch/style_text_rec.py similarity index 100% rename from tools/style_text_rec/arch/style_text_rec.py rename to style_text_rec/arch/style_text_rec.py diff --git a/tools/style_text_rec/configs/config.yml b/style_text_rec/configs/config.yml similarity index 100% rename from tools/style_text_rec/configs/config.yml rename to style_text_rec/configs/config.yml diff --git a/tools/style_text_rec/configs/dataset_config.yml b/style_text_rec/configs/dataset_config.yml similarity index 100% rename from tools/style_text_rec/configs/dataset_config.yml rename to style_text_rec/configs/dataset_config.yml diff --git a/style_text_rec/engine/__init__.py b/style_text_rec/engine/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/tools/style_text_rec/engine/corpus_generators.py b/style_text_rec/engine/corpus_generators.py similarity index 100% rename from tools/style_text_rec/engine/corpus_generators.py rename to style_text_rec/engine/corpus_generators.py diff --git a/tools/style_text_rec/engine/predictors.py b/style_text_rec/engine/predictors.py similarity index 100% rename from tools/style_text_rec/engine/predictors.py rename to style_text_rec/engine/predictors.py diff --git a/tools/style_text_rec/engine/style_samplers.py b/style_text_rec/engine/style_samplers.py similarity index 100% rename from tools/style_text_rec/engine/style_samplers.py rename to style_text_rec/engine/style_samplers.py diff --git a/tools/style_text_rec/engine/synthesisers.py b/style_text_rec/engine/synthesisers.py similarity index 100% rename from tools/style_text_rec/engine/synthesisers.py rename to style_text_rec/engine/synthesisers.py diff --git a/tools/style_text_rec/engine/text_drawers.py b/style_text_rec/engine/text_drawers.py similarity index 100% rename from tools/style_text_rec/engine/text_drawers.py rename to style_text_rec/engine/text_drawers.py diff --git a/tools/style_text_rec/engine/writers.py b/style_text_rec/engine/writers.py similarity index 100% rename from tools/style_text_rec/engine/writers.py rename to style_text_rec/engine/writers.py diff --git a/tools/style_text_rec/examples/corpus/example.txt b/style_text_rec/examples/corpus/example.txt similarity index 100% rename from tools/style_text_rec/examples/corpus/example.txt rename to style_text_rec/examples/corpus/example.txt diff --git a/tools/style_text_rec/examples/image_list.txt b/style_text_rec/examples/image_list.txt similarity index 100% rename from tools/style_text_rec/examples/image_list.txt rename to style_text_rec/examples/image_list.txt diff --git a/tools/style_text_rec/examples/style_images/1.jpg b/style_text_rec/examples/style_images/1.jpg similarity index 100% rename from tools/style_text_rec/examples/style_images/1.jpg rename to style_text_rec/examples/style_images/1.jpg diff --git a/tools/style_text_rec/examples/style_images/2.jpg b/style_text_rec/examples/style_images/2.jpg similarity index 100% rename from tools/style_text_rec/examples/style_images/2.jpg rename to style_text_rec/examples/style_images/2.jpg diff --git a/tools/style_text_rec/fonts/ch_standard.ttf b/style_text_rec/fonts/ch_standard.ttf similarity index 100% rename from tools/style_text_rec/fonts/ch_standard.ttf rename to style_text_rec/fonts/ch_standard.ttf diff --git a/tools/style_text_rec/fonts/en_standard.ttf b/style_text_rec/fonts/en_standard.ttf similarity index 100% rename from tools/style_text_rec/fonts/en_standard.ttf rename to style_text_rec/fonts/en_standard.ttf diff --git a/tools/style_text_rec/fonts/ko_standard.ttf b/style_text_rec/fonts/ko_standard.ttf similarity index 100% rename from tools/style_text_rec/fonts/ko_standard.ttf rename to style_text_rec/fonts/ko_standard.ttf diff --git a/style_text_rec/tools/__init__.py b/style_text_rec/tools/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/tools/style_text_rec/tools/synth_dataset.py b/style_text_rec/tools/synth_dataset.py similarity index 100% rename from tools/style_text_rec/tools/synth_dataset.py rename to style_text_rec/tools/synth_dataset.py diff --git a/tools/style_text_rec/tools/synth_image.py b/style_text_rec/tools/synth_image.py similarity index 71% rename from tools/style_text_rec/tools/synth_image.py rename to style_text_rec/tools/synth_image.py index b7d3c5d604c53b6c22199b27d0e9015302a54f41..22cfb9ee2b862f5549313c11b3e48a54a4bebd3f 100644 --- a/tools/style_text_rec/tools/synth_image.py +++ b/style_text_rec/tools/synth_image.py @@ -15,6 +15,7 @@ import os import cv2 import sys import glob +from argparse import ArgumentParser, RawDescriptionHelpFormatter from engine.synthesisers import ImageSynthesiser @@ -24,11 +25,14 @@ sys.path.append(os.path.abspath(os.path.join(__dir__, '..'))) def synth_image(): + args = ArgsParser().parse_args() image_synthesiser = ImageSynthesiser() - img = cv2.imread("examples/style_images/1.jpg") - corpus = "PaddleOCR" - language = "en" - synth_result = image_synthesiser.synth_image(corpus, img, language) + style_image_path = args.style_image + img = cv2.imread(style_image_path) + text_corpus = args.text_corpus + language = args.language + + synth_result = image_synthesiser.synth_image(text_corpus, img, language) fake_fusion = synth_result["fake_fusion"] fake_text = synth_result["fake_text"] fake_bg = synth_result["fake_bg"] @@ -73,6 +77,26 @@ def batch_synth_images(): print(cno, corpus_num, sno, style_img_num) +class ArgsParser(ArgumentParser): + def __init__(self): + super(ArgsParser, self).__init__( + formatter_class=RawDescriptionHelpFormatter) + self.add_argument("-c", "--config", help="configuration file to use") + self.add_argument( + "--style_image", default="examples/style_images/1.jpg", help="tag for marking worker") + self.add_argument( + "--text_corpus", default="PaddleOCR", help="tag for marking worker") + self.add_argument( + "--language", default="en", help="tag for marking worker") + + def parse_args(self, argv=None): + args = super(ArgsParser, self).parse_args(argv) + assert args.config is not None, \ + "Please specify --config=configure_file_path." + return args + + + if __name__ == '__main__': # batch_synth_images() synth_image() diff --git a/style_text_rec/utils/__init__.py b/style_text_rec/utils/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/tools/style_text_rec/utils/config.py b/style_text_rec/utils/config.py similarity index 100% rename from tools/style_text_rec/utils/config.py rename to style_text_rec/utils/config.py diff --git a/tools/style_text_rec/utils/load_params.py b/style_text_rec/utils/load_params.py similarity index 100% rename from tools/style_text_rec/utils/load_params.py rename to style_text_rec/utils/load_params.py diff --git a/tools/style_text_rec/utils/logging.py b/style_text_rec/utils/logging.py similarity index 100% rename from tools/style_text_rec/utils/logging.py rename to style_text_rec/utils/logging.py diff --git a/tools/style_text_rec/utils/math_functions.py b/style_text_rec/utils/math_functions.py similarity index 100% rename from tools/style_text_rec/utils/math_functions.py rename to style_text_rec/utils/math_functions.py diff --git a/tools/style_text_rec/utils/sys_funcs.py b/style_text_rec/utils/sys_funcs.py similarity index 100% rename from tools/style_text_rec/utils/sys_funcs.py rename to style_text_rec/utils/sys_funcs.py