{ "cells": [ { "cell_type": "markdown", "id": "03cfffa3-2398-4d55-bf1e-fe3e01f10d68", "metadata": {}, "source": [ "## 1. PP-Matting 模型简介\n", "\n", "在众多图像抠图算法中，为了追求精度，往往需要输入trimap作为辅助信息，但这极大限制了算法的使用性。PP-Matting作为一种trimap-free的抠图方法，有效克服了辅助信息带来的弊端，在Composition-1k和Distinctions-646数据集中取得了SOTA的效果。PP-Matting利用语义分支（SCB）提取图片高级语义信息并通过引导流设计（Guidance Flow）逐步引导高分辨率细节分支（HRDB）对过度区域的细节提取，最后通过融合模块实现语义和细节的融合得到最终的alpha matte。\n", "\n", "更多细节可参考技术报告：https://arxiv.org/abs/2204.09433 。\n", "\n", "更多关于PaddleMatting的内容，可以点击 https://github.com/PaddlePaddle/PaddleSeg/tree/develop/Matting 进行了解。\n", "\n" ] }, { "cell_type": "markdown", "id": "8cbcf510-dc56-43f9-9864-5e1de3c7b272", "metadata": {}, "source": [ "## 2. 模型效果及应用场景\n", "PP-Matting在人像上的抠图效果如下：\n", "

\n", "\n", "

\n" ] }, { "cell_type": "markdown", "id": "8d9da224-edd4-4c9e-ab2d-cba7ee5a92a4", "metadata": {}, "source": [ "## 3. 模型如何使用" ] }, { "cell_type": "markdown", "id": "2ac57be7-c00f-441e-ad82-635f6268b6bd", "metadata": {}, "source": [ "### 3.1 模型推理\n", "* 下载" ] }, { "cell_type": "code", "execution_count": null, "id": "3ea22cb4-5bed-4ce0-858b-3bbb342e8ccf", "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "!mkdir -p ~/work\n", "%cd ~/work\n", "!git clone --depth 1 https://gitee.com/paddlepaddle/PaddleSeg" ] }, { "cell_type": "markdown", "id": "fd7142e9-5202-4a4d-8a03-36fcc5ea0259", "metadata": {}, "source": [ "* 安装" ] }, { "cell_type": "code", "execution_count": null, "id": "a7fd9575-4116-4fa7-86ee-0b2db7417300", "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "%cd ~/work/PaddleSeg/Matting\n", "!pip install -r requirements.txt" ] }, { "cell_type": "markdown", "id": "a0dd2390-e635-432b-9e0b-d7c9323f81a9", "metadata": {}, "source": [ "* 快速体验\n", "恭喜！ 您已经成功安装了PaddleSeg，接下来快速体验图像抠图效果" ] }, { "cell_type": "code", "execution_count": null, "id": "81736dd2-e90d-4dac-b3c0-58424e2e8dc4", "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "# 下载预训练模型\n", "!wget https://paddleseg.bj.bcebos.com/matting/models/ppmatting-hrnet_w18-human_512.pdparams\n", "# 下载图片\n", "!wget https://user-images.githubusercontent.com/30919197/200645066-6898ec5a-f1c5-4bf7-aa41-473a29977a86.jpeg\n", "# 在GPU上预测一张图片\n", "!export CUDA_VISIBLE_DEVICES=0\n", "!python tools/predict.py \\\n", " --config configs/ppmatting/ppmatting-hrnet_w18-human_512.yml \\\n", " --model_path ppmatting-hrnet_w18-human_512.pdparams \\\n", " --image_path 200645066-6898ec5a-f1c5-4bf7-aa41-473a29977a86.jpeg \\\n", " --save_dir ./output/results \\\n", " --fg_estimate True" ] }, { "cell_type": "markdown", "id": "3ab60ad4-2908-40e2-9c08-34ac7978ef16", "metadata": {}, "source": [ "会在output/results文件夹下生成一个alpha图像和一个rgba图像。\n", "\n", "结果如下图：\n", "
\n", " \n", "\n", "*
\n" ] }, { "cell_type": "markdown", "id": "98c083d9-acd1-4e0a-855b-0a048983f251", "metadata": {}, "source": [ "### 3.2 模型训练\n", " * 克隆PaddleSeg仓库（详见3.1）。\n", " \n", " 具体训练参考PaddleSeg仓库Matting子目录相关教程\n", "https://github.com/PaddlePaddle/PaddleSeg/tree/develop/Matting 。" ] }, { "cell_type": "markdown", "id": "e9b05c1e-cb59-4dd7-83c0-cb30dfefba60", "metadata": {}, "source": [ "## 4. 原理\n", "\n", "
\n", "\n", "*
\n", "\n", "* 分支设计，明确语义预测和细节预测任务。\n", "* SCB(Semantic Context Branch)分支进行语义预测，保证图像整体预测正确性，其将图像粗略的分为三个部分，即前景、背景和过度区域。\n", "* HRDB（High-Resolution Detail Branch）维持高分辨率特征提取过程，保证细节不受损失。\n", "* 引导流结构设计（Guidance Flow）使HRDB分支获取SCB分支提取的语义信息，使HRDB分支能专注与过度区域的细节预测。\n" ] }, { "cell_type": "markdown", "id": "ae7e1030-9044-44f5-a0d3-5148e59d4753", "metadata": {}, "source": [ "## 5. 注意事项\n", "* 基于公开数据集Composition-1K和Distinctions-646的训练和预测需发邮件向作者申请。\n", "* PP-Matting开源了人像预训练模型，可根据具体的人像抠图场景，标注少量的数据进行finetune。" ] }, { "cell_type": "markdown", "id": "7e238357-2f19-48c3-902e-9ba3c93f7818", "metadata": {}, "source": [ "## 6. 相关论文以及引用信息\n", "@article{chen2022pp,\n", " title={PP-Matting: High-Accuracy Natural Image Matting},\n", " author={Chen, Guowei and Liu, Yi and Wang, Jian and Peng, Juncai and Hao, Yuying and Chu, Lutao and Tang, Shiyu and Wu, Zewu and Chen, Zeyu and Yu, Zhiliang and others},\n", " journal={arXiv preprint arXiv:2204.09433},\n", " year={2022}\n", "}" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "py35-paddle1.2.0" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" } }, "nbformat": 4, "nbformat_minor": 5 }