introduction_cn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "03cfffa3-2398-4d55-bf1e-fe3e01f10d68",
   "metadata": {},
   "source": [
    "## 1. PP-Matting 模型简介\n",
    "\n",
    "在众多图像抠图算法中，为了追求精度，往往需要输入trimap作为辅助信息，但这极大限制了算法的使用性。PP-Matting作为一种trimap-free的抠图方法，有效克服了辅助信息带来的弊端，在Composition-1k和Distinctions-646数据集中取得了SOTA的效果。PP-Matting利用语义分支（SCB）提取图片高级语义信息并通过引导流设计（Guidance Flow）逐步引导高分辨率细节分支（HRBP）对过度区域的细节提取，最后通过融合模块实现语义和细节的融合得到最终的alpha matte。\n",
    "\n",
    "更多细节可参考技术报告：https://arxiv.org/abs/2204.09433 。\n",
    "\n",
    "更多关于PaddleMatting的内容，可以点击 https://github.com/PaddlePaddle/PaddleSeg/tree/develop/Matting 进行了解。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8cbcf510-dc56-43f9-9864-5e1de3c7b272",
   "metadata": {},
   "source": [
    "## 2. 模型效果及应用场景\n",
    "PP-Matting在人像上的抠图效果如下：\n",
    "<p align=\"center\">\n",
    "<img src=\"https://user-images.githubusercontent.com/30919197/179751613-d26f2261-7bcf-4066-a0a4-4c818e7065f0.gif\" width=\"100%\" height=\"100%\">\n",
    "</p>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d9da224-edd4-4c9e-ab2d-cba7ee5a92a4",
   "metadata": {},
   "source": [
    "## 3. 模型如何使用"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ac57be7-c00f-441e-ad82-635f6268b6bd",
   "metadata": {},
   "source": [
    "### 3.1 模型推理\n",
    "* 下载"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3ea22cb4-5bed-4ce0-858b-3bbb342e8ccf",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "%cd ~/work\n",
    "!git clone --depth 1  https://gitee.com/paddlepaddle/PaddleSeg"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd7142e9-5202-4a4d-8a03-36fcc5ea0259",
   "metadata": {},
   "source": [
    "* 安装"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a7fd9575-4116-4fa7-86ee-0b2db7417300",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "%cd ~/work/PaddleSeg/Matting\n",
    "!pip install -r requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0dd2390-e635-432b-9e0b-d7c9323f81a9",
   "metadata": {},
   "source": [
    "* 快速体验\n",
    "恭喜！ 您已经成功安装了PaddleSeg，接下来快速体验图像抠图效果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "81736dd2-e90d-4dac-b3c0-58424e2e8dc4",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# 下载预训练模型\n",
    "!wget https://paddleseg.bj.bcebos.com/matting/models/ppmatting-hrnet_w18-human_512.pdparams\n",
    "# 下载图片\n",
    "!wget https://user-images.githubusercontent.com/30919197/200645066-6898ec5a-f1c5-4bf7-aa41-473a29977a86.jpeg\n",
    "# 在GPU上预测一张图片\n",
    "!export CUDA_VISIBLE_DEVICES=0\n",
    "!python tools/predict.py \\\n",
    "    --config configs/ppmatting/ppmatting-hrnet_w18-human_512.yml \\\n",
    "    --model_path ppmatting-hrnet_w18-human_512.pdparams \\\n",
    "    --image_path 200645066-6898ec5a-f1c5-4bf7-aa41-473a29977a86.jpeg \\\n",
    "    --save_dir ./output/results \\\n",
    "    --fg_estimate True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ab60ad4-2908-40e2-9c08-34ac7978ef16",
   "metadata": {},
   "source": [
    "会在output/results文件夹下生成一个alpha图像和一个rgba图像。\n",
    "\n",
    "结果如下图：\n",
    "<div align=\"center\">\n",
    "<img src=\"https://user-images.githubusercontent.com/30919197/200647513-e744118a-bd2b-4de2-b740-6eaeaccf47b6.png\"  width = \"40%\"  /> \n",
    "<img src=\"https://user-images.githubusercontent.com/30919197/200647492-7fd5fd80-2c2a-4775-ad9e-fd1d476cba48.png\"  width = \"40%\"  />\n",
    "* </div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98c083d9-acd1-4e0a-855b-0a048983f251",
   "metadata": {},
   "source": [
    "### 3.2 模型训练\n",
    " * 克隆PaddleSeg仓库（详见3.1）。\n",
    " \n",
    " 具体训练参考PaddleSeg仓库Matting子目录相关教程\n",
    "https://github.com/PaddlePaddle/PaddleSeg/tree/develop/Matting 。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9b05c1e-cb59-4dd7-83c0-cb30dfefba60",
   "metadata": {},
   "source": [
    "## 4. 原理\n",
    "\n",
    "<div align=\"center\">\n",
    "<img src=\"https://user-images.githubusercontent.com/30919197/200649929-3fa07082-7980-49f8-9f7f-38c31c46de20.png\"  width = \"80%\"  />\n",
    "* </div>\n",
    "\n",
    "* 分支设计，明确语义预测和细节预测任务。\n",
    "* SCB(Semantic Context Branch)分支进行语义预测，保证图像整体预测正确性，其将图像粗略的分为三个部分，即前景、背景和过度区域。\n",
    "* HRDB（High-Resolution Detail Branch）维持高分辨率特征提取过程，保证细节不受损失。\n",
    "* 引导流结构设计（Guidance Flow）使HRBD分支获取SCB分支提取的语义信息，使HRDB分支能专注与过度区域的细节预测。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae7e1030-9044-44f5-a0d3-5148e59d4753",
   "metadata": {},
   "source": [
    "## 5. 注意事项\n",
    "* 基于公开数据集Composition-1K和Distinctions-646的训练和预测需发邮件向作者申请。\n",
    "* PP-Matting开源了人像预训练模型，可根据具体的人像抠图场景，标注少量的数据进行finetune。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e238357-2f19-48c3-902e-9ba3c93f7818",
   "metadata": {},
   "source": [
    "## 6. 相关论文以及引用信息\n",
    "@article{chen2022pp,\n",
    "  title={PP-Matting: High-Accuracy Natural Image Matting},\n",
    "  author={Chen, Guowei and Liu, Yi and Wang, Jian and Peng, Juncai and Hao, Yuying and Chu, Lutao and Tang, Shiyu and Wu, Zewu and Chen, Zeyu and Yu, Zhiliang and others},\n",
    "  journal={arXiv preprint arXiv:2204.09433},\n",
    "  year={2022}\n",
    "}"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "py35-paddle1.2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}