introduction_cn.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "1、模型简介\n",
    "\n",
    "Mask Image Modeling (MIM) 方法，在 NLP 领域 (例如BERT) 得到了广泛的应用。随着 ViT 的提出和发展，人们也尝试将 MIM 应用到视觉领域并取得了一定进展。在此之前，视觉自监督算法主要沿着 contrastive learning 的思路去设计，而 MIM 打开了新的大门。“Context Autoencoder for Self-Supervised Representation Learning”，即VEMER-CAE,提出一种新的 MIM 方法 CAE。VIMER-CAE基于自监督图像掩码恢复原理，创新性地提出“图像表征和掩码预测独立学习”的预训练框架，通过编码模块对输入的图像块进行特征表达，并利用隐式上下文回归和解码模块对输入图像的掩码块进行特征表达恢复，在恢复建模问题上提高了预训练模型的视觉表征能力。基于自监督方法训练的预训练模型在下游各类图像任务上取得了明显的效果提升，其中在目标检测、实力分割、语义分割等任务的指标上达到SOT R效果。\n",
    "\n",
    "<div align=center><img src=\"https://github.com/PaddlePaddle/VIMER/blob/main/CAE/figs/CAE2.png\" width=\"60%\"></div>\n",
    "\n",
    "\n",
    "2、模型效果\n",
    "\n",
    "1）分类场景ImageNet-1K数据集上的结果 \n",
    "\n",
    "|   model  | pretrain | Linear Prob(Top-1) | Attentive Prob(Top-1) | Finetune(Top-1) \n",
    "|:--------:|:--------:|:--------:|:--------:| :--------:| \n",
    "| [Vit-Base](https://vimer.bj.bcebos.com/CAE/pt_ep800_fp32_checkpoint-799.pd) |   800e   |   69.3%   |   76.7%  |   83.7%   | \n",
    "| Vit-Large |  1600e   |   78.1%   |   81.2%  |   86.3%   |\n",
    "\n",
    "\n",
    "2）分割场景ADE20K数据集上的结果\n",
    "\n",
    "| Backbone | Method | Crop Size | Lr Schd | mIoU | #params | FLOPs | \n",
    "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | \n",
    "| Vit-Base-800e | UperNet | 512x512 | 160K | 49.69 | 81M | 1038G |\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "## Citing Context Autoencoder for Self-Supervised Representation Learning\n",
    "\n",
    "@article{chen2022context,\n",
    "  title={Context autoencoder for self-supervised representation learning},\n",
    "  author={Chen, Xiaokang and Ding, Mingyu and Wang, Xiaodi and Xin, Ying and Mo, Shentong and Wang, Yunhao and Han, Shumin and Luo, Ping and Zeng, Gang and Wang, Jingdong},\n",
    "  journal={arXiv preprint arXiv:2202.03026},\n",
    "  year={2022}\n",
    "}\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 查看工作区文件, 该目录下的变更将会持久保存. 请及时清理不必要的文件, 避免加载过慢.\n",
    "# View personal work directory. \n",
    "# All changes under this directory will be kept even after reset. \n",
    "# Please clean unnecessary files in time to speed up environment loading. \n",
    "!ls /home/aistudio/work"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 如果需要进行持久化安装, 需要使用持久化路径, 如下方代码示例:\n",
    "# If a persistence installation is required, \n",
    "# you need to use the persistence path as the following: \n",
    "!mkdir /home/aistudio/external-libraries\n",
    "!pip install beautifulsoup4 -t /home/aistudio/external-libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 同时添加如下代码, 这样每次环境(kernel)启动的时候只要运行下方代码即可: \n",
    "# Also add the following code, \n",
    "# so that every time the environment (kernel) starts, \n",
    "# just run the following code: \n",
    "import sys \n",
    "sys.path.append('/home/aistudio/external-libraries')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请点击[此处](https://ai.baidu.com/docs#/AIStudio_Project_Notebook/a38e5576)查看本环境基本用法.  <br>\n",
    "Please click [here ](https://ai.baidu.com/docs#/AIStudio_Project_Notebook/a38e5576) for more detailed instructions. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "py35-paddle1.2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}