diff --git a/doc/fluid/dev/index_cn.rst b/doc/fluid/dev/index_cn.rst index f627437f354a12c79cad25c959409db29ecbd874..b123b756e2251c38f319e1aefa2cb04fd7a36b03 100644 --- a/doc/fluid/dev/index_cn.rst +++ b/doc/fluid/dev/index_cn.rst @@ -9,5 +9,5 @@ use_eigen_cn.md name_convention.md support_new_device.md - releasing_process.md + releasing_process_cn.md op_markdown_format.md diff --git a/doc/fluid/dev/index_en.rst b/doc/fluid/dev/index_en.rst index 0b65fed67ad45eb399b624184485a99a082d79e9..98988fc22dcedecdbcd67fb3bf761377bf046337 100644 --- a/doc/fluid/dev/index_en.rst +++ b/doc/fluid/dev/index_en.rst @@ -9,5 +9,5 @@ Development use_eigen_en.md name_convention.md support_new_device.md - releasing_process.md + releasing_process_en.md op_markdown_format.md diff --git a/doc/fluid/dev/releasing_process.md b/doc/fluid/dev/releasing_process_cn.md similarity index 74% rename from doc/fluid/dev/releasing_process.md rename to doc/fluid/dev/releasing_process_cn.md index c5943ccd81c2ae2aaacd2676da12509db889f54a..4c6728fba7150b0f1e180e57590f18a5b677c70d 100644 --- a/doc/fluid/dev/releasing_process.md +++ b/doc/fluid/dev/releasing_process_cn.md @@ -10,19 +10,10 @@ PaddlePaddleæ¯æ¬¡å‘新的版本,éµå¾ªä»¥ä¸‹æµç¨‹: * 使用Regression Test List作为检查列表,测试本次releaseçš„æ£ç¡®æ€§ã€‚ * 如果失败,记录下所有失败的例å,在这个`release/版本å·`分支ä¸ï¼Œä¿®å¤æ‰€æœ‰bugåŽï¼ŒPatchå·åŠ ä¸€ï¼Œåˆ°ç¬¬äºŒæ¥ * 修改`python/setup.py.in`ä¸çš„版本信æ¯,并将`istaged`å—段设为`True`。 - * 编译这个版本的python wheel包,并å‘布到pypi。 - * 由于pypi.python.orgç›®å‰éµå¾ª[ä¸¥æ ¼çš„å‘½å规范PEP 513](https://www.python.org/dev/peps/pep-0513),在使用twineä¸Šä¼ ä¹‹å‰ï¼Œéœ€è¦é‡å‘½åwheel包ä¸platform相关的åŽç¼€ï¼Œæ¯”如将`linux_x86_64`修改æˆ`manylinux1_x86_64`。 - * pypi上的packageå称为paddlepaddleå’Œpaddlepaddle_gpu,如果è¦ä¸Šä¼ GPU版本的包,需è¦ä¿®æ”¹build/python/setup.pyä¸ï¼Œname: "paddlepaddle_gpu"并é‡æ–°æ‰“包wheel包:`python setup.py bdist_wheel`。 - * ä¸Šä¼ æ–¹æ³•ï¼š - ``` - cd build/python - pip install twine - twine upload dist/[package to upload] - ``` - * 编译这个版本的Dockerå‘行镜åƒï¼Œå‘布到dockerhub。如果失败,修å¤Docker编译镜åƒé—®é¢˜ï¼ŒPatchå·åŠ ä¸€ï¼Œè¿”å›žç¬¬äºŒæ¥ -1. 第三æ¥å®ŒæˆåŽï¼Œå°†`release/版本å·`分支åˆå…¥masteråˆ†æ”¯ï¼Œå¹¶åˆ é™¤`release/版本å·`分支。将master分支的åˆå…¥commit打上tag,tag为`版本å·`。åŒæ—¶å†å°†`master`分支åˆå…¥`develop`分支。最åŽåˆ 除`release/版本å·`分支。 -1. ååŒå®ŒæˆRelease Note的书写 - + * 将这个版本的python wheel包å‘布到pypi。 + * æ›´æ–°Dockeré•œåƒï¼ˆå‚考åŽé¢çš„æ“作细节)。 +1. 第三æ¥å®ŒæˆåŽï¼Œå°†`release/版本å·`分支åˆå…¥master分支,将master分支的åˆå…¥commit打上tag,tag为`版本å·`。åŒæ—¶å†å°†`master`分支åˆå…¥`develop`分支。 +1. ååŒå®ŒæˆRelease Note的书写。 需è¦æ³¨æ„的是: @@ -31,13 +22,18 @@ PaddlePaddleæ¯æ¬¡å‘新的版本,éµå¾ªä»¥ä¸‹æµç¨‹: ## å‘布wheel包到pypi -使用[PaddlePaddle CI](https://paddleci.ngrok.io/project.html?projectId=Manylinux1&tab=projectOverview) +1. 使用[PaddlePaddle CI](https://paddleci.ngrok.io/project.html?projectId=Manylinux1&tab=projectOverview) 完æˆè‡ªåŠ¨åŒ–二进制编译,å‚考下图,选择需è¦å‘布的版本(通常包å«ä¸€ä¸ªCPU版本和一个GPU版本),点击"run"å³ä¾§çš„"..."按钮,å¯ä»¥ -弹出下é¢çš„选择框,在第二个tab (Changes)里选择需è¦å‘布的分支,这里选择0.11.0,然åŽç‚¹å‡»"Run Build"按钮。ç‰å¾…编译完æˆåŽ -å¯ä»¥åœ¨æ¤é¡µé¢çš„"Artifacts"下拉框ä¸æ‰¾åˆ°ç”Ÿæˆçš„3个二进制文件,分别对应CAPI,`cp27m`å’Œ`cp27mu`的版本。然åŽæŒ‰ç…§ä¸Šè¿°çš„方法 -使用`twine`å·¥å…·ä¸Šä¼ å³å¯ã€‚ - -<img src="https://raw.githubusercontent.com/PaddlePaddle/Paddle/develop/doc/fluid/images/ci_build_whl.png"> +弹出下é¢çš„选择框,在第二个tab (Changes)里选择需è¦å‘布的分支,这里选择0.11.0,然åŽç‚¹å‡»"Run Build"按钮。 + <img src="https://raw.githubusercontent.com/PaddlePaddle/Paddle/develop/doc/fluid/images/ci_build_whl.png"> +1. ç‰å¾…编译完æˆåŽå¯ä»¥åœ¨æ¤é¡µé¢çš„"Artifacts"下拉框ä¸æ‰¾åˆ°ç”Ÿæˆçš„3个二进制文件,分别对应CAPI,`cp27m`å’Œ`cp27mu`的版本。 +1. 由于pypi.python.orgç›®å‰éµå¾ª[ä¸¥æ ¼çš„å‘½å规范PEP 513](https://www.python.org/dev/peps/pep-0513),在使用twineä¸Šä¼ ä¹‹å‰ï¼Œéœ€è¦é‡å‘½åwheel包ä¸platform相关的åŽç¼€ï¼Œæ¯”如将`linux_x86_64`修改æˆ`manylinux1_x86_64`。 +1. ä¸Šä¼ ï¼š +``` +cd build/python +pip install twine +twine upload dist/[package to upload] +``` * 注:CI环境使用 https://github.com/PaddlePaddle/buildtools 这里的DockerImage作为编译环境以支æŒæ›´å¤šçš„Linux å‘型版,如果需è¦æ‰‹åŠ¨ç¼–译,也å¯ä»¥ä½¿ç”¨è¿™äº›é•œåƒã€‚这些镜åƒä¹Ÿå¯ä»¥ä»Ž https://hub.docker.com/r/paddlepaddle/paddle_manylinux_devel/tags/ 下载得到。 @@ -48,10 +44,20 @@ PaddlePaddleæ¯æ¬¡å‘新的版本,éµå¾ªä»¥ä¸‹æµç¨‹: 上述PaddlePaddle CI编译wheel完æˆåŽä¼šè‡ªåŠ¨å°†Dockeré•œåƒpush到DockerHub,所以,å‘布Dockeré•œåƒåªéœ€è¦å¯¹è‡ªåŠ¨pushçš„é•œåƒæ‰“上 版本å·å¯¹åº”çš„tagå³å¯ï¼š -1. 进入 https://hub.docker.com/r/paddlepaddle/paddle/tags/ 查看latest tag的更新时间是å¦åœ¨ä¸Šè¿°ç¼–译wheel包完æˆåŽæ˜¯å¦æœ€æ–°ã€‚ -1. 执行 `docker pull paddlepaddle/paddle:[latest tag]`,latest tagå¯ä»¥æ˜¯latest或latest-gpuç‰ã€‚ -1. 执行 `docker tag paddlepaddle/paddle:[latest tag] paddlepaddle/paddle:[version]` -1. 执行 `docker push paddlepaddle/paddle:[version]` +``` +docker pull [é•œåƒ]:latest +docker tag [é•œåƒ]:latest [é•œåƒ]:[version] +docker push [é•œåƒ]:[version] +``` + +需è¦æ›´æ–°çš„é•œåƒtag包括: + +* `[version]`: CPU版本 +* `[version]-openblas`: openblas版本 +* `[version]-gpu`: GPU版本(CUDA 8.0 cudnn 5) +* `[version]-gpu-[cudaver]-[cudnnver]`: ä¸åŒcuda, cudnnç‰ˆæœ¬çš„é•œåƒ + +之åŽå¯è¿›å…¥ https://hub.docker.com/r/paddlepaddle/paddle/tags/ 查看是å¦å‘布æˆåŠŸã€‚ ## PaddlePaddle 分支规范 @@ -76,7 +82,7 @@ PaddlePaddleå¼€å‘过程使用[git-flow](http://nvie.com/posts/a-successful-git- ### PaddlePaddle Bookä¸æ‰€æœ‰ç« 节 -PaddlePaddleæ¯æ¬¡å‘版本首先è¦ä¿è¯PaddlePaddle Bookä¸æ‰€æœ‰ç« 节功能的æ£ç¡®æ€§ã€‚功能的æ£ç¡®æ€§åŒ…括验è¯PaddlePaddleç›®å‰çš„`paddle_trainer`è®ç»ƒå’Œçº¯ä½¿ç”¨`Python`è®ç»ƒæ¨¡åž‹æ£ç¡®æ€§ã€‚ +PaddlePaddleæ¯æ¬¡å‘版本首先è¦ä¿è¯PaddlePaddle Bookä¸æ‰€æœ‰ç« 节功能的æ£ç¡®æ€§ã€‚功能的æ£ç¡®æ€§åŒ…括验è¯PaddlePaddleç›®å‰çš„`paddle_trainer`è®ç»ƒå’Œçº¯ä½¿ç”¨`Python`è®ç»ƒï¼ˆV2å’ŒFluid)模型æ£ç¡®æ€§ã€‚ <table> <thead> diff --git a/doc/fluid/dev/releasing_process_en.md b/doc/fluid/dev/releasing_process_en.md new file mode 100644 index 0000000000000000000000000000000000000000..f989b964d6d1a329bbe31adc7ec10db017acaefa --- /dev/null +++ b/doc/fluid/dev/releasing_process_en.md @@ -0,0 +1,210 @@ +# PaddlePaddle Releasing Process + +PaddlePaddle manages its branches using "git-flow branching model", and [Semantic Versioning](http://semver.org/) as it's version number semantics. + +Each time we release a new PaddlePaddle version, we should follow the below steps: + +1. Fork a new branch from `develop` named `release/[version]`, e.g. `release/0.10.0`. +1. Push a new tag on the release branch, the tag name should be like `[version]rc.patch`. The + first tag should be `0.10.0rc1`, and the second should be `0.10.0.rc2` and so on. +1. After that, we should do: + * Run all regression test on the Regression Test List (see PaddlePaddle TeamCity CI), to confirm + that this release has no major bugs. + * If regression test fails, we must fix those bugs and create a new `release/[version]` + branch from previous release branch. + * Modify `python/setup.py.in`, change the version number and change `ISTAGED` to `True`. + * Publish PaddlePaddle release wheel packages to pypi (see below instructions for detail). + * Update the Docker images (see below instructions for detail). +1. After above step, merge `release/[version]` branch to master and push a tag on the master commit, + then merge `master` to `develop`. +1. Update the Release Note. + +***NOTE:*** + +* Do ***NOT*** merge commits from develop branch to release branches to keep the release branch contain + features only for current release, so that we can test on that version. +* If we want to fix bugs on release branches, we must merge the fix to master, develop and release branch. + +## Publish Wheel Packages to pypi + +1. Use our [CI tool](https://paddleci.ngrok.io/project.html?projectId=Manylinux1&tab=projectOverview) + to build all wheel packages needed to publish. As shown in the following picture, choose a build + version, click "..." button on the right side of "Run" button, and switch to the second tab in the +pop-up box, choose the current release branch and click "Run Build" button. You may repeat this + step to start different versions of builds. + <img src="https://raw.githubusercontent.com/PaddlePaddle/Paddle/develop/doc/fluid/images/ci_build_whl.png"> +1. After the build succeeds, download the outputs under "Artifacts" including capi, `cp27m` and `cp27mu`. +1. Since pypi.python.org follows [PEP 513](https://www.python.org/dev/peps/pep-0513), before we + upload the package using `twine`, we need to rename the package from `linux_x86_64` to + `manylinux1_x86_64`. +1. Start the upload: + ``` + cd build/python + pip install twine + twine upload dist/[package to upload] + ``` + +* NOTE: We use a special Docker image to build our releases to support more Linux distributions, you can + download it from https://hub.docker.com/r/paddlepaddle/paddle_manylinux_devel/tags/, or build it using + scripts under `tools/manylinux1`. +* pypi does not allow overwrite the already uploaded version of wheel package, even if you delete the + old version. you must change the version number before upload a new one. + +## Publish Docker Images + +Our CI tool will push latest images to DockerHub, so we only need to push a version tag like: + +``` +docker pull [image]:latest +docker tag [image]:latest [image]:[version] +docker push [image]:[version] +``` + +Tags that need to be updated are: +* `[version]`: CPU only version image +* `[version]-openblas`: openblas version image +* `[version]-gpu`: GPU version(using CUDA 8.0 cudnn 5) +* `[version]-gpu-[cudaver]-[cudnnver]`: tag for different cuda, cudnn versions + +You can then checkout the latest pushed tags at https://hub.docker.com/r/paddlepaddle/paddle/tags/. + +## Branching Model + +We use [git-flow](http://nvie.com/posts/a-successful-git-branching-model/) as our branching model, +with some modifications: + +* `master` branch is the stable branch. Each version on the master branch is tested and guaranteed. +* `develop` branch is for development. Each commit on develop branch has passed CI unit test, but no + regression tests are run. +* `release/[version]` branch is used to publish each release. Latest release version branches have + bugfix only for that version, but no feature updates. +* Developer forks are not required to follow + [git-flow](http://nvie.com/posts/a-successful-git-branching-model/) + branching model, all forks is like a feature branch. + * Advise: developer fork's develop branch is used to sync up with main repo's develop branch. + * Advise: developer use it's fork's develop branch to for new branch to start developing. + * Use that branch on developer's fork to create pull requests and start reviews. + * developer can push new commits to that branch when the pull request is open. +* Bug fixes are also started from developers forked repo. And, bug fixes branch can merge to + `master`, `develop` and `releases`. + +## PaddlePaddle Regression Test List + +### All Chapters of PaddlePaddle Book + +We need to guarantee that all the chapters of PaddlePaddle Book can run correctly. Including +V1 (`paddle_trainer` training) and V2 training and Fluid training. + +<table> +<thead> +<tr> +<th></th> +<th>Linear Regression</th> +<th>Recognize Digits</th> +<th>Image Classification</th> +<th>Word2Vec</th> +<th>Personalized Recommendation</th> +<th>Sentiment Analysis</th> +<th>Semantic Role Labeling</th> +<th>Machine Translation</th> +</tr> +</thead> + +<tbody> +<tr> +<td>API.V2 + Docker + GPU </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +</tr> + +<tr> +<td> API.V2 + Docker + CPU </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +</tr> + +<tr> +<td>`paddle_trainer` + Docker + GPU </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +</tr> + +<tr> +<td>`paddle_trainer` + Docker + CPU </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +</tr> + +<tr> +<td> API.V2 + Ubuntu + GPU</td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +</tr> + +<tr> +<td>API.V2 + Ubuntu + CPU </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +</tr> + +<tr> +<td> `paddle_trainer` + Ubuntu + GPU</td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +</tr> + +<tr> +<td> `paddle_trainer` + Ubuntu + CPU</td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +<td> </td> +</tr> +</tbody> +</table>