提交 177d35af 编写于 作者: G gongweibao

modify commands

上级 ee454f93
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
## Objetive ## Objetive
在本文档中,我们设计说明了用户上传、下载、管理自己在PaddlePaddle Cloud上的文件所涉及到的模块和流程 在本文档中,我们设计说明了用户上传、下载、管理自己在PaddlePaddle Cloud上的文件所涉及到的模块和流程
<image src=./src/filemanager.png width=600> <image src=./src/filemanager.png width=8900>
## Module ## Module
### PFS Client ### PFS Client
...@@ -25,7 +25,7 @@ ...@@ -25,7 +25,7 @@
- 在kubernets中运行 - 在kubernets中运行
- [RESTAPI](./RESTAPI.md)接口 - [RESTAPI](./RESTAPI.md)接口
## 流程 ## 解释
### 为什么有chunk的抽象: ### 为什么有chunk的抽象:
用户文件可能是比较大的,上传到Cloud或者下载到本地的时间可能比较长,而且在传输的过程中也可能出现网络不稳定的情况。为了应对以上的问题,我们提出了chunk的概念,一个chunk由所在的文件偏移、数据、数据长度及校验值组成。文件数据内容的上传和下载都是都过chunk的操作来实现的。由于chunk比较小(默认256K),完成一个传输动作的transaction的时间也比较短,不容易出错。 用户文件可能是比较大的,上传到Cloud或者下载到本地的时间可能比较长,而且在传输的过程中也可能出现网络不稳定的情况。为了应对以上的问题,我们提出了chunk的概念,一个chunk由所在的文件偏移、数据、数据长度及校验值组成。文件数据内容的上传和下载都是都过chunk的操作来实现的。由于chunk比较小(默认256K),完成一个传输动作的transaction的时间也比较短,不容易出错。
......
# REST API # REST API Interface
- file - file
``` ```
......
# Name # Name
pfs_cp - copy files and directories cp - copy files and directories
# Synopsis # Synopsis
` pfs_cp ` cp [OPTION]...
<LocalPath> <PFSPath> or <PFSPath> <LocalPath> or <PFSPath> <PFSPath>` <LocalPath> <PFSPath> or <PFSPath> <LocalPath> or <PFSPath> <PFSPath>`
# Description # Description
``` ```
--only-show-errors (boolean)
Only errors and warnings are displayed. All other output is suppressed.
--page-size (integer)
The number of results to return in each response to a list operation. The default value is 1000 (the maximum allowed). Using a lower value may help if an operation times out.
--preserve--links --preserve--links
Reserve links when copy files Reserve links when copy files
...@@ -22,10 +16,10 @@ pfs_cp - copy files and directories ...@@ -22,10 +16,10 @@ pfs_cp - copy files and directories
``` ```
# Examples # Examples
- The following command cp a single file to pfs - The following command copies a single file to pfs
``` ```
pfs_cp ./text1.txt pfs://mydir/text1.txt paddle pfs cp ./text1.txt pfs://mydir/text1.txt
``` ```
Output Output
...@@ -34,10 +28,10 @@ Output ...@@ -34,10 +28,10 @@ Output
upload ./text1.txt to pfs://mydir/text1.txt upload ./text1.txt to pfs://mydir/text1.txt
``` ```
- The following command cp pfs file to a local file - The following command copies pfs file to a local file
``` ```
pfs_cp pfs://mydir/text1.txt ./text1.txt paddle pfs cp pfs://mydir/text1.txt ./text1.txt
``` ```
Output Output
......
# Name # Name
pfs_ls - list directory contents o ls - list directory contents or a file attributes
# Synopsis # Synopsis
` pfs_ls [OPTION]... <PFSPath>` ` ls [OPTION]... <PFSPath>`
# Description # Description
``` ```
-h, --help
Display this help and exit
--version
Output version information and exit
--only-show-errors (boolean)
Only errors and warnings are displayed. All other output is suppressed.
--page-size (integer)
The number of results to return in each response to a list operation. The default value is 1000 (the maximum allowed). Using a lower value may help if an operation times out.
-R, -r, --recursive -R, -r, --recursive
Copy directories recursively Copy directories recursively
--page-size (integer)
The number of results to return in each response to a list operation. The default value is 1000 (the maximum allowed). Using a lower value may help if operation time out.
``` ```
# Examples # Examples
- The following command cp a single file to pfs - The following command lists a single file
``` ```
pfs_cp ./text1.txt pfs://mydir/text1.txt paddle pfs ls pfs://mydir/text1.txt
``` ```
Output Output
``` ```
upload ./text1.txt to pfs://mydir/text1.txt 2017-05-5 17:04:30 text1.txt
``` ```
- The following command cp pfs file to a local file - The following command lists directory contents
``` ```
pfs_cp pfs://mydir/text1.txt ./text1.txt paddle pfs ls pfs://mydir
``` ```
Output Output
``` ```
download pfs://mydir/text1.txt to ./text1.txt 2017-05-5 17:04:30 text1.txt
2017-05-5 17:04:30 text2.txt
...
``` ```
# Name # Name
pdf_mkdir - mkdir directory(ies) mkdir - mkdir directory(ies)
# Synopsis # Synopsis
`pfs_mkdir [OPTION]... PFS_DIRECTORY...` `mkdir [OPTION]... <PFSPath> ...`
# Description # Description
Create the pfs directory(ies), if they do not already exist. Create the pfs directory(ies), if they do not already exist. And create intermediate directories as required
```
-h, --help
display this help and exit
--version
output version information and exit
--only-show-errors (boolean)
Only errors and warnings are displayed. All other output is suppressed.
--page-size (integer)
The number of results to return in each response to a list operation. The default value is 1000 (the maximum allowed). Using a lower value may help if an operation times out.
```
# Examples # Examples
``` ```
pfs_mkdir pfs://mydir1 paddle pfs mkdir pfs://mydir1
``` ```
Output Output
......
...@@ -3,7 +3,8 @@ mv - move (rename) files or directories ...@@ -3,7 +3,8 @@ mv - move (rename) files or directories
# Synopsis # Synopsis
If destination already exist, please rm it first. If destination already exist, please [rm](rm.md) it first.
``` ```
mv [OPTION]... mv [OPTION]...
<LocalPath> <PFSPath> or <PFSPath> <LocalPath> or <PFSPath> <PFSPath> <LocalPath> <PFSPath> or <PFSPath> <LocalPath> or <PFSPath> <PFSPath>
...@@ -11,28 +12,17 @@ mv [OPTION]... ...@@ -11,28 +12,17 @@ mv [OPTION]...
# Description # Description
``` ```
-h, --help
display this help and exit
-v, --version
output version information and exit
--only-show-errors (boolean)
Only errors and warnings are displayed. All other output is suppressed.
--page-size (integer)
The number of results to return in each response to a list operation. The default value is 1000 (the maximum allowed). Using a lower value may help if an operation times out.
``` ```
# Examples # Examples
- The following command move a single file to pfs - The following command move a single file to pfs
``` ```
mv ./text1.txt pfs://mydirectory/test1.txt paddle pfs mv ./text1.txt pfs://mydirectory/test1.txt
``` ```
Output Output
``` ```
move text1.txt pfs://mydirectory/test1.txt move ./text1.txt pfs://mydirectory/test1.txt
``` ```
...@@ -12,20 +12,26 @@ paddle [options] pfs <subcommand> [parameters] ...@@ -12,20 +12,26 @@ paddle [options] pfs <subcommand> [parameters]
``` ```
--profile (string) --profile (string)
Use a specific profile from your credential file. Use a specific profile from your credential file.
--help pfs
Display more information about pfs --help (string)
Display more information about command
--version --version
Output version information and exit Output version information and exit
--debug --debug
Show detailed debugging log Show detailed debugging log
--only-show-errors (boolean)
Only errors and warnings are displayed. All other output is suppressed.
``` ```
## Path Arguments ## Path Arguments
When using a commnd, we need to specify path arguments. There are two path argument type: `localpath` and `pfspath`. When using a command, we need to specify path arguments. There are two path argument type: `localpath` and `pfspath`.
A pfspath begin with `pfs://`, eg: `pfs://mydir/text1.txt`. A `pfspath` begin with `pfs://`, eg: `pfs://mydir/text1.txt`.
## Path Arguments‘s order ## order of Path Arguments
Commonly, there maybe two path arguments. The first is source, and the second is destination. Commonly, if there are two path arguments. The first is the source, and the second is the destination.
## Subcommonds ## Subcommonds
- [rm](rm.md) - [rm](rm.md)
......
...@@ -2,30 +2,25 @@ ...@@ -2,30 +2,25 @@
rm - remove files or directories rm - remove files or directories
# Synopsis # Synopsis
`rm [OPTION]... PFS_DIRECTORY...` ```
rm [OPTION]... <PFSPath>...
```
# Description # Description
Create the directory, if it does not already exits
``` ```
-r, -R, --recursive -r, -R, --recursive
remove directories and their contents recursively remove directories and their contents recursively
-v, --version
output version information and exit
--only-show-errors (boolean)
Only errors and warnings are displayed. All other output is suppressed.
--page-size (integer) --page-size (integer)
The number of results to return in each response to a list operation. The default value is 1000 (the maximum allowed). Using a lower value may help if an operation times out. The number of results to return in each response to a list operation. The default value is 1000 (the maximum allowed). Using a lower value may help if an operation times out
``` ```
# Examples # Examples
- The following command delete a single file - The following command deletes a single files
``` ```
pfs_rm pfs://mydirectory/test1.txt paddle pfs rm pfs://mydirectory/test1.txt
``` ```
Output Output
...@@ -35,10 +30,10 @@ delete pfs://mydirectory/test1.txt ...@@ -35,10 +30,10 @@ delete pfs://mydirectory/test1.txt
``` ```
- The following command delete a directory recursively. - The following command deletes a directory recursively.
``` ```
pfs_rm -r pfs://mydirectory paddle pfs rm -r pfs://mydirectory
``` ```
Output Output
......
# Name # Name
pfs_cp - copy files and directories sync - sync directories. Recursively copies new and updated files from the source directory to the destination
# Synopsis # Synopsis
` pfs_cp ` sync [OPTIONS]...
<LocalPath> <PFSPath> or <PFSPath> <LocalPath> or <PFSPath> <PFSPath>` <LocalPath> <PFSPath> or <PFSPath> <LocalPath> or <PFSPath> <PFSPath>`
# Description # Description
``` ```
-h, --help -l, --links
Display this help and exit copy symlinks as symlinks
--version
Output version information and exit
--only-show-errors (boolean)
Only errors and warnings are displayed. All other output is suppressed.
--page-size (integer)
The number of results to return in each response to a list operation. The default value is 1000 (the maximum allowed). Using a lower value may help if an operation times out.
--preserve--links
Reserve links when copy files
-R, -r, --recursive
Copy directories recursively
``` ```
# Examples # Examples
- The following command cp a single file to pfs - The following command sync locally directory to pfs
``` ```
pfs_cp ./text1.txt pfs://mydir/text1.txt sync ./dir1 pfs://mydir1
``` ```
Output Output
``` ```
upload ./text1.txt to pfs://mydir/text1.txt upload ./dir1/text1.txt to pfs://mydir1/text2.txt
upload ./dir1/text2.txt to pfs://mydir1/text2.txt
...
``` ```
- The following command cp pfs file to a local file - The following command sync pfs directory to local
``` ```
pfs_cp pfs://mydir/text1.txt ./text1.txt sync pfs://mydir1 .
``` ```
Output Output
``` ```
download pfs://mydir/text1.txt to ./text1.txt download pfs://mydir1/text1.txt to ./text1.txt
download pfs://mydir1/text2.txt to ./text2.txt
...
``` ```
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册