Skip to content

P
PaddleSeg

PaddlePaddle / PaddleSeg

通知 286
Star 8
Fork 1
P
PaddleSeg
提交 a1fbffca 编写于 作者: W wuzewu
浏览文件
操作

Update configs

上级 de57839b
隐藏空白更改
内联 并排
Showing with 52 addition and 111 deletion
configs/cityscape.yaml
浏览文件 @ a1fbffca
......@@ -10,18 +10,6 @@ AUG:
MIN_SCALE_FACTOR: 0.5 # for stepscaling
SCALE_STEP_SIZE: 0.25 # for stepscaling
MIRROR: True
RICH_CROP:
ENABLE: False
ASPECT_RATIO: 0.33
BLUR: True
BLUR_RATIO: 0.1
FLIP: True
FLIP_RATIO: 0.2
MAX_ROTATION: 15
MIN_AREA_RATIO: 0.5
BRIGHTNESS_JITTER_RATIO: 0.5
CONTRAST_JITTER_RATIO: 0.5
SATURATION_JITTER_RATIO: 0.5
BATCH_SIZE: 4
DATASET:
DATA_DIR: "./dataset/cityscapes/"
......@@ -44,8 +32,7 @@ TEST:
TEST_MODEL: "snapshots/cityscape_v5/final/"
TRAIN:
MODEL_SAVE_DIR: "snapshots/cityscape_v7/"
PRETRAINED_MODEL: u"pretrain/deeplabv3plus_gn_init"
RESUME: False
PRETRAINED_MODEL_DIR: "pretrain/deeplabv3plus_gn_init"
SNAPSHOT_EPOCH: 10
SOLVER:
LR: 0.001
......
configs/unet_pet.yaml
浏览文件 @ a1fbffca
......@@ -12,18 +12,6 @@ AUG:
MIN_SCALE_FACTOR: 0.75 # for stepscaling
SCALE_STEP_SIZE: 0.25 # for stepscaling
MIRROR: True
RICH_CROP:
ENABLE: False
ASPECT_RATIO: 0.33
BLUR: True
BLUR_RATIO: 0.1
FLIP: True
FLIP_RATIO: 0.2
MAX_ROTATION: 15
MIN_AREA_RATIO: 0.5
BRIGHTNESS_JITTER_RATIO: 0.5
CONTRAST_JITTER_RATIO: 0.5
SATURATION_JITTER_RATIO: 0.5
BATCH_SIZE: 4
DATASET:
DATA_DIR: "./dataset/mini_pet/"
......@@ -45,8 +33,7 @@ TEST:
TEST_MODEL: "./test/saved_model/unet_pet/final/"
TRAIN:
MODEL_SAVE_DIR: "./test/saved_models/unet_pet/"
PRETRAINED_MODEL: "./test/models/unet_coco/"
RESUME: False
PRETRAINED_MODEL_DIR: "./test/models/unet_coco/"
SNAPSHOT_EPOCH: 10
SOLVER:
NUM_EPOCHS: 500
......
docs/configs/train_group.md
浏览文件 @ a1fbffca
......@@ -11,7 +11,7 @@ TRAIN Group存放所有和训练相关的配置
<br/>
<br/>
## `PRETRAINED_MODEL`
## `PRETRAINED_MODEL_DIR`
预训练模型路径
## 默认值
......@@ -28,19 +28,15 @@ TRAIN Group存放所有和训练相关的配置
<br/>
<br/>
## `RESUME`
是否从预训练模型中恢复参数并继续训练
## `RESUME_MODEL_DIR`
从指定路径中恢复参数并继续训练
## 默认值
False
无
## 注意事项
* 当该字段被置为True且`PRETRAINED_MODEL`不存在时，该选项不生效
* 当该字段被置为True且`PRETRAINED_MODEL`存在时，PaddleSeg会恢复到上一次训练的最近一个epoch，并且恢复训练过程中的临时变量（如已经衰减过的学习率，Optimizer的动量数据等）
* 当该字段被置为True且`PRETRAINED_MODEL`存在时，`PRETRAINED_MODEL`路径的最后一个目录必须为int数值或者字符串final，PaddleSeg会将int数值作为当前起始EPOCH继续训练，若目录为final，则不会继续训练。若目录不满足上述条件，PaddleSeg会抛出错误。
* 当`RESUME_MODEL_DIR`存在时，PaddleSeg会恢复到上一次训练的最近一个epoch，并且恢复训练过程中的临时变量（如已经衰减过的学习率，Optimizer的动量数据等），`PRETRAINED_MODEL`路径的最后一个目录必须为int数值或者字符串final，PaddleSeg会将int数值作为当前起始EPOCH继续训练，若目录为final，则不会继续训练。若目录不满足上述条件，PaddleSeg会抛出错误。
<br/>
<br/>
......@@ -57,4 +53,4 @@ False
* 仅在GPU多卡训练时该开关有效（Windows不支持多卡训练，因此无需打开该开关）
* GPU多卡训练时，建议开启该开关，可以提升模型的训练效果
\ No newline at end of file
* GPU多卡训练时，建议开启该开关，可以提升模型的训练效果
docs/data_aug.md
浏览文件 @ a1fbffca
......@@ -35,9 +35,6 @@ rich crop是指对图像进行多种变换，保证在训练过程中数据的
- blur
图像加模糊，使用开关`AUG.RICH_CROP.BLUR`，为False时该项功能关闭。`AUG.RICH_CROP.BLUR_RATIO`控制加入模糊的概率。
- flip
图像上下翻转，使用开关`AUG.RICH_CROP.FLIP`，为False时该项功能关闭。`AUG.RICH_CROP.FLIP_RATIO`控制上下翻转的概率。
- rotation
图像旋转，`AUG.RICH_CROP.MAX_ROTATION`控制最大旋转角度。旋转产生的多余的区域的填充值为均值。
......
docs/usage.md
浏览文件 @ a1fbffca
......@@ -76,7 +76,7 @@ python pdseg/train.py --use_gpu \
--tb_log_dir train_log \
--cfg configs/unet_pet.yaml \
BATCH_SIZE 4 \
TRAIN.PRETRAINED_MODEL pretrained_model/unet_bn_coco \
TRAIN.PRETRAINED_MODEL_DIR pretrained_model/unet_bn_coco \
TRAIN.SYNC_BATCH_NORM True \
SOLVER.LR 5e-5
```
......
pdseg/reader.py
浏览文件 @ a1fbffca
......@@ -261,17 +261,18 @@ class SegDataset(object):
SATURATION_JITTER_RATIO,
contrast_jitter_ratio=cfg.AUG.RICH_CROP.
CONTRAST_JITTER_RATIO)
if cfg.AUG.RICH_CROP.FLIP:
if cfg.AUG.RICH_CROP.FLIP_RATIO <= 0:
n = 0
elif cfg.AUG.RICH_CROP.FLIP_RATIO >= 1:
n = 1
else:
n = int(1.0 / cfg.AUG.RICH_CROP.FLIP_RATIO)
if n > 0:
if np.random.randint(0, n) == 0:
img = img[::-1, :, :]
grt = grt[::-1, :]
if cfg.AUG.FLIP:
if cfg.AUG.FLIP_RATIO <= 0:
n = 0
elif cfg.AUG.FLIP_RATIO >= 1:
n = 1
else:
n = int(1.0 / cfg.AUG.FLIP_RATIO)
if n > 0:
if np.random.randint(0, n) == 0:
img = img[::-1, :, :]
grt = grt[::-1, :]
if cfg.AUG.MIRROR:
if np.random.randint(0, 2) == 1:
......
pdseg/train.py
浏览文件 @ a1fbffca
......@@ -152,15 +152,15 @@ def load_checkpoint(exe, program):
Load checkpoiont from pretrained model directory for resume training
"""
print('Resume model training from:', cfg.TRAIN.PRETRAINED_MODEL)
if not os.path.exists(cfg.TRAIN.PRETRAINED_MODEL):
print('Resume model training from:', cfg.TRAIN.RESUME_MODEL_DIR)
if not os.path.exists(cfg.TRAIN.RESUME_MODEL_DIR):
raise ValueError("TRAIN.PRETRAIN_MODEL {} not exist!".format(
cfg.TRAIN.PRETRAINED_MODEL))
cfg.TRAIN.RESUME_MODEL_DIR))
fluid.io.load_persistables(
exe, cfg.TRAIN.PRETRAINED_MODEL, main_program=program)
exe, cfg.TRAIN.RESUME_MODEL_DIR, main_program=program)
model_path = cfg.TRAIN.PRETRAINED_MODEL
model_path = cfg.TRAIN.RESUME_MODEL_DIR
# Check is path ended by path spearator
if model_path[-1] == os.sep:
model_path = model_path[0:-1]
......@@ -255,11 +255,11 @@ def train(cfg):
# Resume training
begin_epoch = cfg.SOLVER.BEGIN_EPOCH
if cfg.TRAIN.RESUME:
if cfg.TRAIN.RESUME_MODEL_DIR:
begin_epoch = load_checkpoint(exe, train_prog)
# Load pretrained model
elif os.path.exists(cfg.TRAIN.PRETRAINED_MODEL):
print('Pretrained model dir:', cfg.TRAIN.PRETRAINED_MODEL)
elif os.path.exists(cfg.TRAIN.PRETRAINED_MODEL_DIR):
print('Pretrained model dir:', cfg.TRAIN.PRETRAINED_MODEL_DIR)
load_vars = []
load_fail_vars = []
......@@ -268,10 +268,10 @@ def train(cfg):
Check whehter persitable variable shape is match with current network
"""
var_exist = os.path.exists(
os.path.join(cfg.TRAIN.PRETRAINED_MODEL, var.name))
os.path.join(cfg.TRAIN.PRETRAINED_MODEL_DIR, var.name))
if var_exist:
var_shape = parse_shape_from_file(
os.path.join(cfg.TRAIN.PRETRAINED_MODEL, var.name))
os.path.join(cfg.TRAIN.PRETRAINED_MODEL_DIR, var.name))
return var_shape == shape
return False
......@@ -285,10 +285,10 @@ def train(cfg):
load_fail_vars.append(x)
if cfg.MODEL.FP16:
# If open FP16 training mode, load FP16 var separate
load_fp16_vars(exe, cfg.TRAIN.PRETRAINED_MODEL, train_prog)
load_fp16_vars(exe, cfg.TRAIN.PRETRAINED_MODEL_DIR, train_prog)
else:
fluid.io.load_vars(
exe, dirname=cfg.TRAIN.PRETRAINED_MODEL, vars=load_vars)
exe, dirname=cfg.TRAIN.PRETRAINED_MODEL_DIR, vars=load_vars)
for var in load_vars:
print("Parameter[{}] loaded sucessfully!".format(var.name))
for var in load_fail_vars:
......@@ -299,7 +299,7 @@ def train(cfg):
len(load_vars) + len(load_fail_vars)))
else:
print('Pretrained model dir {} not exists, training from scratch...'.
format(cfg.TRAIN.PRETRAINED_MODEL))
format(cfg.TRAIN.PRETRAINED_MODEL_DIR))
fetch_list = [avg_loss.name, lr.name]
if args.debug:
......
pdseg/utils/config.py
浏览文件 @ a1fbffca
......@@ -69,6 +69,10 @@ cfg.DATASET.IGNORE_INDEX = 255
########################### 数据增强配置 ######################################
# 图像镜像左右翻转
cfg.AUG.MIRROR = True
# 图像上下翻转开关，True/False
cfg.AUG.FLIP = False
# 图像启动上下翻转的概率，0-1
cfg.AUG.FLIP_RATIO = 0.5
# 图像resize的固定尺寸（宽，高），非负
cfg.AUG.FIX_RESIZE_SIZE = tuple()
# 图像resize的方式有三种：
......@@ -107,18 +111,14 @@ cfg.AUG.RICH_CROP.CONTRAST_JITTER_RATIO = 0.5
cfg.AUG.RICH_CROP.BLUR = False
# 图像启动模糊百分比，0-1
cfg.AUG.RICH_CROP.BLUR_RATIO = 0.1
# 图像上下翻转开关，True/False
cfg.AUG.RICH_CROP.FLIP = False
# 图像启动上下翻转的概率，0-1
cfg.AUG.RICH_CROP.FLIP_RATIO = 0.2
########################### 训练配置 ##########################################
# 模型保存路径
cfg.TRAIN.MODEL_SAVE_DIR = ''
# 预训练模型路径
cfg.TRAIN.PRETRAINED_MODEL = ''
cfg.TRAIN.PRETRAINED_MODEL_DIR = ''
# 是否resume，继续训练
cfg.TRAIN.RESUME = False
cfg.TRAIN.RESUME_MODEL_DIR = ''
# 是否使用多卡间同步BatchNorm均值和方差
cfg.TRAIN.SYNC_BATCH_NORM = False
# 模型参数保存的epoch间隔数，可用来继续训练中断的模型
......
test/configs/deeplabv3p_xception65_cityscapes.yaml
浏览文件 @ a1fbffca
......@@ -10,18 +10,6 @@ AUG:
MIN_SCALE_FACTOR: 0.5 # for stepscaling
SCALE_STEP_SIZE: 0.25 # for stepscaling
MIRROR: True
RICH_CROP:
ENABLE: False
ASPECT_RATIO: 0.33
BLUR: True
BLUR_RATIO: 0.1
FLIP: True
FLIP_RATIO: 0.2
MAX_ROTATION: 15
MIN_AREA_RATIO: 0.5
BRIGHTNESS_JITTER_RATIO: 0.5
CONTRAST_JITTER_RATIO: 0.5
SATURATION_JITTER_RATIO: 0.5
BATCH_SIZE: 4
DATASET:
DATA_DIR: "./dataset/cityscapes/"
......@@ -46,8 +34,7 @@ TEST:
TEST_MODEL: "snapshots/cityscape_v5/final/"
TRAIN:
MODEL_SAVE_DIR: "snapshots/cityscape_v5/"
PRETRAINED_MODEL: "pretrain/deeplabv3plus_gn_init"
RESUME: False
PRETRAINED_MODEL_DIR: "pretrain/deeplabv3plus_gn_init"
SNAPSHOT_EPOCH: 10
SOLVER:
LR: 0.001
......
test/configs/unet_pet.yaml
浏览文件 @ a1fbffca
......@@ -12,18 +12,6 @@ AUG:
MIN_SCALE_FACTOR: 0.75 # for stepscaling
SCALE_STEP_SIZE: 0.25 # for stepscaling
MIRROR: True
RICH_CROP:
ENABLE: False
ASPECT_RATIO: 0.33
BLUR: True
BLUR_RATIO: 0.1
FLIP: True
FLIP_RATIO: 0.2
MAX_ROTATION: 15
MIN_AREA_RATIO: 0.5
BRIGHTNESS_JITTER_RATIO: 0.5
CONTRAST_JITTER_RATIO: 0.5
SATURATION_JITTER_RATIO: 0.5
BATCH_SIZE: 6
DATASET:
DATA_DIR: "./dataset/pet/"
......@@ -45,8 +33,7 @@ TEST:
TEST_MODEL: "./test/saved_model/unet_pet/final/"
TRAIN:
MODEL_SAVE_DIR: "./test/saved_models/unet_pet/"
PRETRAINED_MODEL: "./test/models/unet_coco/"
RESUME: False
PRETRAINED_MODEL_DIR: "./test/models/unet_coco/"
SNAPSHOT_EPOCH: 10
SOLVER:
NUM_EPOCHS: 500
......
test/local_test_cityscapes.py
浏览文件 @ a1fbffca
......@@ -45,7 +45,8 @@ if __name__ == "__main__":
saved_model = os.path.join(LOCAL_PATH, "saved_model", model_name)
parser = argparse.ArgumentParser(description="PaddleSeg loacl test")
parser.add_argument("--devices",
parser.add_argument(
"--devices",
dest="devices",
help="GPU id of running. if more than one, use spacing to separate.",
nargs="+",
......@@ -75,7 +76,7 @@ if __name__ == "__main__":
train(
flags=["--cfg", cfg, "--use_gpu", "--log_steps", "10"],
options=[
"SOLVER.NUM_EPOCHS", "1", "TRAIN.PRETRAINED_MODEL", test_model,
"SOLVER.NUM_EPOCHS", "1", "TRAIN.PRETRAINED_MODEL_DIR", test_model,
"TRAIN.MODEL_SAVE_DIR", saved_model
],
devices=devices)
test/local_test_pet.py
浏览文件 @ a1fbffca
......@@ -46,7 +46,8 @@ if __name__ == "__main__":
saved_model = os.path.join(LOCAL_PATH, "saved_model", model_name)
parser = argparse.ArgumentParser(description="PaddleSeg loacl test")
parser.add_argument("--devices",
parser.add_argument(
"--devices",
dest="devices",
help="GPU id of running. if more than one, use spacing to separate.",
nargs="+",
......@@ -59,7 +60,7 @@ if __name__ == "__main__":
train(
flags=["--cfg", cfg, "--use_gpu", "--log_steps", "10"],
options=[
"SOLVER.NUM_EPOCHS", "1", "TRAIN.PRETRAINED_MODEL", test_model,
"SOLVER.NUM_EPOCHS", "1", "TRAIN.PRETRAINED_MODEL_DIR", test_model,
"TRAIN.MODEL_SAVE_DIR", saved_model, "DATASET.TRAIN_FILE_LIST",
os.path.join(DATASET_PATH, "mini_pet", "file_list",
"train_list.txt"), "DATASET.VAL_FILE_LIST",
......
turtorial/finetune_deeplabv3plus.md
浏览文件 @ a1fbffca
......@@ -67,7 +67,7 @@ MODEL:
DEEPLAB:
BACKBONE: "xception_65"
TRAIN:
PRETRAINED_MODEL: "./pretrained_model/deeplabv3p_xception65_bn_pet/"
PRETRAINED_MODEL_DIR: "./pretrained_model/deeplabv3p_xception65_bn_pet/"
# 其他配置
......@@ -79,7 +79,6 @@ AUG:
BATCH_SIZE: 4
TRAIN:
MODEL_SAVE_DIR: "./finetune/deeplabv3p_xception65_bn_pet/"
RESUME: False
SNAPSHOT_EPOCH: 10
TEST:
TEST_MODEL: "./finetune/deeplabv3p_xception65_bn_pet/final"
......
turtorial/finetune_icnet.md
浏览文件 @ a1fbffca
......@@ -66,7 +66,7 @@ MODEL:
DEFAULT_NORM_TYPE: "bn"
MULTI_LOSS_WEIGHT: "[1.0, 0.4, 0.16]"
TRAIN:
PRETRAINED_MODEL: "./pretrained_model/icnet_bn_cityscapes/"
PRETRAINED_MODEL_DIR: "./pretrained_model/icnet_bn_cityscapes/"
# 其他配置
......@@ -78,7 +78,6 @@ AUG:
BATCH_SIZE: 4
TRAIN:
MODEL_SAVE_DIR: "./finetune/icnet_pet/"
RESUME: False
SNAPSHOT_EPOCH: 10
TEST:
TEST_MODEL: "./finetune/icnet_pet/final"
......@@ -118,4 +117,4 @@ python pdseg/eval.py --use_gpu --cfg ./configs/test_pet.yaml
|预训练模型名称|BackBone|Norm|数据集|配置|
|-|-|-|-|-|
|icnet_bn_cityscapes|-|bn|Cityscapes|MODEL.MODEL_NAME: icnet <br> MODEL.DEFAULT_NORM_TYPE: bn|
|icnet_bn_cityscapes|-|bn|Cityscapes|MODEL.MODEL_NAME: icnet <br> MODEL.DEFAULT_NORM_TYPE: bn <br> MULTI_LOSS_WEIGHT: [1.0, 0.4, 0.16]|
turtorial/finetune_unet.md
浏览文件 @ a1fbffca
......@@ -65,7 +65,7 @@ MODEL:
MODEL_NAME: "unet"
DEFAULT_NORM_TYPE: "bn"
TRAIN:
PRETRAINED_MODEL: "./pretrained_model/unet_bn_coco/"
PRETRAINED_MODEL_DIR: "./pretrained_model/unet_bn_coco/"
# 其他配置
......@@ -77,7 +77,6 @@ AUG:
BATCH_SIZE: 4
TRAIN:
MODEL_SAVE_DIR: "./finetune/unet_pet/"
RESUME: False
SNAPSHOT_EPOCH: 10
TEST:
TEST_MODEL: "./finetune/unet_pet/final"
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或