modify labelme jingling + fix label bug (#91)

* modify annotation stript path

modify labelme jingling + fix label bug (#91)
* modify annotation stript path
1dcfc05d · LutaoChu · wuzewu · f63d7dc0 · f63d7dc0 · 1dcfc05d
18 changed file
--- a/docs/annotation/data_annotated/aa63d7e6db0d03137883772c246c6761fc201059.json
+++ b/docs/annotation/data_annotated/aa63d7e6db0d03137883772c246c6761fc201059.json
-{"path":"/Users/chulutao/dataset/humanseg/aa63d7e6db0d03137883772c246c6761fc201059.jpg","outputs":{"object":[{"name":"person","polygon":{"x1":321.99,"y1":63,"x2":293,"y2":98.00999999999999,"x3":245.01,"y3":141.01,"x4":221,"y4":194,"x5":231.99,"y5":237,"x6":231.99,"y6":348.01,"x7":191,"y7":429,"x8":197,"y8":465.01,"x9":193,"y9":586,"x10":151,"y10":618.01,"x11":124,"y11":622,"x12":100,"y12":703,"x13":121.99,"y13":744,"x14":141.99,"y14":724,"x15":163,"y15":658.01,"x16":238.01,"y16":646,"x17":259,"y17":627,"x18":313,"y18":618.01,"x19":416,"y19":639,"x20":464,"y20":606,"x21":454,"y21":555.01,"x22":404,"y22":508.01,"x23":430,"y23":489,"x24":407,"y24":464,"x25":397,"y25":365.01,"x26":407,"y26":290,"x27":361.99,"y27":252,"x28":376,"y28":215.01,"x29":391.99,"y29":189,"x30":388.01,"y30":135.01,"x31":340,"y31":120,"x32":313,"y32":161.01,"x33":307,"y33":188.01,"x34":311,"y34":207,"x35":277,"y35":186,"x36":293,"y36":137,"x37":308.01,"y37":117,"x38":361,"y38":93}}]},"time_labeled":1568101256852,"labeled":true,"size":{"width":706,"height":1000,"depth":3}}
\ No newline at end of file
--- a/docs/annotation/jingling2seg.md
+++ b/docs/annotation/jingling2seg.md
@@ -44,7 +44,7 @@
 **注意：导出的标注文件位于`保存位置`下的`outputs`目录。**
-精灵标注产出的真值文件可参考我们给出的文件夹`data_annotated`。
+精灵标注产出的真值文件可参考我们给出的文件夹`docs/annotation/jingling_demo`。
 <div align="center">
    <img src="../imgs/annotation/jingling-4.png" width="300px"/>
@@ -55,21 +55,24 @@
 ## 3 数据格式转换
-* 我们用于完成语义分割的数据集目录结构如下：
+* 经过数据格式转换后的数据集目录结构如下：
 ```
 my_dataset                 # 根目录 
- |-- JPEGImages             # 数据集图片
+ |-- outputs                # 标注工具导出目录
- |-- SegmentationClassPNG   # 数据集真值 
+ |   |-- annotations            # 数据集真值 
- |   |-- xxx.png            # 像素级别的真值信息 
+ |       |-- xxx.png            # 像素级别的真值信息 
- |   |... 
+ |       |... 
- |-- class_names.txt        # 数据集的类别名称
+ |   |-- class_names.txt        # 数据集的类别名称
+ |   |-- xxx.json               # 标注json文件
+ |-- xxx.jpg(png or other)  # 数据集原图
+ |-- ...
 ```
 <div align="center">
-    <img src="../imgs/annotation/image-6.png" width="600px"/>
+    <img src="../imgs/annotation/image-6-2.png" width="600px"/>
-    <p>图5 训练所需的数据集目录的结构示意图</p>
+    <p>图5 格式转换后的数据集目录的结构示意图</p>
 </div>
 * 运行转换脚本需要依赖labelme和pillow，如未安装，请先安装。Labelme的具体安装流程请参见[官方安装指南](https://github.com/wkentaro/labelme)。Pillow的安装：
@@ -81,16 +84,15 @@ pip install pillow
 * 运行以下代码，将标注后的数据转换成满足以上格式的数据集：
 ```
-  python docs/annotation/jingling2seg.py <path/to/label_json_file> <path/to/output_dataset>
+  python pdseg/tools/jingling2seg.py <path/to/label_json_file> 
 ```
-其中，`<path/to/label_json_files>`为精灵标注产出的json文件所在文件夹的目录，一般为精灵工具使用（3）中`保存位置`下的`outputs`目录。`<path/to/output_dataset>`为转换后的数据集所在文件夹的目录。
+其中，`<path/to/label_json_files>`为精灵标注产出的json文件所在文件夹的目录，一般为精灵工具使用（3）中`保存位置`下的`outputs`目录。
-**注意：`<path/to/output_dataset>`不用预先创建，脚本运行时会自动创建，否则会报错。**
-转换得到的数据集可参考我们给出的文件夹`my_dataset`。其中，文件`class_names.txt`是数据集中所有标注类别的名称，包含背景类；文件夹`JPEGImages`保存的是数据集的图片；文件夹`SegmentationClassPNG`保存的是各图片的像素级别的真值信息，背景类`_background_`对应为0，其它目标类别从1开始递增，至多为255。
+转换得到的数据集可参考我们给出的文件夹`docs/annotation/jingling_demo`。其中，文件`class_names.txt`是数据集中所有标注类别的名称，包含背景类；文件夹`annotations`保存的是各图片的像素级别的真值信息，背景类`_background_`对应为0，其它目标类别从1开始递增，至多为255。
 <div align="center">
    <img src="../imgs/annotation/jingling-5.png" width="600px"/>
-    <p>图6 训练所需的数据集各目录的内容示意图</p>
+    <p>图6 格式转换后的数据集各目录的内容示意图</p>
 </div>	
--- a/docs/annotation/data_annotated/aa63d7e6db0d03137883772c246c6761fc201059.jpg
+++ b/docs/annotation/data_annotated/aa63d7e6db0d03137883772c246c6761fc201059.jpg
--- a/docs/annotation/jingling_demo/outputs/aa63d7e6db0d03137883772c246c6761fc201059.json
+++ b/docs/annotation/jingling_demo/outputs/aa63d7e6db0d03137883772c246c6761fc201059.json
+{"path":"/Users/dataset/aa63d7e6db0d03137883772c246c6761fc201059.jpg","outputs":{"object":[{"name":"person","polygon":{"x1":321.99,"y1":63,"x2":293,"y2":98.00999999999999,"x3":245.01,"y3":141.01,"x4":221,"y4":194,"x5":231.99,"y5":237,"x6":231.99,"y6":348.01,"x7":191,"y7":429,"x8":197,"y8":465.01,"x9":193,"y9":586,"x10":151,"y10":618.01,"x11":124,"y11":622,"x12":100,"y12":703,"x13":121.99,"y13":744,"x14":141.99,"y14":724,"x15":163,"y15":658.01,"x16":238.01,"y16":646,"x17":259,"y17":627,"x18":313,"y18":618.01,"x19":416,"y19":639,"x20":464,"y20":606,"x21":454,"y21":555.01,"x22":404,"y22":508.01,"x23":430,"y23":489,"x24":407,"y24":464,"x25":397,"y25":365.01,"x26":407,"y26":290,"x27":361.99,"y27":252,"x28":376,"y28":215.01,"x29":391.99,"y29":189,"x30":388.01,"y30":135.01,"x31":340,"y31":120,"x32":313,"y32":161.01,"x33":307,"y33":188.01,"x34":311,"y34":207,"x35":277,"y35":186,"x36":293,"y36":137,"x37":308.01,"y37":117,"x38":361,"y38":93}}]},"time_labeled":1568101256852,"labeled":true,"size":{"width":706,"height":1000,"depth":3}}
\ No newline at end of file
--- a/docs/annotation/jingling_demo/outputs/annotations/aa63d7e6db0d03137883772c246c6761fc201059.png
+++ b/docs/annotation/jingling_demo/outputs/annotations/aa63d7e6db0d03137883772c246c6761fc201059.png
--- a/docs/annotation/jingling_demo/outputs/class_names.txt
+++ b/docs/annotation/jingling_demo/outputs/class_names.txt
+_background_
+person
\ No newline at end of file
--- a/docs/annotation/labelme2seg.md
+++ b/docs/annotation/labelme2seg.md
@@ -47,7 +47,7 @@ git clone https://github.com/wkentaro/labelme
 		(3)   图片中所有目标的标注都完成后，点击`Save`保存json文件，**请将json文件和图片放在同一个文件夹里**，点击`Next Image`标注下一张图片。
-LableMe产出的真值文件可参考我们给出的文件夹`data_annotated`。
+LableMe产出的真值文件可参考我们给出的文件夹`docs/annotation/labelme_demo`。
 <div align="center">
    <img src="../imgs/annotation/image-5.png" width="600px"/>
@@ -65,21 +65,24 @@ LableMe产出的真值文件可参考我们给出的文件夹`data_annotated`。
 ## 3 数据格式转换
-* 我们用于完成语义分割的数据集目录结构如下：
+* 经过数据格式转换后的数据集目录结构如下：
 ```
 my_dataset                 # 根目录 
- |-- JPEGImages             # 数据集图片
+ |-- annotations            # 数据集真值 
- |-- SegmentationClassPNG   # 数据集真值 
 |   |-- xxx.png            # 像素级别的真值信息 
 |   |... 
 |-- class_names.txt        # 数据集的类别名称
+ |-- xxx.jpg(png or other)  # 数据集原图
+ |-- ...
+ |-- xxx.json               # 标注json文件
+ |-- ...
 ```
 <div align="center">
    <img src="../imgs/annotation/image-6.png" width="600px"/>
-    <p>图7 训练所需的数据集目录的结构示意图</p>
+    <p>图7 格式转换后的数据集目录的结构示意图</p>
 </div>
 * 运行转换脚本需要依赖labelme和pillow，如未安装，请先安装。Labelme的具体安装流程请参见[官方安装指南](https://github.com/wkentaro/labelme)。Pillow的安装：
@@ -91,14 +94,14 @@ pip install pillow
 * 运行以下代码，将标注后的数据转换成满足以上格式的数据集：
 ```
-  python docs/annotation/labelme2seg.py <path/to/label_json_file> <path/to/output_dataset>
+  python pdseg/tools/labelme2seg.py <path/to/label_json_file>
 ```
-其中，`<path/to/label_json_files>`为图片以及LabelMe产出的json文件所在文件夹的目录，`<path/to/output_dataset>`为转换后的数据集所在文件夹的目录。**需注意的是：`<path/to/output_dataset>`不用预先创建，脚本运行时会自动创建，否则会报错。**
+其中，`<path/to/label_json_files>`为图片以及LabelMe产出的json文件所在文件夹的目录，同时也是转换后的标注集所在文件夹的目录。
-转换得到的数据集可参考我们给出的文件夹`my_dataset`。其中，文件`class_names.txt`是数据集中所有标注类别的名称，包含背景类；文件夹`JPEGImages`保存的是数据集的图片；文件夹`SegmentationClassPNG`保存的是各图片的像素级别的真值信息，背景类`_background_`对应为0，其它目标类别从1开始递增，至多为255。
+转换得到的数据集可参考我们给出的文件夹`docs/annotation/labelme_demo`。其中，文件`class_names.txt`是数据集中所有标注类别的名称，包含背景类；文件夹`annotations`保存的是各图片的像素级别的真值信息，背景类`_background_`对应为0，其它目标类别从1开始递增，至多为255。
 <div align="center">
    <img src="../imgs/annotation/image-7.png" width="600px"/>
-    <p>图8 训练所需的数据集各目录的内容示意图</p>
+    <p>图8 格式转换后的数据集各目录的内容示意图</p>
 </div>	
--- a/docs/annotation/data_annotated/2011_000025.jpg
+++ b/docs/annotation/data_annotated/2011_000025.jpg
--- a/docs/annotation/data_annotated/2011_000025.json
+++ b/docs/annotation/data_annotated/2011_000025.json
--- a/docs/annotation/labelme_demo/annotations/2011_000025.png
+++ b/docs/annotation/labelme_demo/annotations/2011_000025.png
--- a/docs/annotation/my_dataset/class_names.txt
+++ b/docs/annotation/my_dataset/class_names.txt
--- a/docs/imgs/annotation/image-5.png
+++ b/docs/imgs/annotation/image-5.png
--- a/docs/imgs/annotation/image-6-2.png
+++ b/docs/imgs/annotation/image-6-2.png
--- a/docs/imgs/annotation/image-6.png
+++ b/docs/imgs/annotation/image-6.png
--- a/docs/imgs/annotation/image-7.png
+++ b/docs/imgs/annotation/image-7.png
--- a/docs/imgs/annotation/jingling-5.png
+++ b/docs/imgs/annotation/jingling-5.png
--- a/docs/annotation/jingling2seg.py
+++ b/docs/annotation/jingling2seg.py
@@ -7,7 +7,6 @@ import glob
 import json
 import os
 import os.path as osp
-import sys
 import numpy as np
 import PIL.Image
@@ -15,26 +14,24 @@ import PIL.Image
 import labelme
-def main():
+def parse_args():
    parser = argparse.ArgumentParser(
        formatter_class=argparse.ArgumentDefaultsHelpFormatter
    )
-    parser.add_argument('label_dir', help='input annotated directory')
+    parser.add_argument('input_dir',
-    parser.add_argument('output_dir', help='output dataset directory')
+                        help='input annotated directory')
-    args = parser.parse_args()
+    return parser.parse_args()
-    if osp.exists(args.output_dir):
-        print('Output directory already exists:', args.output_dir)
-        sys.exit(1)
-    os.makedirs(args.output_dir)
+def main(args):
-    os.makedirs(osp.join(args.output_dir, 'JPEGImages'))
+    output_dir = osp.join(args.input_dir, 'annotations')
-    os.makedirs(osp.join(args.output_dir, 'SegmentationClassPNG'))
+    if not osp.exists(output_dir):
-    print('Creating dataset:', args.output_dir)
+        os.makedirs(output_dir)
+        print('Creating annotations directory:', output_dir)
    # get the all class names for the given dataset
    class_names = ['_background_']
-    for label_file in glob.glob(osp.join(args.label_dir, '*.json')):
+    for label_file in glob.glob(osp.join(args.input_dir, '*.json')):
        with open(label_file) as f:
            data = json.load(f)
            if data['outputs']:
@@ -53,19 +50,17 @@ def main():
    class_names = tuple(class_names)
    print('class_names:', class_names)
-    out_class_names_file = osp.join(args.output_dir, 'class_names.txt')
+    out_class_names_file = osp.join(args.input_dir, 'class_names.txt')
    with open(out_class_names_file, 'w') as f:
        f.writelines('\n'.join(class_names))
    print('Saved class_names:', out_class_names_file)
-    for label_file in glob.glob(osp.join(args.label_dir, '*.json')):
+    for label_file in glob.glob(osp.join(args.input_dir, '*.json')):
        print('Generating dataset from:', label_file)
        with open(label_file) as f:
            base = osp.splitext(osp.basename(label_file))[0]
-            out_img_file = osp.join(
-                args.output_dir, 'JPEGImages', base + '.jpg')
            out_png_file = osp.join(
-                args.output_dir, 'SegmentationClassPNG', base + '.png')
+                output_dir, base + '.png')
            data = json.load(f)
@@ -83,19 +78,18 @@ def main():
                        shape = {'label': name, 'points': points, 'shape_type': 'polygon'}
                        data_shapes.append(shape)
-            img_file = osp.join(osp.dirname(label_file), data['path'])
+            data_size = data['size']
-            img = np.asarray(PIL.Image.open(img_file))
+            img_shape = (data_size['height'], data_size['width'], data_size['depth'])
-            PIL.Image.fromarray(img).save(out_img_file)
            lbl = labelme.utils.shapes_to_label(
-                img_shape=img.shape,
+                img_shape=img_shape,
                shapes=data_shapes,
                label_name_to_value=class_name_to_id,
            )
            if osp.splitext(out_png_file)[1] != '.png':
                out_png_file += '.png'
-            # Assume label ranses [0, 255] for uint8,
+            # Assume label ranges [0, 255] for uint8,
            if lbl.min() >= 0 and lbl.max() <= 255:
                lbl_pil = PIL.Image.fromarray(lbl.astype(np.uint8), mode='L')
                lbl_pil.save(out_png_file)
@@ -107,4 +101,5 @@ def main():
 if __name__ == '__main__':
-    main()
+    args = parse_args()
+    main(args)
--- a/docs/annotation/labelme2seg.py
+++ b/docs/annotation/labelme2seg.py
@@ -7,7 +7,6 @@ import glob
 import json
 import os
 import os.path as osp
-import sys
 import numpy as np
 import PIL.Image
@@ -15,21 +14,20 @@ import PIL.Image
 import labelme
-def main():
+def parse_args():
    parser = argparse.ArgumentParser(
        formatter_class=argparse.ArgumentDefaultsHelpFormatter
    )
-    parser.add_argument('input_dir', help='input annotated directory')
+    parser.add_argument('input_dir',
-    parser.add_argument('output_dir', help='output dataset directory')
+                        help='input annotated directory')
-    args = parser.parse_args()
+    return parser.parse_args()
-    if osp.exists(args.output_dir):
-        print('Output directory already exists:', args.output_dir)
+def main(args):
-        sys.exit(1)
+    output_dir = osp.join(args.input_dir, 'annotations')
-    os.makedirs(args.output_dir)
+    if not osp.exists(output_dir):
-    os.makedirs(osp.join(args.output_dir, 'JPEGImages'))
+        os.makedirs(output_dir)
-    os.makedirs(osp.join(args.output_dir, 'SegmentationClassPNG'))
+        print('Creating annotations directory:', output_dir)
-    print('Creating dataset:', args.output_dir)
    # get the all class names for the given dataset
    class_names = ['_background_']
@@ -45,14 +43,14 @@ def main():
    class_name_to_id = {}
    for i, class_name in enumerate(class_names):
-        class_id = i # starts with 0
+        class_id = i  # starts with 0
        class_name_to_id[class_name] = class_id
        if class_id == 0:
            assert class_name == '_background_'
    class_names = tuple(class_names)
    print('class_names:', class_names)
-    out_class_names_file = osp.join(args.output_dir, 'class_names.txt')
+    out_class_names_file = osp.join(args.input_dir, 'class_names.txt')
    with open(out_class_names_file, 'w') as f:
        f.writelines('\n'.join(class_names))
    print('Saved class_names:', out_class_names_file)
@@ -61,16 +59,13 @@ def main():
        print('Generating dataset from:', label_file)
        with open(label_file) as f:
            base = osp.splitext(osp.basename(label_file))[0]
-            out_img_file = osp.join(
-                args.output_dir, 'JPEGImages', base + '.jpg')
            out_png_file = osp.join(
-                args.output_dir, 'SegmentationClassPNG', base + '.png')
+                output_dir, base + '.png')
            data = json.load(f)
            img_file = osp.join(osp.dirname(label_file), data['imagePath'])
            img = np.asarray(PIL.Image.open(img_file))
-            PIL.Image.fromarray(img).save(out_img_file)
            lbl = labelme.utils.shapes_to_label(
                img_shape=img.shape,
@@ -80,7 +75,7 @@ def main():
            if osp.splitext(out_png_file)[1] != '.png':
                out_png_file += '.png'
-            # Assume label ranses [0, 255] for uint8,
+            # Assume label ranges [0, 255] for uint8,
            if lbl.min() >= 0 and lbl.max() <= 255:
                lbl_pil = PIL.Image.fromarray(lbl.astype(np.uint8), mode='L')
                lbl_pil.save(out_png_file)
@@ -90,5 +85,7 @@ def main():
                    'Please consider using the .npy format.' % out_png_file
                )
 if __name__ == '__main__':
-    main()
+    args = parse_args()
+    main(args)