Before training model, It is friendly if paddle can indicate whether the device's memory is enough or not !355
Created by: chengduoZH
When I use docker to train image classification model, the training process is killed by the operating system because the device storage is not enough. Just like this:
root@a2f510f45102:/book/03.image_classification# python train.py
I0801 03:08:32.183323 27 Util.cpp:166] commandline: --use_gpu=False --trainer_count=1
[INFO 2017-08-01 03:08:32,192 layers.py:2188] output for __conv_0__: c = 64, h = 32, w = 32, size = 65536
[INFO 2017-08-01 03:08:32,194 layers.py:2188] output for __conv_1__: c = 64, h = 32, w = 32, size = 65536
[INFO 2017-08-01 03:08:32,196 layers.py:2313] output for __pool_0__: c = 64, h = 16, w = 16, size = 16384
[INFO 2017-08-01 03:08:32,197 layers.py:2188] output for __conv_2__: c = 128, h = 16, w = 16, size = 32768
[INFO 2017-08-01 03:08:32,199 layers.py:2188] output for __conv_3__: c = 128, h = 16, w = 16, size = 32768
[INFO 2017-08-01 03:08:32,201 layers.py:2313] output for __pool_1__: c = 128, h = 8, w = 8, size = 8192
[INFO 2017-08-01 03:08:32,202 layers.py:2188] output for __conv_4__: c = 256, h = 8, w = 8, size = 16384
[INFO 2017-08-01 03:08:32,204 layers.py:2188] output for __conv_5__: c = 256, h = 8, w = 8, size = 16384
[INFO 2017-08-01 03:08:32,205 layers.py:2188] output for __conv_6__: c = 256, h = 8, w = 8, size = 16384
[INFO 2017-08-01 03:08:32,207 layers.py:2313] output for __pool_2__: c = 256, h = 4, w = 4, size = 4096
[INFO 2017-08-01 03:08:32,208 layers.py:2188] output for __conv_7__: c = 512, h = 4, w = 4, size = 8192
[INFO 2017-08-01 03:08:32,210 layers.py:2188] output for __conv_8__: c = 512, h = 4, w = 4, size = 8192
[INFO 2017-08-01 03:08:32,212 layers.py:2188] output for __conv_9__: c = 512, h = 4, w = 4, size = 8192
[INFO 2017-08-01 03:08:32,214 layers.py:2313] output for __pool_3__: c = 512, h = 2, w = 2, size = 2048
[INFO 2017-08-01 03:08:32,215 layers.py:2188] output for __conv_10__: c = 512, h = 2, w = 2, size = 2048
[INFO 2017-08-01 03:08:32,216 layers.py:2188] output for __conv_11__: c = 512, h = 2, w = 2, size = 2048
[INFO 2017-08-01 03:08:32,218 layers.py:2188] output for __conv_12__: c = 512, h = 2, w = 2, size = 2048
[INFO 2017-08-01 03:08:32,220 layers.py:2313] output for __pool_4__: c = 512, h = 1, w = 1, size = 512
[INFO 2017-08-01 03:08:32,225 networks.py:1482] The input order is [image, label]
[INFO 2017-08-01 03:08:32,225 networks.py:1488] The output order is [__classification_cost_0__]
[INFO 2017-08-01 03:08:32,228 layers.py:2188] output for __conv_0__: c = 64, h = 32, w = 32, size = 65536
[INFO 2017-08-01 03:08:32,230 layers.py:2188] output for __conv_1__: c = 64, h = 32, w = 32, size = 65536
[INFO 2017-08-01 03:08:32,231 layers.py:2313] output for __pool_0__: c = 64, h = 16, w = 16, size = 16384
[INFO 2017-08-01 03:08:32,232 layers.py:2188] output for __conv_2__: c = 128, h = 16, w = 16, size = 32768
[INFO 2017-08-01 03:08:32,234 layers.py:2188] output for __conv_3__: c = 128, h = 16, w = 16, size = 32768
[INFO 2017-08-01 03:08:32,236 layers.py:2313] output for __pool_1__: c = 128, h = 8, w = 8, size = 8192
[INFO 2017-08-01 03:08:32,237 layers.py:2188] output for __conv_4__: c = 256, h = 8, w = 8, size = 16384
[INFO 2017-08-01 03:08:32,239 layers.py:2188] output for __conv_5__: c = 256, h = 8, w = 8, size = 16384
[INFO 2017-08-01 03:08:32,240 layers.py:2188] output for __conv_6__: c = 256, h = 8, w = 8, size = 16384
[INFO 2017-08-01 03:08:32,242 layers.py:2313] output for __pool_2__: c = 256, h = 4, w = 4, size = 4096
[INFO 2017-08-01 03:08:32,243 layers.py:2188] output for __conv_7__: c = 512, h = 4, w = 4, size = 8192
[INFO 2017-08-01 03:08:32,245 layers.py:2188] output for __conv_8__: c = 512, h = 4, w = 4, size = 8192
[INFO 2017-08-01 03:08:32,247 layers.py:2188] output for __conv_9__: c = 512, h = 4, w = 4, size = 8192
[INFO 2017-08-01 03:08:32,248 layers.py:2313] output for __pool_3__: c = 512, h = 2, w = 2, size = 2048
[INFO 2017-08-01 03:08:32,249 layers.py:2188] output for __conv_10__: c = 512, h = 2, w = 2, size = 2048
[INFO 2017-08-01 03:08:32,251 layers.py:2188] output for __conv_11__: c = 512, h = 2, w = 2, size = 2048
[INFO 2017-08-01 03:08:32,253 layers.py:2188] output for __conv_12__: c = 512, h = 2, w = 2, size = 2048
[INFO 2017-08-01 03:08:32,255 layers.py:2313] output for __pool_4__: c = 512, h = 1, w = 1, size = 512
[INFO 2017-08-01 03:08:32,258 networks.py:1482] The input order is [image, label]
[INFO 2017-08-01 03:08:32,259 networks.py:1488] The output order is [__classification_cost_0__]
I0801 03:08:32.382684 27 GradientMachine.cpp:86] Initing parameters..
I0801 03:08:33.513288 27 GradientMachine.cpp:93] Init parameters done.
Killed
I think that It will be friendly if paddle can indicate whether the device's memory is enough or not, before training the model