Big storage needed for PaddlePaddle compilation
Created by: luotao1
This question is from Rysztowski, Marcin in intel.
Compilation of PaddlePaddle requires big amount of free disk space (several gigabytes) and the number continues to increase. Our internal validation bases on docker files and the situation leads to few problems:
- As disk space requirement increases, we run to lack of free space in image and compilation errors (so we need to increase image size manually)
- We create an image once and send it to various servers for test runs- sending 60GB image over network takes a lot of time
- Also very often we cannot even fit image on server, as physical disk space may be insufficient (especially if we have few docker files at the same time)
Do you see any chance to limit disk consumption during compilation time?