In the future, we'd like to encourage user to save
everything during training.
This allows us to
1. Do more flexible optimization passes
2. Re-train and fune-tune