* Support testing during training by ParallelExecutor.

* Add unit test.

* Improve the interface.

* Follow comments.