[tests] Consolidate miscellaneous tests into one IntegrationTestCase to reuse minicluster and speed up tests

[tests] Adapted CoGroupITCase to use "collect()" rather than writing temp files (increases robustness)

[streaming] Integrate new checkpointed interface with StreamTask, StreamOperator, and PersistentKafkaSource

[FLINK-1973] [jobmanager] Task execution state messages are logged on INFO level, rather than on DEBUG level

[FLINK-1953] [runtime] Integrate new snapshot checkpoint coordinator with jobgraph and execution graph

This closes #651

hide warning if output file does not exist

This closes #370

This closes #615

This closes #662

This closes #668

Bash can run out of environment memory if environment variables store
too much data. When the memory is exceeded, bash can't run system
commands anymore and, from then on, cryptically fails every command with
"Argument list too long".

This closes #670.

This closes #473

 - This simplifies and hardens the failure handling during task startup
 - Guarantees that no actor system threads are blocked by task bootstrap, or task canceling
 - Corrects some previously erroneous corner case state transitions
 - Adds simple and robust tests

 - Gives a proper exception when a non-cached file is accessed
 - Forwards I/O exceptions that happen during file transfer, rather than only returning null when transfer failed
 - Consistently keeps reference counts and copies only when needed
 - Properly removes all files when shutdown
 - Uses a shutdown hook to remove files when process is killed

[streaming] New Source and state checkpointing interfaces that allow operations to interact with the state checkpointing in a more precise manner.

- Fix JobInfo to report milliseconds
- Added documentation to indicate that the return type is in milliseconds
- Added an getNetRuntime method which accepts a desired time unit for easy conversion

This closes #652

show/hide graphs, add summary for metrics

This closes #553

(Also close the mesos pull request)
This closes #251

The code that was generated for this used "==", now it uses .equals()

Added Stochastic Gradient Descent initial version and some tests.

Added L1, L2 regularization.

Added tests for regularization, fixed parameter setting.

Added documentation.

Added option to provide UDF for the prediction function, moved SGD regularization to update step.

Added prediction function class to allow non-linear optimization in the future.

Small refactoring to allow calculation of regularized loss separatly from
regularized gradient.

This closes #613.

[scala] Introduces trait type to TypeAnalyzer which avoids compiler warnings due to misclassification as a pojo type

[ml] [docs] Adds distance formulas to documentation. Automatically include latex_commands.html with mathjax markdown files.

This closes #657

[ml] [docs] Add a overview documentation for Distance Metrics

[FLINK-1933] [ml] Moves metrics.distances package to org.apache.flink.ml

This closes #629.

This closes #636.