This closes #5111.

This closes #5070.

Algorithm result types no longer extend Tuple but are now custom POJOs.
This prevents the use of DataSet#writeAsCsv which accepts custom line
and field delimiters. Anticipating this lose the result types override

This patch uses DataSet#writeAsCsv for types extending Tuple and
DataSet#writeAsText otherwise. Since most algorithm result types are now
custom POJOs the custom delimiters are now largely ineffective.

[hotfix] Rename RestfulGateway#requestJobDetails into requestMultipleJobDetails to avoid name conflicts with JobMasterGateway

By letting the existing REST handlers work with an arbitrary RestfulGateway,
they can be used by the Dispatcher as well as the JobMaster, once it implements
the RestfulGateway.

This closes #4985.

This closes #5069.

This closes #5017.

This closes #5075.

Instead of storing the JobID and the JobStatus in a Tuple2 which is serialized
as an array of values in JSON, this commit introduces the JobIdWithStatus which
is serialized as a proper JSON object with an id and a status field.

This closes #4734.

Closes #5096

We always add the dist.jar ourselves, but it could also be inside a shipped
folder such as the "lib/" folder and was then distributed multiple times.

This closes #4951.

Migrate logic in
org.apache.flink.runtime.rest.handler.legacy.metrics.JobMetrisHandler to new
handler, and add new handler to DispatcherRestEndpoint.

[FLINK-7694][flip6] Use jobid path parameter constant in JobMetricsHandlerTest

This closes #5084.

Migrate logic from
org.apache.flink.runtime.rest.handler.legacy.metrics.SubtaskMetricsHandler to
new handler. Add new handler to DispatcherRestEndpoint.

[FLINK-8143][flip6] Assert that SubtaskIndexPathParameter is mandatory

[FLINK-8143][flip6] Use path parameter constants in SubtaskMetricsHandlerTest

This closes #5082.

Migrate logic in
org.apache.flink.runtime.rest.handler.legacy.metrics.JobManagerMetricsHandler
to new handler and add new handler to DispatcherRestEndpoint.

[FLINK-7716][Javadoc] Deprecate method MetricStore#getJobManager().

There is a semantically equivalent method in MetricStore.

This closes #5083.

Migrate logic in
org.apache.flink.runtime.rest.handler.legacy.metrics.TaskManagerMetricsHandler
to new handler, and add new handler to DispatcherRestEndpoint.

[FLINK-7717][flip6] Use taskmanagerid constant in TaskManagerMetricsHandlerTest

This closes #5081.

Before, the TaskExecutor's InstanceID was exposed as TaskExecutor id. This was wrong
since the InstanceID is bound the registration of a TaskExecutor whereas the
ResourceID is bound to the lifetime of the TaskExecutor. Thus, it is better to identify
the TaskExecutor by its ResourceID which does not change.

This commit changes the behaviour accordingly on the ResourceManager and the
TaskManagerDetailsHandler.

This closes #5093.

This closes #5001.

This closes #4574

When printing the job plan the operator description is typically
formatted as the operator name followed by the user given or generated
function name in parenthesis. For example, "Reduce (My Function)".

This normalizes the node names to include a space between the operator
and function names.

This closes #4383

Add "Remote connection to [null] failed with java.nio.channels.NotYetConnectedException"
to the list of whitelisted log statements in YarnTestBase. This logging statement
seems to appear since we moved from Flakka to Akka 2.4.0.

This closes #5085.

This closes #4748

This closes #4946

This closes #5034

This closes #5067

This closes #5061

This closes #5077.

This closes #5049.

Increasing timeout for reading the records from 30s to 60s seems to solve the issue
for failing at-least-one tests.