This closes #4211.

This closes #6575.

This closes #3776.

Previously, YARN log aggregation fails because it depoends on the HDFS
delegation token, which we do not upload if Kerberos keytabs are used.
We did not include HDFS delegation tokens when keytabs are used because
the UGI would prioritize the delegation token (which expires) if both
are present.

To address this, changes include:

1. Change Flink YARN client to always upload delegation tokens when
security is enabled. This would then allow log aggregation.

2. Filter out HDFS delegation token from the tokens fetched from HDFS
when populating the UGI. This allows the UGI to always use Kerberos
tickets instead of the HDFS delegation token.

This closes #3855.

This closes #4216.

The root cause for the instability in ES 1.x is that we're testing both
embedded client mode and transport client mode against the same
Elasticsearch environment. This commit removes the embedded mode
testing, as that method is generally discouraged anyways.

This closes #4191.

The testStartFromKafkaCommitOffsets ITCases are covering too much within
one single test. The case verifies that whatever offset was committed to
Kafka, Flink rreads it correctly and can uses that as the correct
starting point for exactly-once.

This over-engineered test was instable that we needed to first read some
records and wait until some records is committed. This wait is hard to
define.

It is in fact sufficient to have 2 separate tests to cover the tested
behaviour:
- test that committed Kafka offsets are correct (there is already a
  ITCase for this, i.e. `runCommitOffsetsToKafka`)
- test that committed offsets are correctly picked up and used correctly
  (there is actually also a test for this, i.e.
`runStartFromGroupOffsets`)

Hence, this test can be removed without harming test coverage.

This closes #4190.

This closes #3746.

This closes #3889.

This closes #4212.

- Move Java example tests into a single ITCase
- Add ITCase for Scala examples

This closes #2761.

This closes #4195.

The current organization of the Gelly examples retains full flexibility
by handling the Graph input to the algorithm Driver and having the
Driver overload interfaces for the various output types. The outputs
must be made independent in order to support Transforms which are
applied between the Driver and Output (and also between the Input and
Driver).

This closes #4179

Some internal methods which are required for the interplay between the TaskMonitor,
LaunchCoordinator and the MesosResourceManager were exposed as RPC methods. In order
to keep the RPC interface as lean as possible, these methods have been removed.

Fix checkstyle violations

The MesosResourceManagerTest#testAdapter tests the AkkaAdapter class. The tests
are executed asynchronously and thus it is necessary to introduce timeouts for
the verify calls. This commit fixes the test instability by introducing timeouts.

- Make the RPC gateway of the ResourceManager extensible to allow for framework-specific RPC methods
- Introduce FLIP-6 MesosResourceManager w/ tests
- Introduce a Mesos-specific RPC gateway for callbacks from child actors and from the Mesos scheduler client
- Enhance the persistent Mesos worker store to track the resource profile associated with a worker
- Convert RegisteredMesosWorkerNode to Java
- Decline TE registration if framework doesn’t recognize the worker

This closes #3942.

This closes #4200

- enable sudo for more memory
- increase java heap size
- replace usage of oraclejdk7 since it is no longer supporter
- manually install maven 3.2.5

This closes #4182.

This closes #4186.

This closes #4161.

This closes #4143.

This closes #4175.

This closes #4181.

This closes #4152

This closes #4177

This closes #4017.

This closes #4151.

This closes #4034.

This closes #4148.

This closes #4159.

This closes #4140.

This closes #4074.

This closes #4125.

This closes #3894.

This closes #4024.