The flip6 build profile only runs the Flip-6 related test cases. Moreover,
all Flip-6 related test cases are excluded when not running the flip6 build
profile. This should reduce testing time when adding more and more Flip-6
test cases.

Include flink-test-utils-junit in all submodules to make the Category marker interfaces Flip6 and OldAndFlip6 available

This closes #4889.

This closes #5020.

Before, we had it in places that require it. This doesn't work when
running mvn javadoc:aggregate because this will only run for the root
pom and can then not find the "bundle" dependencies.

Disable it in most modules.

This closes #4923.

We can do this now that we dropped support for Scala 2.10.

This closes #4807

Add dependencies for batch and streaming WordCount programs and copies
the jar files into a new target/programs directory. The integration
tests now directly references the program jar files rather than the
prior brittle search.

This removes the flink-yarn-tests build-time dependency on the examples
modules (there remains a build-time dependency on flink-dist).

This closes #4264

Update Hadoop versions and replace Hadoop 2.3 with 2.8 in build and
continuous integration scripts. flink-yarn-tests can now be enabled for
all supported Hadoop versions and is made non-optional.

This closes #3832

This closes #4005.

Use scala.binary.version as defined in the parent POM and remove the
script to swap scala version identifiers.

This closes #3800

This closes #3132

This uses Flakka (a custom Akka 2.3 build) to resolve the issue that
the bind address needs to be matching the external address of the
JobManager. With the changes applied, we can now bind to all
interfaces, e.g. via 0.0.0.0 (IPv4) or :: (IPv6).

For this to work properly, the configuration entry
JOB_MANAGER_IPC_ADDRESS now represents the external address of the
JobManager. Consequently, it should not be resolved to an IP address
anymore because it may not be resolvable from within containered
environments. Akka treats this address as the logical address. Any
messages which are not tagged with this address will be received by
the Actor System (because we listen on all interfaces) but will be
dropped subsequently. In addition, we need the external address for
the JobManager to be able to publish it to Zookeeper for HA setups.

Flakka: https://github.com/mxm/flakka
Patch applied: https://github.com/akka/akka/pull/15610

- convert host to lower case
- use consistent format for IPv6 address
- adapt config and test cases
- adapt documentation to clarify the address config entry
- TaskManager: resolve the initial hostname of the StandaloneLeaderRetrievalService

This closes #2917.

This closes #2850

[FLINK-3929] Added Keytab based Kerberos support to enable secure Flink cluster deployment(addresses HDHS, Kafka and ZK services)

FLINK-3929 Added MiniKDC support for Kafka, Zookeeper, RollingFS and Yarn integration test modules

Rename _configuration to originalConfiguration

Remove testing classes from main scope in flink-runtime

Previously, the ForkableFlinkMiniCluster which resided in flink-test-utils required
these files to be in the main scope of flink-runtime. With the removal of the
ForkableFlinkMiniCluster, these classes are now no longer needed and can be moved
back to the test scope.

This closes #2450.

This closes #2324

- Add YarnFlinkResourceManager test to reaccept task manager registrations from a re-elected job manager

- Remove unnecessary sync logic between JobManager and ResourceManager

- Avoid duplicate reigstration attempts in case of a refused registration

- Add test case to check that not an excessive amount of RegisterTaskManager messages are sent

- Remove containersLaunched from YarnFlinkResourceManager and instead not clearing registeredWorkers when JobManager loses leadership

- Let YarnFlinkResourceManagerTest extend TestLogger

- Harden YarnFlinkResourceManager.getContainersFromPreviousAttempts

- Add FatalErrorOccurred message handler to FlinkResourceManager;
  Increase timeout for YarnFlinkResourceManagerTest;
  Add additional constructor to TestingYarnFlinkResourceManager for tests

- Rename registeredWorkers field into startedWorkers
Additionally, the RegisterResource message is renamed into NotifyResourceStarted which
tells the RM that a resource has been started. This reflects the current semantics of
the startedWorkers map in the resource manager.

- Fix concurrency issues in TestingLeaderRetrievalService

This closes #2257

Makes the JUnit test utils (TestLogger, retry rules, ...) properly available to
other projects without the 'flink-core' test-jar, via the 'flink-test-utils-junit' project.

Makes the ForkableMiniCluster, TestEnvironment,  and other test utilities available in the 'main'
scope of the 'flink-test-utils' project.

Creates a 'flink-test-utils-parent' project that holds the 'flink-test-utils-junit' and
'flink-test-utils' project.

Also moves some tests between projects and inlines some very simple utility functions in
order to simplify some test jar dependencies.

- ClusterDescriptor: base interface for cluster deployment descriptors
- ClusterDescriptor: YarnClusterDescriptor

- ClusterClient: base class for ClusterClients, handles lifecycle of cluster
- ClusterClient: shares configuration with the implementations
- ClusterClient: StandaloneClusterClient, YarnClusterClient
- ClusterClient: remove run methods and enable detached mode via flag

- CliFrontend: remove all Yarn specific logic
- CliFrontend: remove all cluster setup logic

- CustomCommandLine: interface for other cluster implementations
- Customcommandline: enables creation of new cluster or resuming from existing

- Yarn: move Yarn classes and functionality to the yarn module (yarn
  properties, yarn interfaces)
- Yarn: improve reliability of cluster startup
- Yarn Tests: only disable parallel execution of ITCases

This closes #1978

This closes #2042

As of 70978f56, the Yarn tests were not
executed anymore. They were moved to the test directory but there was
still a Maven configuration in place to change the test directory
location.

This closes #2012

Failures during execution of the integration tests with the Maven
Failsafe plugin were silently ignored on Travis with Maven version
3.2.5. The problem is that failures are not passed on correctly from the
'integration-test' phase (where failures are recorded and tolerated) to
the 'verify' phase.

The cause of the error is most likely SUREFIRE-1127. An exception in the
'integration-test' is sometimes not flushed back to disk where it is
evaluated in the 'verify' phase. Bumping the version of the Failsafe
plugin from 2.18.1 to 2.19.1 fixes the issue but introduces classpath
related problems.

We don't have to rely on Failsafe for running integration
tests. Surefire offers the same feature set we use from the Failsafe
plugin.

- replace Failsafe with Surefire
- update to latest Surefire version
- get rid of duplicate property declaration

This closes #2003

This closes #1915

Consolidation is done via plugin management.

Closes #1753

- add suffixes
- adapt change-scala-version script

This closes #1529.

This closes #1409

0.10-SNAPSHOT continues on branch release-0.10

This squashes the following commits:
- Refactor JobManager's start actors method to be reusable
- Yarn refactoring to introduce yarn testing functionality
- Add support for testing yarn cluster. Extracted JobManager's and TaskManager's testing messages into stackable traits.
- Implement YarnHighAvailabilityITCase using Akka messages for synchronization.
- Logging statements
- Fix registration at JobManager when the leader address is null
- Fix curator dependency conflict
- Shades Flink's curator dependency in flink-runtime so that it cannot be overriden by external dependencies in the class path. This solves the problem with Hadoop 2.6.0 which adds Curator 2.6.0 to the class path. The curator version of this Hadoop version is not compatible to Flink's Curator version 2.8.0. Furthermore, Flink's Guava version is forced to be included in flink-shaded-curator jar to avoid to many different Guava version in the resulting dist jar.
- Unify two shade executions of flink-runtime into one
- Exclude log4j and slf4j-log4j12 dependency from flink-shaded-curator
- Set default number of application attempts to 1 in standalone case

This closes #1213

This is needed because the Hadoop IF/OF's are using Hadoop's FileSystem stack, which is using
the security credentials passed in the JobConf / Job class in the getSplits() method.

Note that access to secured Hadoop 1.x using Hadoop IF/OF's is not possible with this change.
This limitation is due to missing methods in the old APIs.

- Add some comments & change dependency scope to test

Closes #851

This closes #837.

This closes #689

Adds pipeline comments

Adds pipeline IT case

[FLINK-2050] [ml] Introduces new pipelining mechanism using implicit classes to wrap the algorithm logic

This closes #704.