(The groups being tests starting from A-N and N-Z)

This closes #2201

Before the WebRuntimeMonitor was not properly stopped after the ActorSystem has
been terminated.

This also fixes FLINK-4020: remove shard list querying from Kinesis consumer constructor

This closes #2131

This closes #2200

This closes #2198

This closes #2148

Test utilities include the StreamingMultipleProgramsTestBase and StreamingTestEnvironment.

This moves the ITCases for streaming into 'flink-tests' to achieve that.

This closes #2092

Makes the JUnit test utils (TestLogger, retry rules, ...) properly available to
other projects without the 'flink-core' test-jar, via the 'flink-test-utils-junit' project.

Makes the ForkableMiniCluster, TestEnvironment,  and other test utilities available in the 'main'
scope of the 'flink-test-utils' project.

Creates a 'flink-test-utils-parent' project that holds the 'flink-test-utils-junit' and
'flink-test-utils' project.

Also moves some tests between projects and inlines some very simple utility functions in
order to simplify some test jar dependencies.

Before, job would fail with an exception if the checkpoint filesystem
was not accessible from the client. This is preventing some correct jobs
from running.

Now, only print a warning.

This closes #2157

Use distinct non-default port ranges for all tests.

This closes #2126.

Before, when processing an element that would end up in a late window
(when using a MergingWindowAssigner), the element would be added to the
MergingWindowSet. After determining that the window is late it would not
be removed from the MergingWindowSet. This can lead to problems with
other elements being merged into these "phantom" windows and causing
triggers to be added for empty windows.

This also fixes the same code in EvictingWindowOperator.

This closes #2173.

Before, this was trying to cast the StreamTaskState directly to a
StateHandle<DataInputView> while it should take the operator state, i.e:

StateHandle<DataInputView> stateHandle =
  (StateHandle<DataInputView>) state.getOperatorState();

This closes #2108

This closes #2058 because its an invalid pull request.

Some of the changes include:

- Remove unneeded dependencies (nano, wget)
- Remove apt lists to reduce image size
- Reduce number of layers on the docker image (best docker practice)
- Remove useless variables and base the code in generic ones e.g.
FLINK_HOME
- Change the default JDK from oracle to openjdk-8-jre-headless, based on
two reasons:

1. You cannot legally repackage the oracle jdk in docker images
2. The open-jdk headless is more appropriate for a server image (no GUI stuff)

- Return port assignation to the standard FLINK one:

Variable: docker-flink -> flink

taskmanager.rpc.port: 6121 -> 6122
taskmanager.data.port: 6122 -> 6121
jobmanager.web.port: 8080 -> 8081

This closes #2176

Adds a GraphAnalytic to replace the checksumHashCode Java and Scala
utility functions.

This closes #2188

A delegating GraphAlgorithm wraps a GraphAlgorithm result with a
delegating proxy object. The delegated object can be replaced when the
same algorithm is run on the same input with a mergeable configuration.
This allows algorithms to be composed of implicitly reusable algorithms
without publicly sharing intermediate DataSets.

This closes #2032

This closes #2191

This removes the leaderUpdated method from the framework. Further it
lets the RM client thread communicate directly with the
ResourceManager actor. This is fine since the two are always spawned
together. Failures of the ResourceManager actor will lead to dropped
messages of the RM client thread. Failures of the RM client thread will
inform the JobManager.

The leaderUpdated() method was used to signal the ResourceManager
framework that a new leader was elected. However, the method was not
always called when the leader changed, only when a new leader was
elected. This dropped all messages from the async Yarn RM client
thread (YarnResourceManagerCallbackHandler) for the time that the old
leader had failed and no new leader had been elected. The Yarn RM client
thread used leader tagged messages to communicate with the main Flink
ResourceManager actor.

This closes #2190

This fixes a regression of 0483ba58
where the classpath would start off with the application master's home
dir.

- always ship the lib folder
- properly setup the classpath from the supplied ship files
- cleanup deploy() method of YarnClusterDescriptor
- add test case

This closes #2187

This closes #2168

- lazily initialize ActorSystem
- make sure it is not created before job execution
- print connection information on the CLI

This closes #2189

- user specifies no parallelism
  -> parallelism is adjusted to #taskSlots * #nodes.

- user specifies parallelism but no #taskSlots or too few slots
  -> #taskSlots are set such that they meet the parallelism

-formatting
-don't use implicit conversions
-fixed ObjectName for mapped bufferpool