This closes #996

This closes #991.

This closes #1063.

  - check for memory availability before probing
  - correctly compute memory required for recursive build fast path
  - remove all temp files properly

- also adapt config.sh to use the same cross-platform compatible mechanism

This closes #1049.

This is needed because the Hadoop IF/OF's are using Hadoop's FileSystem stack, which is using
the security credentials passed in the JobConf / Job class in the getSplits() method.

Note that access to secured Hadoop 1.x using Hadoop IF/OF's is not possible with this change.
This limitation is due to missing methods in the old APIs.

- Add some comments & change dependency scope to test

This closes #1056.

- Address PR comments

This closes #1051.

- Address PR comments

This closes #1050.

  - Expand the verification boundary for random sampler test
  - Increase the number of times we sample

This closes #1047

[FLINK-2563] [gelly] extended the run() method of GraphAlgorithm interface to return an arbitrary type

This closes #1042

  - disabled avro-maven-plugin within Eclipse via <pluginManagement> ... <lifecyleMappingMetaData>

This closes #1041

[gelly] made the number of vertices an optional parameter of PageRank; added the edge weight initialization to the library methods

[FLINK-2451] [gelly] re-organized tests; compare with collect() instead of temp files where possible

This closes #1000

[FLINK-2451] [gelly] removed redundant examples; added comments describing which gelly method each example illustrates.

 - enabled .declareStream() and connect via stream name
 - enabled multiplt output streams
 - added .split() / .select() / strip pattern
 - added helpers in new package utils
 - adapted and extended JUnit tests
 - adapted examples
some minor improvements (FlinkClient, integration of Tuple0)

This closes #1011

[FLINK-1901] [core] enable sample with fixed size on the whole dataset.

[FLINK-1901] [core] add more comments for RandomSamplerTest.

[FLINK-1901] [core] refactor PoissonSampler output Iterator.

[FLINK-1901] [core] move sample/sampleWithSize operator to DataSetUtils.

Adds notes for commons-math3 to LICENSE and NOTICE file

This closes #949.

The DataExchangeMode of union nodes was not respected when translating an OptimizedPlan
to a JobGraph. This could result in deadlocks, when a branched data flow was closed.

Union nodes with a batch exchange will propagate their exchange mode to all inputs of
their inputs when the JobGraph is generated.

This closes #1036

This decouples the building of the StreamGraph from the API methods.
Before the methods would build the StreamGraph as they go. Now the API
methods build a hierachy of StreamTransformation nodes. From these a
StreamGraph is generated upon execution.

This also introduces some API breaking changes:

 - The result of methods that create sinks is now DataStreamSink instead
   of DataStream
 - Iterations cannot have feedback edges with differing parallelism
 - "Preserve partitioning" is not the default for feedback edges. The
   previous option for this is removed.
 - You can close an iteration several times, no need for a union.
 - Strict checking of whether partitioning and parallelism work
   together. I.e. if upstream and downstream parallelism don't match it
   is not legal to have Forward partitioning anymore. This was not very
   transparent: When you went from low parallelism to high dop some
   downstream  operators would never get any input. When you went from high
   parallelism to low dop you would get skew in the downstream operators
   because all elements that would be forwarded to an operator that is not
   "there" go to another operator. This requires insertion of global()
   or rebalance() in some places. For example with most sources which
   have parallelism one.

This also makes StreamExecutionEnvironment.execute() behave consistently
across different execution environments (local, remote ...): The list of
operators to be executed are cleared after execute is called.

[tests] Replaces Scala mixins for the Job/TaskManager with classes which extend the respective classes. Adds proper logger registration for sub-classes of FlinkUntypedActor.

This closes #1022

This closes #1004

This closes #977

[tests] Hardens TaskManagerRegistrationTest.testTaskManagerResumesConnectAfterJobManagerFailure test case by checking testing actor's complete mailbox for a Terminated message