Update Hadoop versions and replace Hadoop 2.3 with 2.8 in build and
continuous integration scripts. flink-yarn-tests can now be enabled for
all supported Hadoop versions and is made non-optional.

This closes #3832