This closes #11983

Some noise reduction.

 Brief change log:
  - remove the scala version suffix for connector-hive and queryable-state-client-java
  - add the scala dependencies for table-api-scala and flink-sql-connectors
  - correct the scala-free check logic in `verify_scala_suffixes.sh`

The mechanism uses child-first classloading and creates classloaders from jars that are discovered
from a directory hierarchy.

This also adadpts update_branch_version.sh to correctly update versions
that have a prefix.

This closes #4801.

The flip6 build profile only runs the Flip-6 related test cases. Moreover,
all Flip-6 related test cases are excluded when not running the flip6 build
profile. This should reduce testing time when adding more and more Flip-6
test cases.

Include flink-test-utils-junit in all submodules to make the Category marker interfaces Flip6 and OldAndFlip6 available

This closes #4889.

Disable it in most modules.

Shading the ZooKeeper dependency makes sure that this specific version of
ZooKeeper is used by the Flink runtime module. The ZooKeeper version is
sensitive, because we depend on bug fixes in later ZooKeeper versions
for Flink's high availability.

This prevents situations where for example a set of added dependencies (for
example transtive dependencies of Hadoop) cause a different ZooKeeper version
to be in the classpath and be loaded.

This commit also removes the 'flink-shaded-curator' module, which was originally
created to shade guava within curator, but is now obsolete, because newer
versions of curator shade guava already.

This closes #4923.

We can do this now that we dropped support for Scala 2.10.

This closes #4807

This also makes them optional in flink-runtime, which is enabled by the
previous changes to only use Hadoop dependencies if they are available.

This also requires adding a few explicit dependencies in other modules
because they were using transitive dependencies of the Hadoop deps. The
most common dependency there is, ha!, commons-io.

This closes #4503.

This closes #4453.

This closes #4295

This commit removes the use of Java serialization for serializing the
enum class and constants in the Java EnumSerializerConfigSnapshot and
Scala ScalaEnumSerializerConfigSnapshot.

This closes #4147.

The added PojoSerializerUpgradeTest tests the state migration behaviour when the
underlying pojo type changes and one tries to recover from old state. Currently
not all tests could be activated, because there still some pending issues to be
fixed first. We should arm these tests once the issues have been fixed.

[FLINK-6808] Implement snapshotConfiguration/ensureCompatibility for CoGroupedStreams.UnionSerializer

This closes #4052.

Use scala.binary.version as defined in the parent POM and remove the
script to swap scala version identifiers.

This closes #3800

This closes #3619.