[FLINK-3459] [build] Fix conflicting dependencies commons-collections,...

[FLINK-3459] [build] Fix conflicting dependencies commons-collections, commons-beanutils and commons-beanutils-core

The Hadoop dependencies have a dependency on commons-configuration which pulls in transitively the commons-collection, commons-beanutils and common-beanutils-core depedencies. Commons-beanutils and commons-collection contain classes which live in the same namespace. They are also binary compatible but not binary identical. This is a problem for the sbt assembly plugin which checks for binary identity. In order to solve the problem, we bump the commons-configuration version to 1.7 so that only commons-beanutils is pulled in. This is necessary, because the transitive promotion of dependencies of the shade plugin only excludes the commons-beanutils dependency only from the directly depending dependency. All parent dependencies won't have the exclusion. This is a problem for SBT which will pull the dependency as part of one of the parents, then. Moreover, we replace commons-beanutils by commons-beanutils-bean-collections which contains only the non-conflicting classes wrt commons-collections.

This closes #1682.

flink-core/pom.xml

@@ -40,12 +40,6 @@ under the License.
 			<artifactId>flink-annotations</artifactId>
 			<version>${project.version}</version>
 		</dependency>
-		
-		<dependency>
-			<groupId>commons-collections</groupId>
-			<artifactId>commons-collections</artifactId>
-			<!-- managed version -->
-		</dependency>
 
 		<dependency>
 			<groupId>com.esotericsoftware.kryo</groupId>

flink-shaded-hadoop/flink-shaded-hadoop2/pom.xml

@@ -141,9 +141,21 @@ under the License.
 					<groupId>com.sun.jersey.contribs</groupId>
 					<artifactId>jersey-guice</artifactId>
 				</exclusion>
+				<!--We have to exclude beanutils because it is overlapping with commons-collections-->
+				<exclusion>
+					<groupId>commons-beanutils</groupId>
+					<artifactId>commons-beanutils</artifactId>
+				</exclusion>
 			</exclusions>
 		</dependency>
 
+		<!--Here we include all the missing parts of beanutils which are not included in the
+		commons-collections dependency-->
+		<dependency>
+			<groupId>commons-beanutils</groupId>
+			<artifactId>commons-beanutils-bean-collections</artifactId>
+		</dependency>
+
 		<dependency>
 			<groupId>org.apache.hadoop</groupId>
 			<artifactId>hadoop-hdfs</artifactId>

pom.xml

@@ -207,11 +207,25 @@ under the License.
 				<version>2.4</version>
 			</dependency>
 
-			<!-- common-collections is used by us and by hadoop, so we need to define a common version -->
+			<!-- common-beanutils-bean-collections is used by flink-shaded-hadoop2 -->
 			<dependency>
-				<groupId>commons-collections</groupId>
-				<artifactId>commons-collections</artifactId>
-				<version>3.2.2</version>
+				<groupId>commons-beanutils</groupId>
+				<artifactId>commons-beanutils-bean-collections</artifactId>
+				<version>1.8.3</version>
+			</dependency>
+
+			<!--We have to bump the commons-configuration to version 1.7 because Hadoop uses per
+			default 1.6. This version has the problem that it depends on commons-beanutils-core and
+			commons-digester. Commons-digester depends on commons-beanutils. Both dependencies are
+			contains classes of commons-collections. Since the dependency reduced pom does not
+			exclude commons-beanutils from commons-configuration, sbt would pull it in again. The
+			solution is setting the version of commons-configuration to 1.7 which also depends on
+			common-beanutils. Consequently, the dependency reduced pom will also contain an
+			exclusion for commons-beanutils for commons-configuration. -->
+			<dependency>
+				<groupId>commons-configuration</groupId>
+				<artifactId>commons-configuration</artifactId>
+				<version>1.7</version>
 			</dependency>
 
 			<dependency>