- 22 9月, 2014 12 次提交
-
-
由 Aljoscha Krettek 提交于
Don't bother looking at the diff, this is almost a complete rewrite of the previous Scala API. This uses all the work put into the Java API, such as TypeInformation, the serializers and comparators and the operators. The Scala DataSet and ExecutionEnvironment wrap their respective Java equivalents. TypeInformation is generated by a macro that uses TypeInformationGen and other macro support classes. The Java TypeExtractor is completely bypassed but the TypeInformation and sub-classes are created by the Scala type analyzer. There is special support for Scala Tuples in the form of ScalaTupleTypeInfo, ScalaTupleSerializer, and ScalaTupleComparator. This also adds tests to flink-scala that are ports of the tests in flink-java. There are not yet any Scala specific tests in flink-tests. All the scala example ITCases are commented out, as well as the examples themselves. Those will be uncommented once the examples are ported. This will happen in separate commits.
-
由 Aljoscha Krettek 提交于
They were in api.java before but they can be used from Scala, just like the regular functions.
-
由 Aljoscha Krettek 提交于
Previously both sides were always wrapped/unwrapped, even if one side was a tuple type. This was not compatible with the Scala tuples and is also not necessary anymore because of the GenericPairComparator.
-
由 Aljoscha Krettek 提交于
This is necessary because the previous test didn't work for Scala tuples.
-
由 Aljoscha Krettek 提交于
Was TupleComparator before which does not work when used from the Scala API.
-
由 Aljoscha Krettek 提交于
I'm doing this in preparation for the Scala API rework, since then I can just copy over the operators package and use them for Scala.
-
由 Aljoscha Krettek 提交于
Before, FlatMapOperator, GroupReduceOperator, MapOperator, and MapPartitionOperator performed the Type extraction themselves while the other Operators had TypeInformation parameters. Now the are all unified, which makes it possible to use them from the Scala API. Also Key extraction for selector functions is moved outside of Keys.java
-
由 Aljoscha Krettek 提交于
Now we have TupleTypeInfoBase, TupleSerializerBase, and TupleComparatorBase. They are now super classes of TupleTypeInfo and the others. Also rename compare on DataInputView to compareSerialized because Scala cannot distinguish between the to compare methods for some reason. This change is necessary for allowing the Scala API to reuse most of the functionality. The GenericPairComparator uses the new extractKeys method of TypeComparator to compare values of any type. This replaces TuplePairComparator and some other special-case pair comparators. This is preparatory work for enabling support for Scala Tuples and POJO comparators.
-
由 Fabian Tschirschnitz 提交于
This closes #94
-
由 Sebastian Kruse 提交于
This closes #117 * each key is associated with the number of times it was inserted into the accumulator * backed histogram with a tree map to present the entries sorted by key
-
由 twalthr 提交于
This closes #120
-
由 Stephan Ewen 提交于
This closes #124
-
- 21 9月, 2014 28 次提交
-
-
由 Stephan Ewen 提交于
Temporarily disabled streaming tests that validate jobgraphs that used to be illegal, but are now legal.
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
Small fix in integration test
-
由 Stephan Ewen 提交于
Tolerate concurrent scheduling attempts
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
Make Co-Location constraints resilient aginst out of order scheduling and depply integrate them with slot sharing Fix miscellaneous checkstyle errors/warnings
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
Improved distribution of IDs. Previous implementation lost bits due to double-to-long multiplication and rounding.
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
Fix comparisons (null pointer safe) in JobManagerITCase
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
Add proper locality of scheduling tracking to scheduler. Add local scheduling to slot sharing groups.
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
-
由 Stephan Ewen 提交于
Add tests for job event classes
-
由 Stephan Ewen 提交于
-