1. 22 9月, 2014 12 次提交
    • A
      Rewrite the Scala API as (somewhat) thin Layer on Java API · b8131fa7
      Aljoscha Krettek 提交于
      Don't bother looking at the diff, this is almost a complete rewrite of
      the previous Scala API. This uses all the work put into the Java API,
      such as TypeInformation, the serializers and comparators and the
      operators.
      
      The Scala DataSet and ExecutionEnvironment wrap their respective Java
      equivalents. TypeInformation is generated by a macro that uses
      TypeInformationGen and other macro support classes. The Java
      TypeExtractor is completely bypassed but the TypeInformation and
      sub-classes are created by the Scala type analyzer. There is special
      support for Scala Tuples in the form of ScalaTupleTypeInfo,
      ScalaTupleSerializer, and ScalaTupleComparator.
      
      This also adds tests to flink-scala that are ports of the tests in
      flink-java.
      
      There are not yet any Scala specific tests in flink-tests. All the
      scala example ITCases are commented out, as well as the examples
      themselves. Those will be uncommented once the examples are ported. This
      will happen in separate commits.
      b8131fa7
    • A
      Move RichFunctions to api.common package · 568dff12
      Aljoscha Krettek 提交于
      They were in api.java before but they can be used from Scala,
      just like the regular functions.
      568dff12
    • A
      Refactor Tuple wrapping/unwrapping in Join and CoGroup · a32890ae
      Aljoscha Krettek 提交于
      Previously both sides were always wrapped/unwrapped, even if one side
      was a tuple type. This was not compatible with the Scala tuples and is
      also not necessary anymore because of the GenericPairComparator.
      a32890ae
    • A
      Change ObjectArrayTypeInfo.getInfoFor to use componentInfo.isTupleType() · 16dad15f
      Aljoscha Krettek 提交于
      This is necessary because the previous test didn't work for Scala tuples.
      16dad15f
    • A
      Make DistinctOperator and Keys use TupleComparatorBase · 785f2c4f
      Aljoscha Krettek 提交于
      Was TupleComparator before which does not work when used from the Scala API.
      785f2c4f
    • A
      Move the Iteration "Operators" to the operators package · dee54cb5
      Aljoscha Krettek 提交于
      I'm doing this in preparation for the Scala API rework, since then I can just copy over
      the operators package and use them for Scala.
      dee54cb5
    • A
      Perform TypeExtraction outside of Java Operators · 6bbe2a05
      Aljoscha Krettek 提交于
      Before, FlatMapOperator, GroupReduceOperator, MapOperator, and MapPartitionOperator
      performed the Type extraction themselves while the other Operators had TypeInformation
      parameters. Now the are all unified, which makes it possible to use them from the
      Scala API.
      
      Also Key extraction for selector functions is moved outside of Keys.java
      6bbe2a05
    • A
      Refactor TupleTypeInfo and add GenericPairComparator · 57b8e66a
      Aljoscha Krettek 提交于
      Now we have TupleTypeInfoBase, TupleSerializerBase, and TupleComparatorBase. They
      are now super classes of TupleTypeInfo and the others.
      
      Also rename compare on DataInputView to compareSerialized because Scala
      cannot distinguish between the to compare methods for some reason.
      
      This change is necessary for allowing the Scala API to reuse most of the
      functionality.
      
      The GenericPairComparator uses the new extractKeys method of
      TypeComparator to compare values of any type. This replaces
      TuplePairComparator and some other special-case pair comparators. This
      is preparatory work for enabling support for Scala Tuples and POJO
      comparators.
      57b8e66a
    • F
      [FLINK-1038] added RemoteCollectorOutputFormat · 7f946cee
      Fabian Tschirschnitz 提交于
      This closes #94
      7f946cee
    • S
      [FLINK-1096] Correction to histogram accumulator · 9463e27b
      Sebastian Kruse 提交于
      This closes #117
      * each key is associated with the number of times it was inserted into the accumulator
      * backed histogram with a tree map to present the entries sorted by key
      9463e27b
    • T
      [FLINK-1077] Improved error message if test fails · e17575b5
      twalthr 提交于
      This closes #120
      e17575b5
    • S
      [FLINK-953] Remove fake tail from iterations · 78618491
      Stephan Ewen 提交于
      This closes #124
      78618491
  2. 21 9月, 2014 28 次提交