未验证 提交 35c5a238 编写于 作者: 片刻小哥哥's avatar 片刻小哥哥 提交者: GitHub

Merge pull request #42 from apachecn/feature/flink_1.7_doc_zh_18

18 完成
# Pre-defined Timestamp Extractors / Watermark Emitters
# Pre-defined Timestamp Extractors / Watermark Emitters 预定时间戳提取器/水印发射器
As described in [timestamps and watermark handling](//ci.apache.org/projects/flink/flink-docs-release-1.7/dev/event_timestamps_watermarks.html), Flink provides abstractions that allow the programmer to assign their own timestamps and emit their own watermarks. More specifically, one can do so by implementing one of the `AssignerWithPeriodicWatermarks` and `AssignerWithPunctuatedWatermarks` interfaces, depending on the use case. In a nutshell, the first will emit watermarks periodically, while the second does so based on some property of the incoming records, e.g. whenever a special element is encountered in the stream.
[时间戳和水印处理](//ci.apache.org/projects/flink/flink-docs-release-1.7/dev/event_timestamps_watermarks.html)中所述,flink提供了允许程序员分配自己的时间戳并发出自己的水印的抽象。更具体地,可以通过实现`AssignerWithPeriodicWatermarks具有周期性水印的分配器``AssignerWithPunctuatedWatermarks 具有标点符号的水印`接口中的一个来实现,这取决于使用情况。简而言之,第一将周期性地发射水印,而第二则基于输入记录的某些属性,例如每当在流中遇到特殊元素时。
In order to further ease the programming effort for such tasks, Flink comes with some pre-implemented timestamp assigners. This section provides a list of them. Apart from their out-of-the-box functionality, their implementation can serve as an example for custom implementations.
为了进一步简化此类任务的编程工作,FLink附带了一些预先实现的时间戳分配器。本节提供了它们的列表。除了它们的开箱即用功能之外,它们的实现可以作为定制实现的示例。
### **Assigners with ascending timestamps**
### **Assigners with ascending timestamps 具有升序时间戳的分配方**
The simplest special case for _periodic_ watermark generation is the case where timestamps seen by a given source task occur in ascending order. In that case, the current timestamp can always act as a watermark, because no earlier timestamps will arrive.
生成周期水印最简单的特例是给定源任务所看到的时间戳按升序发生的情况。在这种情况下,当前时间戳始终可以充当水印,因为不会有更早的时间戳到达。
Note that it is only necessary that timestamps are ascending _per parallel data source task_. For example, if in a specific setup one Kafka partition is read by one parallel data source instance, then it is only necessary that timestamps are ascending within each Kafka partition. Flink’s watermark merging mechanism will generate correct watermarks whenever parallel streams are shuffled, unioned, connected, or merged.
请注意,仅有必要时间戳是升序 _per parallel data source task_。例如,如果在特定的设置中,一个KAFKA分区由一个并行数据源实例读取,则仅有必要在每个Kafka分区中递增时间戳。每当并行流被混洗、统一、连接或合并时,flink的水印合并机制将产生正确的水印。
......@@ -39,9 +40,9 @@ val withTimestampsAndWatermarks = stream.assignAscendingTimestamps( _.getCreatio
### **Assigners allowing a fixed amount of lateness**
### **Assigners allowing a fixed amount of lateness 指定人允许一定数量的迟到 **
Another example of periodic watermark generation is when the watermark lags behind the maximum (event-time) timestamp seen in the stream by a fixed amount of time. This case covers scenarios where the maximum lateness that can be encountered in a stream is known in advance, e.g. when creating a custom source containing elements with timestamps spread within a fixed period of time for testing. For these cases, Flink provides the `BoundedOutOfOrdernessTimestampExtractor` which takes as an argument the `maxOutOfOrderness`, i.e. the maximum amount of time an element is allowed to be late before being ignored when computing the final result for the given window. Lateness corresponds to the result of `t - t_w`, where `t` is the (event-time) timestamp of an element, and `t_w` that of the previous watermark. If `lateness > 0` then the element is considered late and is, by default, ignored when computing the result of the job for its corresponding window. See the documentation about [allowed lateness](//ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/operators/windows.html#allowed-lateness) for more information about working with late elements.
周期性水印生成的另一个示例是当水印滞后于在流中看到的最大(事件时间)时间戳之后的预定时间量时。这种情况包括预先知道在流中可能遇到的最大延迟的情形,例如,当创建包含具有在用于测试的固定时间段内扩展的时间戳的元素的自定义源时。对于这些情况,flink提供了`BoundedOutOfOrdernessTimestampExtractor` ,该“TimeStampExtractor”作为“MaxouToForested”的参数,即允许在计算给定窗口的最终结果时忽略元素之前的最大时间量。迟到对应于“t-t_w”的结果,其中“t”是元素的(事件时间)时间戳,而“t_w”是前一水印的(event-time)时间戳。如果“LAYLE>0”则该元素被认为是延迟的,并且默认情况下在计算其相应窗口的作业结果时被忽略。有关使用延迟元素的详细信息,请参阅有关[允许的迟到](//ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/operators/windows.html#allowed-lateness)的文档。
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册