- 13 5月, 2020 10 次提交
-
-
由 Dawid Wysakowicz 提交于
-
由 Arvid Heise 提交于
-
由 Arvid Heise 提交于
-
由 Arvid Heise 提交于
-
由 Arvid Heise 提交于
When a downstream tasks starts snapshotting input before the upstream task finished snapshotting output, it may happen that a buffer is neither snapshotted upstream or downstream. This commit replicates the logic of RemoteInputChannel to also store these buffers.
-
由 Arvid Heise 提交于
Priority event listener currently does not snapshot any buffers and causes additional synchronization points. We should re-enable/re-evaluate this concept once a proper threading model has been established on input side.
-
由 Piotr Nowojski 提交于
Released deserializers cause NPE in StreamTaskNetworkInput#prepareSnapshot.
-
由 Piotr Nowojski 提交于
[FLINK-17640][network] Temporarily disable unstable RecoveredInputChannelTest#testConcurrentReadStateAndProcessAndRelease
-
由 Kostas Kloudas 提交于
This PR renames the entrypoint but DOES NOT change the scripts as this would be a breaking change for already existing deployments and such a change would require a more thorough and more visible discussion with the community. This closes #12087.
-
由 Kostas Kloudas 提交于
-
- 12 5月, 2020 30 次提交
-
-
由 Aljoscha Krettek 提交于
-
由 wangyang0918 提交于
Currently, for every job execution, the flink lib jars are uploaded to hdfs and then register as Yarn local resources. This PR introduces the yarn.provided.lib.dirs YARN option which allows users to specify HDFS paths to dirs that contain files to be shared across applications, e.g. the FLINK-DIST jar. This makes job submission more efficient as it allows for two optimizations: * Use pre-uploaded flink binary to avoid uploading of flink system jars * By default, the resource visibility for the shared libs is set to PUBLIC so that they will be downloaded only once and shared for all tasks running on the same node. This will make launching a container faster. A command that leverages this feature looks like the following: ./bin/flink run -m yarn-cluster -d \ -yD yarn.provided.lib.dirs="hdfs://$namenode_address/flink-lib/flink-1.11-SNAPSHOT/lib;hdfs://$namenode_address/flink-lib/flink-1.11-SNAPSHOT/plugins" \ examples/streaming/WindowJoin.jar This closes #12040.
-
由 wangyang0918 提交于
-
由 Roman Khachatryan 提交于
Motivation: current behavior of forcing savepoints causes excess of max-concurrent-checkpoints limit. Which violates current Unaligned Checkpoints (UC) assumption of a single concurrent barrier. Changes: 1. Remove periodicTriggeringSuspended. Instead, if periodic request can't be executed now is ignored. 2. Check queue before executing a new request (not after). 3. Execute queued request when pending checkpoint is completed (instead of resuming timer). 4. Don't set CheckpointRequest.force in UC mode. 5. Change requests queue to PriorityQueue to prioritize savepoints. Limit its size. 6. For savepoints, if running in UC mode and limit is reached then enqueue request instead of forcing it 7. Don't throw trigger exceptions. CheckpointFailureManager ignores them
-
由 Roman Khachatryan 提交于
Pre-requsite refactoring to change decision logic
-
由 Roman Khachatryan 提交于
-
由 Jörn Kottmann 提交于
This also adds a test case to ensure this works as expected.
-
由 Guru Prasad 提交于
-
由 Zhenghua Gao 提交于
This closes #11892
-
由 Dawid Wysakowicz 提交于
This closes #11981
-
由 Dawid Wysakowicz 提交于
-
由 Aljoscha Krettek 提交于
Co-authored-by: NDawid Wysakowicz <dwysakowicz@apache.org>
-
由 Dawid Wysakowicz 提交于
parser
-
由 Jingsong Lee 提交于
This closes #12091
-
由 Leonard Xu 提交于
-
由 Dawid Wysakowicz 提交于
This PR adds a way to emit multiple records from KafkaDeserializationSchema. This is possible through a collector, which will buffer deserialized records in a queue and then emit all records atomically. The queue is reused for all incoming Kafka records to minimize creating new objects on the hot path.
-
由 Dawid Wysakowicz 提交于
-
由 Robert Metzger 提交于
This closes #12016
-
由 Yangze Guo 提交于
[FLINK-17536][core] Change the config option for slot max limitation to slotmanager.number-of-slots.max This closes #12067.
-
由 JingsongLi 提交于
-
由 Jingsong Lee 提交于
This closes #12004
-
由 yuzhao.cyz 提交于
[FLINK-17577][table-common] SinkFormat#createSinkFormat should use DynamicTableSink.Context as parameter This closes #12039
-
由 Shuiqiang Chen 提交于
[FLINK-17609][python] Execute the script directly when user specified the entry script with "-py" (#12079)
-
由 vthinkxie 提交于
This closes #11731.
-
由 Zhijiang 提交于
[FLINK-16536][network][checkpointing] Implement InputChannel state recovery for unaligned checkpoint During recovery process for unaligned checkpoint, the input channel state should also be recovered besides with existing operator states. We considered three guarantees during the implementation: 1. Make input recovery happen after the output recovery for providing more floating buffers on output side firstly. 2. Make partition request happen after input recovery for avoiding new data overtaking the previous state data. 3. Introduce a dedicated single IO executor for unspilling the channel state one by one, to avoid potential random IO. This closes #11687.
-
由 Zhijiang 提交于
-
由 Zhijiang 提交于
-
由 Zhijiang 提交于
-
由 Zhijiang 提交于
[hotfix][network] Fix useless backlog value in BufferAndAvailability returned by RemoteInputChannel#getNextBuffer
-
由 Leonard Xu 提交于
This closes #11755
-