- 26 7月, 2016 6 次提交
-
-
由 zentol 提交于
This closes #2286
-
由 Aljoscha Krettek 提交于
This closes #2278.
-
由 Ufuk Celebi 提交于
Savepoints were previously persisted without any meta data using default Java serialization of `CompletedCheckpoint`. This commit introduces a savepoint interface with version-specific serializers and stores savepoints with meta data. Savepoints expose a version number and a Collection<TaskState> for savepoint restore. Currently, there is only one savepoint version: SavepointV0 (Flink 1.1): This is the current savepoint version, which holds a reference to the Checkpoint task state collection, but is serialized with a custom serializater not relying on default Java serialization. Therefore, it should not happen again that we need to stick to certain classes in future Flink versions. The savepoints are stored in `FsSavepointStore` with the following format: MagicNumber SavepointVersion Savepoint - MagicNumber => int - SavepointVersion => int (returned by Savepoint#getVersion()) - Savepoint => bytes (serialized via version-specific SavepointSerializer) The header is minimal (magic number, version). All savepoint-specific meta data can be moved to the savepoint itself. This is also were we would have to add new meta data in future versions, allowing us to differentiate between different savepoint versions when we change the serialization stack. All savepoint related classes have been moved from checkpoint to a new sub package `checkpoint.savepoint`. This closes #2194.
-
由 twalthr 提交于
-
由 smarthi 提交于
This closes #2162.
-
由 Greg Hogan 提交于
The default maximum akka transfer size is 10 MB. This commit reduces the number of generator blocks from 2^20 to 2^15 which removes the limit on graph size. The original limit of one millions blocks was intended to future-proof scalability. This is a temporary fix as graph generation will be reworked in FLINK-3997.
-
- 25 7月, 2016 7 次提交
-
-
由 erli ding 提交于
This closes #2233
-
由 Jark Wu 提交于
This closes #2280.
-
由 twalthr 提交于
This closes #2292.
-
由 twalthr 提交于
This closes #2283.
-
由 Flavio Pompermaier 提交于
-
由 Ufuk Celebi 提交于
The `BlobServer` acts as a local cache for uploaded BLOBs. The life-cycle of each BLOB is bound to the life-cycle of the `BlobServer`. If the BlobServer shuts down (on JobManager shut down), all local files will be removed. With HA, BLOBs are persisted to another file system (e.g. HDFS) via the `BlobStore` in order to have BLOBs available after a JobManager failure (or shut down). These BLOBs are only allowed to be removed when the job that requires them enters a globally terminal state (`FINISHED`, `CANCELLED`, `FAILED`). This commit removes the `BlobStore` clean up call from the `BlobServer` shutdown. The `BlobStore` files will only be cleaned up via the `BlobLibraryCacheManager`'s' clean up task (periodically or on BlobLibraryCacheManager shutdown). This means that there is a chance that BLOBs will linger around after the job has terminated, if the job manager fails before the clean up. This closes #2256.
-
由 zentol 提交于
Prevents the CheckpointCommitter from failing a job, if either commitCheckpoint() or isCheckpointCommitter() failed. Instead, we will try again on the next notify(). This closes #2287
-
- 23 7月, 2016 2 次提交
-
-
由 Greg Hogan 提交于
The user must now specify the ID type as "integer" or "string" when reading a graph from a CSV file. This closes #2250
-
由 Greg Hogan 提交于
This closes #2248
-
- 22 7月, 2016 2 次提交
-
-
由 Ufuk Celebi 提交于
Suspended jobs were leading to shutdown of the checkpoint coordinator and hence removal of checkpoint state. For standalone recovery mode this is OK as no state can be recovered anyways (unchanged in this PR). For HA though this lead to removal of checkpoint state, which we actually want to keep for recovery. We have the following behaviour now: -----------+------------+------------------- | Standalone | High Availability -----------+------------+------------------- SUSPENDED | Discard | Keep -----------+------------+------------------- FINISHED/ | Discard | Discard FAILED/ | | CANCELED | | -----------+------------+------------------- This closes #2276.
-
由 Till Rohrmann 提交于
This closes #2284.
-
- 21 7月, 2016 7 次提交
-
-
由 Aljoscha Krettek 提交于
This closes #2279
-
由 Till Rohrmann 提交于
This PR adds a JM metric which shows the time it took to restart a job. The time is measured between entering the JobStatus.RESTARTING and reaching the JobStatus.RUNNING state. During this time, the restarting time is continuously updated. The metric only shows the time for the last restart attempt. The metric is published in the job metric group under the name of "restartingTime". This closes #2271.
-
由 Aljoscha Krettek 提交于
-
由 Ufuk Celebi 提交于
-
由 Stefan Richter 提交于
-
由 Stefan Richter 提交于
-
由 Aljoscha Krettek 提交于
-
- 20 7月, 2016 6 次提交
-
-
由 kl0u 提交于
Parameters of some input formats that was only possible to be set through the Configuration object now have setter methods that allow the user to do so. Values set by the setters cannot be reset by the configuration object.
-
由 zentol 提交于
This closes #2238.
-
由 Jark Wu 提交于
This closes #2120.
-
由 kl0u 提交于
Integrated PR comments This closes #2259.
-
由 Stefan Richter 提交于
This closes #2249
-
由 Maximilian Michels 提交于
- change CLI message upon cluster retrieval - save JobExecutionResult for interactive executions - only print Collection size in accumulator results - remove unused helper method This closes #2264
-
- 19 7月, 2016 4 次提交
-
-
由 twalthr 提交于
This closes #2221.
-
由 卫乐 提交于
This closes #2182.
-
由 gallenvara 提交于
This closes #2078.
-
由 David Moravek 提交于
This closes #2268
-
- 18 7月, 2016 6 次提交
-
-
由 Maximilian Michels 提交于
-
由 Maximilian Michels 提交于
-
由 Ufuk Celebi 提交于
Disposing savepoints via the JobManager fails for state handles or descriptors, which contain user classes (for example custom folding state or RocksDB handles). With this change, the user can provide the job JAR when disposing a savepoint in order to use the user code class loader of that job. The JAR is optional, hence not breaking the CLI API. This closes #2083.
-
由 Ismaël Mejía 提交于
-
由 Ismaël Mejía 提交于
-
由 Ismaël Mejía 提交于
-