Skip to content

F
flink

doujutun3207 / flink
与 Fork 源项目一致

从无法访问的项目Fork

通知 24
Star 0
Fork 0
F
flink

体验新版 GitCode，发现更多精彩内容 >>

切换分支/标签
  1. 28 12月, 2020 5 次提交
  3. 27 12月, 2020 2 次提交
  5. 25 12月, 2020 13 次提交
  7. 24 12月, 2020 13 次提交
  9. 23 12月, 2020 7 次提交
    • H
      [FLINK-20640][python] Remove the type info of args (#14481) · 62a955f1
      由 HuangXingBo 提交于
      62a955f1
    • H
      [FLINK-20640][python] Use cython to optimize the performance of Python UDAF · 6ed081fc
      由 huangxingbo 提交于 
      This closes #14449.
      6ed081fc
    • T
      [FLINK-20678] Let JobManagerRunnerImpl fail if the JobMasterService fails unexpectedly · 60b0745c
      由 Till Rohrmann 提交于 
      This is a safety net which ensures that the JobManagerRunnerImpl will fail if the
JobMasterService fails unexpectedly.

This closes #14432.
      60b0745c
    • C
      [FLINK-20694][coordination] Do not rematch freed slots · 44280a44
      由 Chesnay Schepler 提交于 
      When a slot was freed the slot pool was trying to find another requirement it could potentially fulfill, using the same code paths as for newly offered slots, which also includes an increase in acquired resources.
As a result we were potentially counting a single slot as multiple resources, causing an inconsistent state. The slot pool hence believed it had enough slots to fulfill the requirements, causing it to reject slots, while the Scheduler was waiting for enough slots to arrive and the RM repeatedly trying to provide the actually required slots.

While it is not necessarily a bad thing to try an find another matching requirement, it is currently not necessary to do so, since the scheduler will use slots however it sees fit anyway. In the worst case we go through another slot allocation cycle.
As such freed slots now remain assigned to the originally matched requirement, and this mapping will only change whe the slot is explicitly reserved for a different requirement.

In the long-term it would be desirable for the slot pool to automatically adjust the matching if a slot was freed. This could prevent cases where we request new slots although we still have idling slots around that we could use, or release slots as idle although there are still outstanding requirements.
However this needs a more sophisticated approach, it has to account for cases where:
- a freed slot could not be matched to another requirement (i.e., it is truly an excess resource)
- a freed slot was matched against another profile, which should go through the same codepath as for slot reservations for a different profile
- also try to re-assign currently excess resources if the requirements are increased
      44280a44
    • C
      [FLINK-20694][coordination] Extend slot pool logging · 337f8dc3
      由 Chesnay Schepler 提交于 
      - log total acquired resources when declaring requirements, or after accepting new / releasing idle slots
- log if an offer could not be matched to any requirement
      337f8dc3
    • C
      [hotfix][coordination] Centralize skipping declaration of unchanged requirements · 4f615b2d
      由 Chesnay Schepler 提交于 
      There are multiple events where resource requirements might not actually change, and so far the user of the slot pool was responsible for checking that the requirements have actually changed to prevent unnecessary RPC calls.
Since this is something that every user should check, we might as well always do this automatically.
      4f615b2d
    • C
      [hotfix][coordination] Move logging of resource declaration to slot pool · 7fcfeb6f
      由 Chesnay Schepler 提交于
      7fcfeb6f