1. 23 5月, 2018 15 次提交
  2. 17 5月, 2018 1 次提交
    • S
      blockjob: do not cancel timer in resume · 4c7e813c
      Stefan Hajnoczi 提交于
      Currently the timer is cancelled and the block job is entered by
      block_job_resume().  This behavior causes drain to run extra blockjob
      iterations when the job was sleeping due to the ratelimit.
      
      This patch leaves the job asleep when block_job_resume() is called.
      Jobs can still be forcibly woken up using block_job_enter(), which is
      used to cancel jobs.
      
      After this patch drain no longer runs extra blockjob iterations.  This
      is the expected behavior that qemu-iotests 185 used to rely on.  We
      temporarily changed the 185 test output to make it pass for the QEMU
      2.12 release but now it's time to address this issue.
      
      Cc: QingFeng Hao <haoqf@linux.vnet.ibm.com>
      Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
      Reviewed-by: NEric Blake <eblake@redhat.com>
      Reviewed-by: NQingFeng Hao <haoqf@linux.vnet.ibm.com>
      Message-id: 20180508135436.30140-3-stefanha@redhat.com
      Reviewed-by: NJeff Cody <jcody@redhat.com>
      Signed-off-by: NJeff Cody <jcody@redhat.com>
      4c7e813c
  3. 15 5月, 2018 6 次提交
  4. 10 5月, 2018 1 次提交
  5. 03 4月, 2018 2 次提交
    • M
      blockjob: use qapi enum helpers · 604343ce
      Marc-André Lureau 提交于
      QAPI generator provide #define helpers for looking up enum string.
      Signed-off-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
      Reviewed-by: NJohn Snow <jsnow@redhat.com>
      Message-id: 20180327153011.29569-1-marcandre.lureau@redhat.com
      Signed-off-by: NJeff Cody <jcody@redhat.com>
      604343ce
    • M
      blockjob: leak fix, remove from txn when failing early · a865cebb
      Marc-André Lureau 提交于
      This fixes leaks found by ASAN such as:
        GTESTER tests/test-blockjob
      =================================================================
      ==31442==ERROR: LeakSanitizer: detected memory leaks
      
      Direct leak of 24 byte(s) in 1 object(s) allocated from:
          #0 0x7f88483cba38 in __interceptor_calloc (/lib64/libasan.so.4+0xdea38)
          #1 0x7f8845e1bd77 in g_malloc0 ../glib/gmem.c:129
          #2 0x7f8845e1c04b in g_malloc0_n ../glib/gmem.c:360
          #3 0x5584d2732498 in block_job_txn_new /home/elmarco/src/qemu/blockjob.c:172
          #4 0x5584d2739b28 in block_job_create /home/elmarco/src/qemu/blockjob.c:973
          #5 0x5584d270ae31 in mk_job /home/elmarco/src/qemu/tests/test-blockjob.c:34
          #6 0x5584d270b1c1 in do_test_id /home/elmarco/src/qemu/tests/test-blockjob.c:57
          #7 0x5584d270b65c in test_job_ids /home/elmarco/src/qemu/tests/test-blockjob.c:118
          #8 0x7f8845e40b69 in test_case_run ../glib/gtestutils.c:2255
          #9 0x7f8845e40f29 in g_test_run_suite_internal ../glib/gtestutils.c:2339
          #10 0x7f8845e40fd2 in g_test_run_suite_internal ../glib/gtestutils.c:2351
          #11 0x7f8845e411e9 in g_test_run_suite ../glib/gtestutils.c:2426
          #12 0x7f8845e3fe72 in g_test_run ../glib/gtestutils.c:1692
          #13 0x5584d270d6e2 in main /home/elmarco/src/qemu/tests/test-blockjob.c:377
          #14 0x7f8843641f29 in __libc_start_main (/lib64/libc.so.6+0x20f29)
      
      Add an assert to make sure that the job doesn't have associated txn before free().
      
      [Jeff Cody: N.B., used updated patch provided by John Snow]
      Signed-off-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
      Signed-off-by: NJeff Cody <jcody@redhat.com>
      a865cebb
  6. 19 3月, 2018 15 次提交
    • L
      block/mirror: change the semantic of 'force' of block-job-cancel · b76e4458
      Liang Li 提交于
      When doing drive mirror to a low speed shared storage, if there was heavy
      BLK IO write workload in VM after the 'ready' event, drive mirror block job
      can't be canceled immediately, it would keep running until the heavy BLK IO
      workload stopped in the VM.
      
      Libvirt depends on the current block-job-cancel semantics, which is that
      when used without a flag after the 'ready' event, the command blocks
      until data is in sync.  However, these semantics are awkward in other
      situations, for example, people may use drive mirror for realtime
      backups while still wanting to use block live migration.  Libvirt cannot
      start a block live migration while another drive mirror is in progress,
      but the user would rather abandon the backup attempt as broken and
      proceed with the live migration than be stuck waiting for the current
      drive mirror backup to finish.
      
      The drive-mirror command already includes a 'force' flag, which libvirt
      does not use, although it documented the flag as only being useful to
      quit a job which is paused.  However, since quitting a paused job has
      the same effect as abandoning a backup in a non-paused job (namely, the
      destination file is not in sync, and the command completes immediately),
      we can just improve the documentation to make the force flag obviously
      useful.
      
      Cc: Paolo Bonzini <pbonzini@redhat.com>
      Cc: Jeff Cody <jcody@redhat.com>
      Cc: Kevin Wolf <kwolf@redhat.com>
      Cc: Max Reitz <mreitz@redhat.com>
      Cc: Eric Blake <eblake@redhat.com>
      Cc: John Snow <jsnow@redhat.com>
      Reported-by: NHuaitong Han <huanhuaitong@didichuxing.com>
      Signed-off-by: NHuaitong Han <huanhuaitong@didichuxing.com>
      Signed-off-by: NLiang Li <liliangleo@didichuxing.com>
      Signed-off-by: NJeff Cody <jcody@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      b76e4458
    • J
      blockjobs: Expose manual property · b40dacdc
      John Snow 提交于
      Expose the "manual" property via QAPI for the backup-related jobs.
      As of this commit, this allows the management API to request the
      "concluded" and "dismiss" semantics for backup jobs.
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      b40dacdc
    • J
      blockjobs: add block-job-finalize · 11b61fbc
      John Snow 提交于
      Instead of automatically transitioning from PENDING to CONCLUDED, gate
      the .prepare() and .commit() phases behind an explicit acknowledgement
      provided by the QMP monitor if auto_finalize = false has been requested.
      
      This allows us to perform graph changes in prepare and/or commit so that
      graph changes do not occur autonomously without knowledge of the
      controlling management layer.
      
      Transactions that have reached the "PENDING" state together can all be
      moved to invoke their finalization methods by issuing block_job_finalize
      to any one job in the transaction.
      
      Jobs in a transaction with mixed job->auto_finalize settings will all
      remain stuck in the "PENDING" state, as if the entire transaction was
      specified with auto_finalize = false. Jobs that specified
      auto_finalize = true, however, will still not emit the PENDING event.
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      11b61fbc
    • J
      blockjobs: add PENDING status and event · 5f241594
      John Snow 提交于
      For jobs utilizing the new manual workflow, we intend to prohibit
      them from modifying the block graph until the management layer provides
      an explicit ACK via block-job-finalize to move the process forward.
      
      To distinguish this runstate from "ready" or "waiting," we add a new
      "pending" event and status.
      
      For now, the transition from PENDING to CONCLUDED/ABORTING is automatic,
      but a future commit will add the explicit block-job-finalize step.
      
      Transitions:
      Waiting -> Pending:   Normal transition.
      Pending -> Concluded: Normal transition.
      Pending -> Aborting:  Late transactional failures and cancellations.
      
      Removed Transitions:
      Waiting -> Concluded: Jobs must go to PENDING first.
      
      Verbs:
      Cancel: Can be applied to a pending job.
      
                   +---------+
                   |UNDEFINED|
                   +--+------+
                      |
                   +--v----+
         +---------+CREATED+-----------------+
         |         +--+----+                 |
         |            |                      |
         |         +--+----+     +------+    |
         +---------+RUNNING<----->PAUSED|    |
         |         +--+-+--+     +------+    |
         |            | |                    |
         |            | +------------------+ |
         |            |                    | |
         |         +--v--+       +-------+ | |
         +---------+READY<------->STANDBY| | |
         |         +--+--+       +-------+ | |
         |            |                    | |
         |         +--v----+               | |
         +---------+WAITING<---------------+ |
         |         +--+----+                 |
         |            |                      |
         |         +--v----+                 |
         +---------+PENDING|                 |
         |         +--+----+                 |
         |            |                      |
      +--v-----+   +--v------+               |
      |ABORTING+--->CONCLUDED|               |
      +--------+   +--+------+               |
                      |                      |
                   +--v-+                    |
                   |NULL<--------------------+
                   +----+
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      5f241594
    • J
      blockjobs: add waiting status · e8af5686
      John Snow 提交于
      For jobs that are stuck waiting on others in a transaction, it would
      be nice to know that they are no longer "running" in that sense, but
      instead are waiting on other jobs in the transaction.
      
      Jobs that are "waiting" in this sense cannot be meaningfully altered
      any longer as they have left their running loop. The only meaningful
      user verb for jobs in this state is "cancel," which will cancel the
      whole transaction, too.
      
      Transitions:
      Running -> Waiting:   Normal transition.
      Ready   -> Waiting:   Normal transition.
      Waiting -> Aborting:  Transactional cancellation.
      Waiting -> Concluded: Normal transition.
      
      Removed Transitions:
      Running -> Concluded: Jobs must go to WAITING first.
      Ready   -> Concluded: Jobs must go to WAITING first.
      
      Verbs:
      Cancel: Can be applied to WAITING jobs.
      
                   +---------+
                   |UNDEFINED|
                   +--+------+
                      |
                   +--v----+
         +---------+CREATED+-----------------+
         |         +--+----+                 |
         |            |                      |
         |         +--v----+     +------+    |
         +---------+RUNNING<----->PAUSED|    |
         |         +--+-+--+     +------+    |
         |            | |                    |
         |            | +------------------+ |
         |            |                    | |
         |         +--v--+       +-------+ | |
         +---------+READY<------->STANDBY| | |
         |         +--+--+       +-------+ | |
         |            |                    | |
         |         +--v----+               | |
         +---------+WAITING<---------------+ |
         |         +--+----+                 |
         |            |                      |
      +--v-----+   +--v------+               |
      |ABORTING+--->CONCLUDED|               |
      +--------+   +--+------+               |
                      |                      |
                   +--v-+                    |
                   |NULL<--------------------+
                   +----+
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      e8af5686
    • J
      blockjobs: add prepare callback · 2da4617a
      John Snow 提交于
      Some jobs upon finalization may need to perform some work that can
      still fail. If these jobs are part of a transaction, it's important
      that these callbacks fail the entire transaction.
      
      We allow for a new callback in addition to commit/abort/clean that
      allows us the opportunity to have fairly late-breaking failures
      in the transactional process.
      
      The expected flow is:
      
      - All jobs in a transaction converge to the PENDING state,
        added in a forthcoming commit.
      - Upon being finalized, either automatically or explicitly
        by the user, jobs prepare to complete.
      - If any job fails preparation, all jobs call .abort.
      - Otherwise, they succeed and call .commit.
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      2da4617a
    • J
      blockjobs: add block_job_txn_apply function · efe4d4b7
      John Snow 提交于
      Simply apply a function transaction-wide.
      A few more uses of this in forthcoming patches.
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      efe4d4b7
    • J
      blockjobs: add commit, abort, clean helpers · 43628d93
      John Snow 提交于
      The completed_single function is getting a little mucked up with
      checking to see which callbacks exist, so let's factor them out.
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Reviewed-by: NEric Blake <eblake@redhat.com>
      Reviewed-by: NKevin Wolf <kwolf@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      43628d93
    • J
      blockjobs: ensure abort is called for cancelled jobs · 35d6b368
      John Snow 提交于
      Presently, even if a job is canceled post-completion as a result of
      a failing peer in a transaction, it will still call .commit because
      nothing has updated or changed its return code.
      
      The reason why this does not cause problems currently is because
      backup's implementation of .commit checks for cancellation itself.
      
      I'd like to simplify this contract:
      
      (1) Abort is called if the job/transaction fails
      (2) Commit is called if the job/transaction succeeds
      
      To this end: A job's return code, if 0, will be forcibly set as
      -ECANCELED if that job has already concluded. Remove the now
      redundant check in the backup job implementation.
      
      We need to check for cancellation in both block_job_completed
      AND block_job_completed_single, because jobs may be cancelled between
      those two calls; for instance in transactions. This also necessitates
      an ABORTING -> ABORTING transition to be allowed.
      
      The check in block_job_completed could be removed, but there's no
      point in starting to attempt to succeed a transaction that we know
      in advance will fail.
      
      This does NOT affect mirror jobs that are "canceled" during their
      synchronous phase. The mirror job itself forcibly sets the canceled
      property to false prior to ceding control, so such cases will invoke
      the "commit" callback.
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Reviewed-by: NEric Blake <eblake@redhat.com>
      Reviewed-by: NKevin Wolf <kwolf@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      35d6b368
    • J
      blockjobs: add block_job_dismiss · 75f71059
      John Snow 提交于
      For jobs that have reached their CONCLUDED state, prior to having their
      last reference put down (meaning jobs that have completed successfully,
      unsuccessfully, or have been canceled), allow the user to dismiss the
      job's lingering status report via block-job-dismiss.
      
      This gives management APIs the chance to conclusively determine if a job
      failed or succeeded, even if the event broadcast was missed.
      
      Note: block_job_do_dismiss and block_job_decommission happen to do
      exactly the same thing, but they're called from different semantic
      contexts, so both aliases are kept to improve readability.
      
      Note 2: Don't worry about the 0x04 flag definition for AUTO_DISMISS, she
      has a friend coming in a future patch to fill the hole where 0x02 is.
      
      Verbs:
      Dismiss: operates on CONCLUDED jobs only.
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      75f71059
    • J
      blockjobs: add NULL state · 3925cd3b
      John Snow 提交于
      Add a new state that specifically demarcates when we begin to permanently
      demolish a job after it has performed all work. This makes the transition
      explicit in the STM table and highlights conditions under which a job may
      be demolished.
      
      Alongside this state, add a new helper command "block_job_decommission",
      which transitions to the NULL state and puts down our implicit reference.
      This separates instances in the code for "block_job_unref" which merely
      undo a matching "block_job_ref" with instances intended to initiate the
      full destruction of the object.
      
      This decommission action also sets a number of fields to make sure that
      block internals or external users that are holding a reference to a job
      to see when it "finishes" are convinced that the job object is "done."
      This is necessary, for instance, to do a block_job_cancel_sync on a
      created object which will not make any progress.
      
      Now, all jobs must go through block_job_decommission prior to being
      freed, giving us start-to-finish state machine coverage for jobs.
      
      Transitions:
      Created   -> Null: Early failure event before the job is started
      Concluded -> Null: Standard transition.
      
      Verbs:
      None. This should not ever be visible to the monitor.
      
                   +---------+
                   |UNDEFINED|
                   +--+------+
                      |
                   +--v----+
         +---------+CREATED+------------------+
         |         +--+----+                  |
         |            |                       |
         |         +--v----+     +------+     |
         +---------+RUNNING<----->PAUSED|     |
         |         +--+-+--+     +------+     |
         |            | |                     |
         |            | +------------------+  |
         |            |                    |  |
         |         +--v--+       +-------+ |  |
         +---------+READY<------->STANDBY| |  |
         |         +--+--+       +-------+ |  |
         |            |                    |  |
      +--v-----+   +--v------+             |  |
      |ABORTING+--->CONCLUDED<-------------+  |
      +--------+   +--+------+                |
                      |                       |
                   +--v-+                     |
                   |NULL<---------------------+
                   +----+
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      3925cd3b
    • J
      blockjobs: add CONCLUDED state · e0cf0364
      John Snow 提交于
      add a new state "CONCLUDED" that identifies a job that has ceased all
      operations. The wording was chosen to avoid any phrasing that might
      imply success, error, or cancellation. The task has simply ceased all
      operation and can never again perform any work.
      
      ("finished", "done", and "completed" might all imply success.)
      
      Transitions:
      Running  -> Concluded: normal completion
      Ready    -> Concluded: normal completion
      Aborting -> Concluded: error and cancellations
      
      Verbs:
      None as of this commit. (a future commit adds 'dismiss')
      
                   +---------+
                   |UNDEFINED|
                   +--+------+
                      |
                   +--v----+
         +---------+CREATED|
         |         +--+----+
         |            |
         |         +--v----+     +------+
         +---------+RUNNING<----->PAUSED|
         |         +--+-+--+     +------+
         |            | |
         |            | +------------------+
         |            |                    |
         |         +--v--+       +-------+ |
         +---------+READY<------->STANDBY| |
         |         +--+--+       +-------+ |
         |            |                    |
      +--v-----+   +--v------+             |
      |ABORTING+--->CONCLUDED<-------------+
      +--------+   +---------+
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      e0cf0364
    • J
      blockjobs: add ABORTING state · 10a3fbb0
      John Snow 提交于
      Add a new state ABORTING.
      
      This makes transitions from normative states to error states explicit
      in the STM, and serves as a disambiguation for which states may complete
      normally when normal end-states (CONCLUDED) are added in future commits.
      
      Notably, Paused/Standby jobs do not transition directly to aborting,
      as they must wake up first and cooperate in their cancellation.
      
      Transitions:
      Created -> Aborting: can be cancelled (by the system)
      Running -> Aborting: can be cancelled or encounter an error
      Ready   -> Aborting: can be cancelled or encounter an error
      
      Verbs:
      None. The job must finish cleaning itself up and report its final status.
      
                   +---------+
                   |UNDEFINED|
                   +--+------+
                      |
                   +--v----+
         +---------+CREATED|
         |         +--+----+
         |            |
         |         +--v----+     +------+
         +---------+RUNNING<----->PAUSED|
         |         +--+----+     +------+
         |            |
         |         +--v--+       +-------+
         +---------+READY<------->STANDBY|
         |         +-----+       +-------+
         |
      +--v-----+
      |ABORTING|
      +--------+
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Reviewed-by: NEric Blake <eblake@redhat.com>
      Reviewed-by: NKevin Wolf <kwolf@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      10a3fbb0
    • J
      blockjobs: add block_job_verb permission table · 0ec4dfb8
      John Snow 提交于
      Which commands ("verbs") are appropriate for jobs in which state is
      also somewhat burdensome to keep track of.
      
      As of this commit, it looks rather useless, but begins to look more
      interesting the more states we add to the STM table.
      
      A recurring theme is that no verb will apply to an 'undefined' job.
      
      Further, it's not presently possible to restrict the "pause" or "resume"
      verbs any more than they are in this commit because of the asynchronous
      nature of how jobs enter the PAUSED state; justifications for some
      seemingly erroneous applications are given below.
      
      =====
      Verbs
      =====
      
      Cancel:    Any state except undefined.
      Pause:     Any state except undefined;
                 'created': Requests that the job pauses as it starts.
                 'running': Normal usage. (PAUSED)
                 'paused':  The job may be paused for internal reasons,
                            but the user may wish to force an indefinite
                            user-pause, so this is allowed.
                 'ready':   Normal usage. (STANDBY)
                 'standby': Same logic as above.
      Resume:    Any state except undefined;
                 'created': Will lift a user's pause-on-start request.
                 'running': Will lift a pause request before it takes effect.
                 'paused':  Normal usage.
                 'ready':   Will lift a pause request before it takes effect.
                 'standby': Normal usage.
      Set-speed: Any state except undefined, though ready may not be meaningful.
      Complete:  Only a 'ready' job may accept a complete request.
      
      =======
      Changes
      =======
      
      (1)
      
      To facilitate "nice" error checking, all five major block-job verb
      interfaces in blockjob.c now support an errp parameter:
      
      - block_job_user_cancel is added as a new interface.
      - block_job_user_pause gains an errp paramter
      - block_job_user_resume gains an errp parameter
      - block_job_set_speed already had an errp parameter.
      - block_job_complete already had an errp parameter.
      
      (2)
      
      block-job-pause and block-job-resume will no longer no-op when trying
      to pause an already paused job, or trying to resume a job that isn't
      paused. These functions will now report that they did not perform the
      action requested because it was not possible.
      
      iotests have been adjusted to address this new behavior.
      
      (3)
      
      block-job-complete doesn't worry about checking !block_job_started,
      because the permission table guards against this.
      
      (4)
      
      test-bdrv-drain's job implementation needs to announce that it is
      'ready' now, in order to be completed.
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Reviewed-by: NKevin Wolf <kwolf@redhat.com>
      Reviewed-by: NEric Blake <eblake@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      0ec4dfb8
    • J
      blockjobs: add state transition table · c9de4050
      John Snow 提交于
      The state transition table has mostly been implied. We're about to make
      it a bit more complex, so let's make the STM explicit instead.
      
      Perform state transitions with a function that for now just asserts the
      transition is appropriate.
      
      Transitions:
      Undefined -> Created: During job initialization.
      Created   -> Running: Once the job is started.
                            Jobs cannot transition from "Created" to "Paused"
                            directly, but will instead synchronously transition
                            to running to paused immediately.
      Running   -> Paused:  Normal workflow for pauses.
      Running   -> Ready:   Normal workflow for jobs reaching their sync point.
                            (e.g. mirror)
      Ready     -> Standby: Normal workflow for pausing ready jobs.
      Paused    -> Running: Normal resume.
      Standby   -> Ready:   Resume of a Standby job.
      
      +---------+
      |UNDEFINED|
      +--+------+
         |
      +--v----+
      |CREATED|
      +--+----+
         |
      +--v----+     +------+
      |RUNNING<----->PAUSED|
      +--+----+     +------+
         |
      +--v--+       +-------+
      |READY<------->STANDBY|
      +-----+       +-------+
      
      Notably, there is no state presently defined as of this commit that
      deals with a job after the "running" or "ready" states, so this table
      will be adjusted alongside the commits that introduce those states.
      Signed-off-by: NJohn Snow <jsnow@redhat.com>
      Signed-off-by: NKevin Wolf <kwolf@redhat.com>
      c9de4050