1. 02 7月, 2014 1 次提交
    • L
      workqueue: clear POOL_DISASSOCIATED in rebind_workers() · 3de5e884
      Lai Jiangshan 提交于
      a9ab775b ("workqueue: directly restore CPU affinity of workers
      from CPU_ONLINE") moved pool locking into rebind_workers() but left
      "pool->flags &= ~POOL_DISASSOCIATED" in workqueue_cpu_up_callback().
      
      There is nothing necessarily wrong with it, but there is no benefit
      either.  Let's move it into rebind_workers() and achieve the following
      benefits:
      
        1) better readability, POOL_DISASSOCIATED is cleared in rebind_workers()
           as expected.
      
        2) we can guarantee that, when POOL_DISASSOCIATED is clear, the
           running workers of the pool are on the local CPU (pool->cpu).
      
      tj: Minor description update.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      3de5e884
  2. 20 6月, 2014 6 次提交
  3. 28 5月, 2014 1 次提交
  4. 22 5月, 2014 2 次提交
    • L
      workqueue: remove the confusing POOL_FREEZING · 74b414ea
      Lai Jiangshan 提交于
      Currently, the global freezing state is propagated to worker_pools via
      POOL_FREEZING and then to each workqueue; however, the middle step -
      propagation through worker_pools - can be skipped as long as one or
      more max_active adjustments happens for each workqueue after the
      update to the global state is visible.  The global workqueue freezing
      state and the max_active adjustments during workqueue creation and
      [un]freezing are serialized with wq_pool_mutex, so it's trivial to
      guarantee that max_actives stay in sync with global freezing state.
      
      POOL_FREEZING is unnecessary and makes the code more confusing and
      complicates freeze_workqueues_begin() and thaw_workqueues() by
      requiring them to walk through all pools.
      
      Remove POOL_FREEZING and use workqueue_freezing directly instead.
      
      tj: Description and comment updates.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      74b414ea
    • L
      workqueue: rename first_worker() to first_idle_worker() · 1037de36
      Lai Jiangshan 提交于
      first_worker() actually returns the first idle workers, the name
      first_idle_worker() which is self-commnet will be better.
      
      All the callers of first_worker() expect it returns an idle worker,
      the name first_idle_worker() with "idle" notation makes reviewers happier.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      1037de36
  5. 20 5月, 2014 10 次提交
    • L
      workqueue: use generic attach/detach routine for rescuers · 51697d39
      Lai Jiangshan 提交于
      There are several problems with the code that rescuers use to bind
      themselve to the target pool's cpumask.
      
        1) It is very different from how the normal workers bind to cpumask,
           increasing code complexity and maintenance overhead.
      
        2) The code of cpu-binding for rescuers is complicated.
      
        3) If one or more cpu hotplugs happen while a rescuer is processing
           its scheduled work items, the rescuer may not stay bound to the
           cpumask of the pool. This is an allowed behavior, but is still
           hairy. It will be better if the cpumask of the rescuer is always
           kept synchronized with the pool across cpu hotplugs.
      
      Using generic attach/detach routine will solve the above problems and
      results in much simpler code.
      
      tj: Minor description updates.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      51697d39
    • L
      workqueue: separate pool-attaching code out from create_worker() · 4736cbf7
      Lai Jiangshan 提交于
      Currently, the code to attach a new worker to its pool is embedded in
      create_worker().  Separating this code out will make the codes clearer
      and will allow rescuers to share the code path later.
      
      tj: Description and comment updates.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      4736cbf7
    • L
      workqueue: rename manager_mutex to attach_mutex · 92f9c5c4
      Lai Jiangshan 提交于
      manager_mutex is only used to protect the attaching for the pool
      and the pool->workers list. It protects the pool->workers and operations
      based on this list, such as:
      
      	cpu-binding for the workers in the pool->workers
      	the operations to set/clear WORKER_UNBOUND
      
      So let's rename manager_mutex to attach_mutex to better reflect its
      role. This patch is a pure rename.
      
      tj: Minor command and description updates.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      92f9c5c4
    • L
      workqueue: narrow the protection range of manager_mutex · 4d757c5c
      Lai Jiangshan 提交于
      In create_worker(), as pool->worker_ida now uses
      ida_simple_get()/ida_simple_put() and doesn't require external
      synchronization, it doesn't need manager_mutex.
      
      struct worker allocation and kthread allocation are not visible by any
      one before attached, so they don't need manager_mutex either.
      
      The above operations are before the attaching operation which attaches
      the worker to the pool. Between attaching and starting the worker, the
      worker is already attached to the pool, so the cpu hotplug will handle
      cpu-binding for the worker correctly and we don't need the
      manager_mutex after attaching.
      
      The conclusion is that only the attaching operation needs manager_mutex,
      so we narrow the protection section of manager_mutex in create_worker().
      
      Some comments about manager_mutex are removed, because we will rename
      it to attach_mutex and add worker_attach_to_pool() later which will be
      self-explanatory.
      
      tj: Minor description updates.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      4d757c5c
    • L
      workqueue: convert worker_idr to worker_ida · 7cda9aae
      Lai Jiangshan 提交于
      We no longer iterate workers via worker_idr and worker_idr is used
      only for allocating/freeing ID, so we can convert it to worker_ida.
      
      By using ida_simple_get/remove(), worker_ida doesn't require external
      synchronization, so we don't need manager_mutex to protect it and the
      ID-removal code is allowed to be moved out from
      worker_detach_from_pool().
      
      In a later patch, worker_detach_from_pool() will be used in rescuers
      which don't have IDs, so we move the ID-removal code out from
      worker_detach_from_pool() into worker_thread().
      
      tj: Minor description updates.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      7cda9aae
    • L
      workqueue: separate iteration role from worker_idr · da028469
      Lai Jiangshan 提交于
      worker_idr has the iteration (iterating for attached workers) and
      worker ID duties. These two duties don't have to be tied together. We
      can separate them and use a list for tracking attached workers and
      iteration.
      
      Before this separation, it wasn't possible to add rescuer workers to
      worker_idr due to rescuer workers couldn't allocate ID dynamically
      because ID-allocation depends on memory-allocation, which rescuer
      can't depend on.
      
      After separation, we can easily add the rescuer workers to the list for
      iteration without any memory-allocation. It is required when we attach
      the rescuer worker to the pool in later patch.
      
      tj: Minor description updates.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      da028469
    • L
      workqueue: destroy worker directly in the idle timeout handler · 3347fc9f
      Lai Jiangshan 提交于
      Since destroy_worker() doesn't need to sleep nor require manager_mutex,
      destroy_worker() can be directly called in the idle timeout
      handler, it helps us remove POOL_MANAGE_WORKERS and
      maybe_destroy_worker() and simplify the manage_workers()
      
      After POOL_MANAGE_WORKERS is removed, worker_thread() doesn't
      need to test whether it needs to manage after processed works.
      So we can remove the test branch.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      3347fc9f
    • L
      workqueue: async worker destruction · 60f5a4bc
      Lai Jiangshan 提交于
      worker destruction includes these parts of code:
      	adjust pool's stats
      	remove the worker from idle list
      	detach the worker from the pool
      	kthread_stop() to wait for the worker's task exit
      	free the worker struct
      
      We can find out that there is no essential work to do after
      kthread_stop(), which means destroy_worker() doesn't need to wait for
      the worker's task exit, so we can remove kthread_stop() and free the
      worker struct in the worker exiting path.
      
      However, put_unbound_pool() still needs to sync the all the workers'
      destruction before destroying the pool; otherwise, the workers may
      access to the invalid pool when they are exiting.
      
      So we also move the code of "detach the worker" to the exiting
      path and let put_unbound_pool() to sync with this code via
      detach_completion.
      
      The code of "detach the worker" is wrapped in a new function
      "worker_detach_from_pool()" although worker_detach_from_pool() is only
      called once (in worker_thread()) after this patch, but we need to wrap
      it for these reasons:
      
        1) The code of "detach the worker" is not short enough to unfold them
           in worker_thread().
        2) the name of "worker_detach_from_pool()" is self-comment, and we add
           some comments above the function.
        3) it will be shared by rescuer in later patch which allows rescuer
           and normal thread use the same attach/detach frameworks.
      
      The worker id is freed when detaching which happens before the worker
      is fully dead, but this id of the dying worker may be re-used for a
      new worker, so the dying worker's task name is changed to
      "worker/dying" to avoid two or several workers having the same name.
      
      Since "detach the worker" is moved out from destroy_worker(),
      destroy_worker() doesn't require manager_mutex, so the
      "lockdep_assert_held(&pool->manager_mutex)" in destroy_worker() is
      removed, and destroy_worker() is not protected by manager_mutex in
      put_unbound_pool().
      
      tj: Minor description updates.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      60f5a4bc
    • L
      workqueue: destroy_worker() should destroy idle workers only · 73eb7fe7
      Lai Jiangshan 提交于
      We used to have the CPU online failure path where a worker is created
      and then destroyed without being started. A worker was created for
      the CPU coming online and if the online operation failed the created worker
      was shut down without being started.  But this behavior was changed.
      The first worker is created and started at the same time for the CPU coming
      online.
      
      It means that we had already ensured in the code that destroy_worker()
      destroys only idle workers and we don't want to allow it to destroy
      any non-idle worker in the future. Otherwise, it may be buggy and it
      may be extremely hard to check. We should force destroy_worker() to
      destroy only idle workers explicitly.
      
      Since destroy_worker() destroys only idle workers, this patch does not
      change any functionality. We just need to update the comments and the
      sanity check code.
      
      In the sanity check code, we will refuse to destroy the worker
      if !(worker->flags & WORKER_IDLE).
      
      If the worker entered idle which means it is already started,
      so we remove the check of "worker->flags & WORKER_STARTED",
      after this removal, WORKER_STARTED is totally unneeded,
      so we remove WORKER_STARTED too.
      
      In the comments for create_worker(), "Create a new worker which is bound..."
      is changed to "... which is attached..." due to we change the name of this
      behavior to attaching.
      
      tj: Minor description / comment updates.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      73eb7fe7
    • L
      workqueue: use manager lock only to protect worker_idr · 9625ab17
      Lai Jiangshan 提交于
      worker_idr is highly bound to managers and is always/only accessed in manager
      lock context. So we don't need pool->lock for it.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      9625ab17
  6. 13 5月, 2014 1 次提交
  7. 19 4月, 2014 2 次提交
    • D
      workqueue: simplify wq_update_unbound_numa() by jumping to use_dfl_pwq if the... · 534a3fbb
      Daeseok Youn 提交于
      workqueue: simplify wq_update_unbound_numa() by jumping to use_dfl_pwq if the target cpumask equals wq's
      
      wq_update_unbound_numa(), when it's decided that the newly updated
      cpumask equals the default, looks at whether the current pwq is
      already the default one and skips setting pwq to the default one.
      This extra step is unnecessary and we can always jump to use_dfl_pwq
      instead. Simplify the code by removing the conditional.
      This doesn't make any functional difference.
      Signed-off-by: NDaeseok Youn <daeseok.youn@gmail.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      534a3fbb
    • L
      workqueue: fix a possible race condition between rescuer and pwq-release · 77668c8b
      Lai Jiangshan 提交于
      There is a race condition between rescuer_thread() and
      pwq_unbound_release_workfn().
      
      Even after a pwq is scheduled for rescue, the associated work items
      may be consumed by any worker.  If all of them are consumed before the
      rescuer gets to them and the pwq's base ref was put due to attribute
      change, the pwq may be released while still being linked on
      @wq->maydays list making the rescuer dereference already freed pwq
      later.
      
      Make send_mayday() pin the target pwq until the rescuer is done with
      it.
      
      tj: Updated comment and patch description.
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Cc: stable@vger.kernel.org # v3.10+
      77668c8b
  8. 18 4月, 2014 2 次提交
  9. 17 4月, 2014 1 次提交
  10. 26 3月, 2014 1 次提交
  11. 23 2月, 2014 1 次提交
  12. 19 2月, 2014 1 次提交
    • L
      workqueue: ensure @task is valid across kthread_stop() · 5bdfff96
      Lai Jiangshan 提交于
      When a kworker should die, the kworkre is notified through WORKER_DIE
      flag instead of kthread_should_stop().  This, IIRC, is primarily to
      keep the test synchronized inside worker_pool lock.  WORKER_DIE is
      first set while holding pool->lock, the lock is dropped and
      kthread_stop() is called.
      
      Unfortunately, this means that there's a slight chance that the target
      kworker may see WORKER_DIE before kthread_stop() finishes and exits
      and frees the target task before or during kthread_stop().
      
      Fix it by pinning the target task before setting WORKER_DIE and
      putting it after kthread_stop() is done.
      
      tj: Improved patch description and comment.  Moved pinning above
          WORKER_DIE for better signify what it's protecting.
      
      CC: stable@vger.kernel.org
      Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      5bdfff96
  13. 12 1月, 2014 1 次提交
  14. 26 11月, 2013 1 次提交
  15. 23 11月, 2013 4 次提交
    • L
      workqueue: fix pool ID allocation leakage and remove BUILD_BUG_ON() in init_workqueues · 4e8b22bd
      Li Bin 提交于
      When one work starts execution, the high bits of work's data contain
      pool ID. It can represent a maximum of WORK_OFFQ_POOL_NONE. Pool ID
      is assigned WORK_OFFQ_POOL_NONE when the work being initialized
      indicating that no pool is associated and get_work_pool() uses it to
      check the associated pool. So if worker_pool_assign_id() assigns a
      ID greater than or equal WORK_OFFQ_POOL_NONE to a pool, it triggers
      leakage, and it may break the non-reentrance guarantee.
      
      This patch fix this issue by modifying the worker_pool_assign_id()
      function calling idr_alloc() by setting @end param WORK_OFFQ_POOL_NONE.
      
      Furthermore, in the current implementation, the BUILD_BUG_ON() in
      init_workqueues makes no sense. The number of worker pools needed
      cannot be determined at compile time, because the number of backing
      pools for UNBOUND workqueues is dynamic based on the assigned custom
      attributes. So remove it.
      
      tj: Minor comment and indentation updates.
      Signed-off-by: NLi Bin <huawei.libin@huawei.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      4e8b22bd
    • L
      workqueue: fix comment typo for __queue_work() · 9ef28a73
      Li Bin 提交于
      It seems the "dying" should be "draining" here.
      Signed-off-by: NLi Bin <huawei.libin@huawei.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      9ef28a73
    • T
      workqueue: fix ordered workqueues in NUMA setups · 8a2b7538
      Tejun Heo 提交于
      An ordered workqueue implements execution ordering by using single
      pool_workqueue with max_active == 1.  On a given pool_workqueue, work
      items are processed in FIFO order and limiting max_active to 1
      enforces the queued work items to be processed one by one.
      
      Unfortunately, 4c16bd32 ("workqueue: implement NUMA affinity for
      unbound workqueues") accidentally broke this guarantee by applying
      NUMA affinity to ordered workqueues too.  On NUMA setups, an ordered
      workqueue would end up with separate pool_workqueues for different
      nodes.  Each pool_workqueue still limits max_active to 1 but multiple
      work items may be executed concurrently and out of order depending on
      which node they are queued to.
      
      Fix it by using dedicated ordered_wq_attrs[] when creating ordered
      workqueues.  The new attrs match the unbound ones except that no_numa
      is always set thus forcing all NUMA nodes to share the default
      pool_workqueue.
      
      While at it, add sanity check in workqueue creation path which
      verifies that an ordered workqueues has only the default
      pool_workqueue.
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Reported-by: NLibin <huawei.libin@huawei.com>
      Cc: stable@vger.kernel.org
      Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
      8a2b7538
    • O
      workqueue: swap set_cpus_allowed_ptr() and PF_NO_SETAFFINITY · 91151228
      Oleg Nesterov 提交于
      Move the setting of PF_NO_SETAFFINITY up before set_cpus_allowed()
      in create_worker(). Otherwise userland can change ->cpus_allowed
      in between.
      Signed-off-by: NOleg Nesterov <oleg@redhat.com>
      Signed-off-by: NTejun Heo <tj@kernel.org>
      91151228
  16. 29 8月, 2013 1 次提交
    • T
      workqueue: cond_resched() after processing each work item · b22ce278
      Tejun Heo 提交于
      If !PREEMPT, a kworker running work items back to back can hog CPU.
      This becomes dangerous when a self-requeueing work item which is
      waiting for something to happen races against stop_machine.  Such
      self-requeueing work item would requeue itself indefinitely hogging
      the kworker and CPU it's running on while stop_machine would wait for
      that CPU to enter stop_machine while preventing anything else from
      happening on all other CPUs.  The two would deadlock.
      
      Jamie Liu reports that this deadlock scenario exists around
      scsi_requeue_run_queue() and libata port multiplier support, where one
      port may exclude command processing from other ports.  With the right
      timing, scsi_requeue_run_queue() can end up requeueing itself trying
      to execute an IO which is asked to be retried while another device has
      an exclusive access, which in turn can't make forward progress due to
      stop_machine.
      
      Fix it by invoking cond_resched() after executing each work item.
      Signed-off-by: NTejun Heo <tj@kernel.org>
      Reported-by: NJamie Liu <jamieliu@google.com>
      References: http://thread.gmane.org/gmane.linux.kernel/1552567
      Cc: stable@vger.kernel.org
      --
       kernel/workqueue.c |    9 +++++++++
       1 file changed, 9 insertions(+)
      b22ce278
  17. 24 8月, 2013 1 次提交
  18. 21 8月, 2013 2 次提交
  19. 20 8月, 2013 1 次提交