Store the semaphore offset in a temporary variable to avoid
having to get the VMA offset twice.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Replace the macro initializer with a programatic loop which
results in smaller code and hopefully just as clear.

v2: Rebase.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

The object needs to be created before semaphores can be initialized
on any ring and it makes sense to pull it out to this semaphore
dedicated helper.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>

v2: Put dispatch_execbuffer before add_request. (Chris Wilson)
v3: Fix add_request and irq_seqno_barrier for gen8+.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

v2: Consistent INTEL_GEN vs IS_GEN usage. (Chris Wilson)
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

All engines apart from render select this based on Gen.

Move it to the common helper as well.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

Introduce a function which initializes vfuncs mostly common
across engines and move write_tail initialization in it since
only one engine overrides the default.
Signed-off-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Reviewed-by: NChris Wilson <chris@chris-wilson.co.uk>

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-53-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-61-git-send-email-chris@chris-wilson.co.uk

Since we have a sequence of register reads and writes, we can reduce the
latency of starting the BSD ring by performing all the mmio operations
under the same forcewake wakeref.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-62-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-60-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-59-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-58-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-57-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-56-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-55-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-54-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-52-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-51-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-50-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-49-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-48-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-47-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-46-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-45-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-44-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-43-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-42-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-41-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-40-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-39-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-38-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-37-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-36-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-35-git-send-email-chris@chris-wilson.co.uk

By using the out-of-line intel_wait_for_register() not only do we can
efficiency from using the hybrid wait_for() contained within, but we
avoid code bloat from the numerous inlined loops, in total (all patches):

   text    data     bss     dec     hex filename
1078551    4557     416 1083524  108884 drivers/gpu/drm/i915/i915.ko
1070775    4557     416 1075748  106a24 drivers/gpu/drm/i915/i915.ko
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NTvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1467297225-21379-34-git-send-email-chris@chris-wilson.co.uk