On i.MX6SX the physical memory is placed above the 2GB mark, so the GPU
linear window has to be moved for the GPU to work at all. This doesn't
mix with the FAST_CLEAR feature, as the TS unit doesn't take the linear
window offset into account and will corrupt memory when used with a
non-zero offset.

Move the linear window if it's necessary for the GPU to work, but avoid
announcing FAST_CLEAR support to userspace in this case.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>
Tested-by: NMarek Vasut <marex@denx.de>

The dumper is only a debugging aid so we don't want to invoke the OOM
killer if buffer for the potentially large GPU state can't be vmalloced.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

The object internal SG table must not be returned, as the caller
will take ownership of the returned table.

Construct a new table from the object pages and return this one
instead.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

Vivante GPUs with HALTI0 feature support a DRAW_INSTANCED command in the
command stream to draw a number of instances of the same geometry.

The information that has been figured out about the command can be found
here: https://github.com/etnaviv/etna_viv/blob/master/rnndb/cmdstream.xml#L270

This command is not allowed currently by the DRM driver because it
was not known before. This patch enables parsing it in command
streams and allows using it by userspace drivers.
Signed-off-by: NWladimir J. van der Laan <laanwj@gmail.com>
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

This adds the required boilerplate to allow direct mmap of exported
etnaviv BOs.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>
Tested-by: NPhilipp Zabel <p.zabel@pengutronix.de>

If we define drm_compat_ioctl NULL on CONFIG_COMPAT=n, we don't have to
check for the config everywhere.
Reviewed-by: NPatrik Jakobsson <patrik.r.jakobsson@gmail.com>
Signed-off-by: NJani Nikula <jani.nikula@intel.com>
Signed-off-by: NSean Paul <seanpaul@chromium.org>
Link: http://patchwork.freedesktop.org/patch/msgid/1478014844-27454-1-git-send-email-jani.nikula@intel.com

Convert DT component matching to use component_match_add_release().
Acked-by: NJyri Sarha <jsarha@ti.com>
Reviewed-by: NJyri Sarha <jsarha@ti.com>
Signed-off-by: NRussell King <rmk+kernel@armlinux.org.uk>
Signed-off-by: NSean Paul <seanpaul@chromium.org>
Link: http://patchwork.freedesktop.org/patch/msgid/E1bwo6l-0005Io-Q1@rmk-PC.armlinux.org.uk

I plan to usurp the short name of struct fence for a core kernel struct,
and so I need to rename the specialised fence/timeline for DMA
operations to make room.

A consensus was reached in
https://lists.freedesktop.org/archives/dri-devel/2016-July/113083.html
that making clear this fence applies to DMA operations was a good thing.
Since then the patch has grown a bit as usage increases, so hopefully it
remains a good thing!

(v2...: rebase, rerun spatch)
v3: Compile on msm, spotted a manual fixup that I broke.
v4: Try again for msm, sorry Daniel

coccinelle script:
@@

@@
- struct fence
+ struct dma_fence
@@

@@
- struct fence_ops
+ struct dma_fence_ops
@@

@@
- struct fence_cb
+ struct dma_fence_cb
@@

@@
- struct fence_array
+ struct dma_fence_array
@@

@@
- enum fence_flag_bits
+ enum dma_fence_flag_bits
@@

@@
(
- fence_init
+ dma_fence_init
|
- fence_release
+ dma_fence_release
|
- fence_free
+ dma_fence_free
|
- fence_get
+ dma_fence_get
|
- fence_get_rcu
+ dma_fence_get_rcu
|
- fence_put
+ dma_fence_put
|
- fence_signal
+ dma_fence_signal
|
- fence_signal_locked
+ dma_fence_signal_locked
|
- fence_default_wait
+ dma_fence_default_wait
|
- fence_add_callback
+ dma_fence_add_callback
|
- fence_remove_callback
+ dma_fence_remove_callback
|
- fence_enable_sw_signaling
+ dma_fence_enable_sw_signaling
|
- fence_is_signaled_locked
+ dma_fence_is_signaled_locked
|
- fence_is_signaled
+ dma_fence_is_signaled
|
- fence_is_later
+ dma_fence_is_later
|
- fence_later
+ dma_fence_later
|
- fence_wait_timeout
+ dma_fence_wait_timeout
|
- fence_wait_any_timeout
+ dma_fence_wait_any_timeout
|
- fence_wait
+ dma_fence_wait
|
- fence_context_alloc
+ dma_fence_context_alloc
|
- fence_array_create
+ dma_fence_array_create
|
- to_fence_array
+ to_dma_fence_array
|
- fence_is_array
+ dma_fence_is_array
|
- trace_fence_emit
+ trace_dma_fence_emit
|
- FENCE_TRACE
+ DMA_FENCE_TRACE
|
- FENCE_WARN
+ DMA_FENCE_WARN
|
- FENCE_ERR
+ DMA_FENCE_ERR
)
 (
 ...
 )
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: NGustavo Padovan <gustavo.padovan@collabora.co.uk>
Acked-by: NSumit Semwal <sumit.semwal@linaro.org>
Acked-by: NChristian König <christian.koenig@amd.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/20161025120045.28839-1-chris@chris-wilson.co.uk

This removes the 'write' and 'force' from get_user_pages_remote() and
replaces them with 'gup_flags' to make the use of FOLL_FORCE explicit in
callers as use of this flag can result in surprising behaviour (and
hence bugs) within the mm subsystem.
Signed-off-by: NLorenzo Stoakes <lstoakes@gmail.com>
Acked-by: NMichal Hocko <mhocko@suse.com>
Reviewed-by: NJan Kara <jack@suse.cz>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Since fence_wait_timeout_reservation_object_wait_timeout_rcu() with a
timeout of 0 becomes reservation_object_test_signaled_rcu(), we do not
need to handle such conversion in the caller. The only challenge are
those callers that wish to differentiate the error code between the
nonblocking busy check and potentially blocking wait.
Signed-off-by: NChris Wilson <chris@chris-wilson.co.uk>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Russell King <linux+etnaviv@armlinux.org.uk>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: NLucas Stach <l.stach@pengutronix.de>
Signed-off-by: NSumit Semwal <sumit.semwal@linaro.org>
Link: http://patchwork.freedesktop.org/patch/msgid/20160829070834.22296-2-chris@chris-wilson.co.uk

To make sure we don't place anything there which might confuse
the FE prefetcher. This gets rid of another case of FE MMU faults
when the address space gets crowded before triggering the reaper.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

If the GPU is done with one user command stream the buffers referenced
by this command stream may go away and get unmapped from the MMU. If
the write caches are still dirty at this point later evictions will run
into MMU faults, killing the GPU.

Make sure the write caches are flushed before signaling completion
of the user command stream.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

There are many reasons other than ENOMEM that drm_dev_init() can
fail. Return ERR_PTR rather than NULL to be able to distinguish
these in the caller.
Signed-off-by: NTom Gundersen <teg@jklm.no>
Signed-off-by: NSean Paul <seanpaul@chromium.org>
Link: http://patchwork.freedesktop.org/patch/msgid/20160921145919.13754-2-teg@jklm.no

If we reset the GPU to get it back into a usable state we lose
all context, not just the MMU one. Mark the whole context as
lost to trigger a restore of the exec and MMU state.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

For cmdbufs the CPU IOVA was recorded instead of the GPU one.
Fix this to make it consistent with other BOs and to make
reading the dumps easier.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

At least on the GC3000 the FE MMU is not properly flushing stale TLB
entries. Make sure to map the cmdbufs with a big enough spacing in
the IOVAs to not hit old/prefetched TLB entries when jumping to a
newly mapped cmdbuf.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

GC2000+ on the i.MX6QP is just a re-branded GC3000, lets call it by
its real name to avoid confusion in other parts of the driver.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

All other parts are now in place, so implement the actual translation
step and hook it up, so the driver claims support for cores with
the new MMU.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

Bit 30 of the interrupt status signals an MMU exception. Handle this
condition properly and dump some useful registers.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

Flushing works differently on MMUv2, in that it's only necessary
to set a single bit in the control register to flush all translation
units. A semaphore stall then makes sure that the flush has propagated
properly.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

Both the safe/scratch address and the master TLB address are per pipe
with the CPU mapped registers not properly propagating to the
different translation units.

The only way to correctly configure all translation units is to have
a command stream snipped executed by the FE, before any other execution
can start.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

With MMUv2 all buffers need to be mapped through the MMU once it
is enabled. Align the buffer size to 4K, as the MMU is only able to
map page aligned buffers.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

With MMUv2 the command buffers need to be mapped through the MMU.
Split out the iova search and MMU reaping logic so it can be reused
for the cmdbuf mapping, where no GEM object is involved.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

Split out into a new externally visible function, as the IOMMUv2
code needs this functionality, too.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

Split out into a new externally visible function, as the IOMMUv2
code needs this functionality, too.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

The GPU virtual address for the command buffers differs depending on
the IOMMU version. Move the calculation of the iova into etnaviv
mmu, to enable proper dispatch.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

This has been there from the original merge, but has never been used.
Get rid of it.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

The GPU code doesn't need to deal with the IOMMU directly, instead
it can all be hidden behind the etnaviv mmu interface. Move the
last remaining part into etnaviv mmu.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

So we can call the v2 restore code once it is there.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

It is only relevant for the V1 MMU, so we should not do this in the
common code.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

This function has external visibility and only handles the Vivant IOMMU
version 1. Rename to make this more clear and allow a clear separation
of the different IOMMU versions.

Also drop the domain parameter, as we can infer it from the GPU we are
dealing with.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

There is no linear window on MMUv2 and the FE can access the full 4GB
address space either directly (as long as the MMU isn't configured) or
through the MMU, once it is up.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

As the comment above the code states, the linear window is only
available on MMUv1. Don't try to use it on MMUv2.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

The driver doesn't ever enable individual clocks alone, so there
is no need to scatter the clock enable/disable sequences through
multiple functions. Fold them into the top one.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

There is no need to initialize variable 'err' with 0 because it will
be properly assigned later on.
Signed-off-by: NFabio Estevam <festevam@gmail.com>

In the etnaviv_gpu_platform_probe() error path the 'fail' label is
used to just return the error code.

This can be simplified by returning the error code immediately, so
get rid of the unneeded 'fail' label.
Signed-off-by: NFabio Estevam <festevam@gmail.com>

clk_prepare_enable() may fail, so we should better check for its return
value and propagate it in the case of failure.
Signed-off-by: NFabio Estevam <festevam@gmail.com>

It's deprecated and only should be used by drivers which still use
drm_platform_init, not by anyone else.

And indeed it's entirely unused and can be nuked.

Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: NLucas Stach <l.stach@pengutronix.de>
Reviewed-by: NSean Paul <seanpaul@chromium.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1471034937-651-6-git-send-email-daniel.vetter@ffwll.ch

Since commit 4984979b ("drm/irq: simplify irq checks in
drm_wait_vblank"), the drm driver feature flag DRIVER_HAVE_IRQ is only
required for drivers that have an IRQ handler managed by the DRM core.
Some drivers, armada, etnaviv, kirin and sti, set this flag without
.irq_handler setup in drm_driver.  These drivers manage IRQ handler
by themselves and the flag DRIVER_HAVE_IRQ makes no sense there.

Drop the flag for these drivers to avoid confusion.
Signed-off-by: NShawn Guo <shawnguo@kernel.org>
Cc: Vincent Abriou <vincent.abriou@st.com>
Cc: Xinliang Liu <z.liuxinliang@hisilicon.com>
Acked-by: Russell King <rmk+kernel@armlinux.org.uk> (for armada and etnaviv)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1471331168-5601-1-git-send-email-shawnguo@kernel.org

Both the fence and event alloc are safe to be done without holding the GPU
lock, as they either don't need any locking (fences) or are protected by
their own lock (events).

This solves a bad locking interaction between the submit path and the
recover worker. If userspace manages to exhaust all available events while
the GPU is hung, the submit will wait for events to become available
holding the GPU lock. The recover worker waits for this lock to become
available before trying to recover the GPU which frees up the allocated
events. Essentially both paths are deadlocked until the submit path
times out waiting for available events, failing the submit that could
otherwise be handled just fine if the recover worker had the chance to
bring the GPU back in a working state.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>
Reviewed-by: NChristian Gmeiner <christian.gmeiner@gmail.com>