The object internal SG table must not be returned, as the caller
will take ownership of the returned table.

Construct a new table from the object pages and return this one
instead.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

Vivante GPUs with HALTI0 feature support a DRAW_INSTANCED command in the
command stream to draw a number of instances of the same geometry.

The information that has been figured out about the command can be found
here: https://github.com/etnaviv/etna_viv/blob/master/rnndb/cmdstream.xml#L270

This command is not allowed currently by the DRM driver because it
was not known before. This patch enables parsing it in command
streams and allows using it by userspace drivers.
Signed-off-by: NWladimir J. van der Laan <laanwj@gmail.com>
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

This adds the required boilerplate to allow direct mmap of exported
etnaviv BOs.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>
Tested-by: NPhilipp Zabel <p.zabel@pengutronix.de>

This removes the 'write' and 'force' from get_user_pages_remote() and
replaces them with 'gup_flags' to make the use of FOLL_FORCE explicit in
callers as use of this flag can result in surprising behaviour (and
hence bugs) within the mm subsystem.
Signed-off-by: NLorenzo Stoakes <lstoakes@gmail.com>
Acked-by: NMichal Hocko <mhocko@suse.com>
Reviewed-by: NJan Kara <jack@suse.cz>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

To make sure we don't place anything there which might confuse
the FE prefetcher. This gets rid of another case of FE MMU faults
when the address space gets crowded before triggering the reaper.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

If the GPU is done with one user command stream the buffers referenced
by this command stream may go away and get unmapped from the MMU. If
the write caches are still dirty at this point later evictions will run
into MMU faults, killing the GPU.

Make sure the write caches are flushed before signaling completion
of the user command stream.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

There are many reasons other than ENOMEM that drm_dev_init() can
fail. Return ERR_PTR rather than NULL to be able to distinguish
these in the caller.
Signed-off-by: NTom Gundersen <teg@jklm.no>
Signed-off-by: NSean Paul <seanpaul@chromium.org>
Link: http://patchwork.freedesktop.org/patch/msgid/20160921145919.13754-2-teg@jklm.no

If we reset the GPU to get it back into a usable state we lose
all context, not just the MMU one. Mark the whole context as
lost to trigger a restore of the exec and MMU state.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

For cmdbufs the CPU IOVA was recorded instead of the GPU one.
Fix this to make it consistent with other BOs and to make
reading the dumps easier.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

At least on the GC3000 the FE MMU is not properly flushing stale TLB
entries. Make sure to map the cmdbufs with a big enough spacing in
the IOVAs to not hit old/prefetched TLB entries when jumping to a
newly mapped cmdbuf.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

GC2000+ on the i.MX6QP is just a re-branded GC3000, lets call it by
its real name to avoid confusion in other parts of the driver.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

All other parts are now in place, so implement the actual translation
step and hook it up, so the driver claims support for cores with
the new MMU.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

Bit 30 of the interrupt status signals an MMU exception. Handle this
condition properly and dump some useful registers.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

Flushing works differently on MMUv2, in that it's only necessary
to set a single bit in the control register to flush all translation
units. A semaphore stall then makes sure that the flush has propagated
properly.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

Both the safe/scratch address and the master TLB address are per pipe
with the CPU mapped registers not properly propagating to the
different translation units.

The only way to correctly configure all translation units is to have
a command stream snipped executed by the FE, before any other execution
can start.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

With MMUv2 all buffers need to be mapped through the MMU once it
is enabled. Align the buffer size to 4K, as the MMU is only able to
map page aligned buffers.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

With MMUv2 the command buffers need to be mapped through the MMU.
Split out the iova search and MMU reaping logic so it can be reused
for the cmdbuf mapping, where no GEM object is involved.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

Split out into a new externally visible function, as the IOMMUv2
code needs this functionality, too.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

Split out into a new externally visible function, as the IOMMUv2
code needs this functionality, too.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

The GPU virtual address for the command buffers differs depending on
the IOMMU version. Move the calculation of the iova into etnaviv
mmu, to enable proper dispatch.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

This has been there from the original merge, but has never been used.
Get rid of it.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

The GPU code doesn't need to deal with the IOMMU directly, instead
it can all be hidden behind the etnaviv mmu interface. Move the
last remaining part into etnaviv mmu.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

So we can call the v2 restore code once it is there.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

It is only relevant for the V1 MMU, so we should not do this in the
common code.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

This function has external visibility and only handles the Vivant IOMMU
version 1. Rename to make this more clear and allow a clear separation
of the different IOMMU versions.

Also drop the domain parameter, as we can infer it from the GPU we are
dealing with.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

There is no linear window on MMUv2 and the FE can access the full 4GB
address space either directly (as long as the MMU isn't configured) or
through the MMU, once it is up.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

As the comment above the code states, the linear window is only
available on MMUv1. Don't try to use it on MMUv2.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

The driver doesn't ever enable individual clocks alone, so there
is no need to scatter the clock enable/disable sequences through
multiple functions. Fold them into the top one.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

There is no need to initialize variable 'err' with 0 because it will
be properly assigned later on.
Signed-off-by: NFabio Estevam <festevam@gmail.com>

In the etnaviv_gpu_platform_probe() error path the 'fail' label is
used to just return the error code.

This can be simplified by returning the error code immediately, so
get rid of the unneeded 'fail' label.
Signed-off-by: NFabio Estevam <festevam@gmail.com>

clk_prepare_enable() may fail, so we should better check for its return
value and propagate it in the case of failure.
Signed-off-by: NFabio Estevam <festevam@gmail.com>

It's deprecated and only should be used by drivers which still use
drm_platform_init, not by anyone else.

And indeed it's entirely unused and can be nuked.

Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Christian Gmeiner <christian.gmeiner@gmail.com>
Acked-by: NLucas Stach <l.stach@pengutronix.de>
Reviewed-by: NSean Paul <seanpaul@chromium.org>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>
Link: http://patchwork.freedesktop.org/patch/msgid/1471034937-651-6-git-send-email-daniel.vetter@ffwll.ch

Since commit 4984979b ("drm/irq: simplify irq checks in
drm_wait_vblank"), the drm driver feature flag DRIVER_HAVE_IRQ is only
required for drivers that have an IRQ handler managed by the DRM core.
Some drivers, armada, etnaviv, kirin and sti, set this flag without
.irq_handler setup in drm_driver.  These drivers manage IRQ handler
by themselves and the flag DRIVER_HAVE_IRQ makes no sense there.

Drop the flag for these drivers to avoid confusion.
Signed-off-by: NShawn Guo <shawnguo@kernel.org>
Cc: Vincent Abriou <vincent.abriou@st.com>
Cc: Xinliang Liu <z.liuxinliang@hisilicon.com>
Acked-by: Russell King <rmk+kernel@armlinux.org.uk> (for armada and etnaviv)
Signed-off-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1471331168-5601-1-git-send-email-shawnguo@kernel.org

Both the fence and event alloc are safe to be done without holding the GPU
lock, as they either don't need any locking (fences) or are protected by
their own lock (events).

This solves a bad locking interaction between the submit path and the
recover worker. If userspace manages to exhaust all available events while
the GPU is hung, the submit will wait for events to become available
holding the GPU lock. The recover worker waits for this lock to become
available before trying to recover the GPU which frees up the allocated
events. Essentially both paths are deadlocked until the submit path
times out waiting for available events, failing the submit that could
otherwise be handled just fine if the recover worker had the chance to
bring the GPU back in a working state.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>
Reviewed-by: NChristian Gmeiner <christian.gmeiner@gmail.com>

Refactor this function implementation so that the
drm_gem_object_unreference_unlocked() function will only be called once
in case of a failure according to the Linux coding style recommendation
for centralized exiting of functions.
Signed-off-by: NMarkus Elfring <elfring@users.sourceforge.net>
[seanpaul tweaked subject]
Signed-off-by: NSean Paul <seanpaul@chromium.org>
Link: http://patchwork.freedesktop.org/patch/msgid/4af34ce6-62c6-7966-1ae3-0877d5ac909d@users.sourceforge.net

The functions drm_gem_object_unreference_unlocked() and vunmap() perform
also input parameter validation.
Thus the tests around their calls are not needed.

This issue was detected by using the Coccinelle software.
Signed-off-by: NMarkus Elfring <elfring@users.sourceforge.net>
[seanpaul tweaked subject]
Signed-off-by: NSean Paul <seanpaul@chromium.org>
Link: http://patchwork.freedesktop.org/patch/msgid/9638cd74-ffc5-d9ee-a40c-9b60e860ad8b@users.sourceforge.net

The GPU init path now reports any errors which might occur more accurately
than what is possible with the generic "something failed" message.

Remove the generic reporting, so we don't log an error into dmesg anymore
if any of the GPU cores are ignored.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>
Reviewed-by: NChristian Gmeiner <christian.gmeiner@gmail.com>

Print error messages that mention the exact cause of the failure on
all paths which may fail the GPU init.
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>
Reviewed-by: NChristian Gmeiner <christian.gmeiner@gmail.com>

Enable GPU module level hardware clock gating, using the conditions
found in the galcore v5 driver.

v2 lst: Split out clock gating enable into separate function, as
there might be more conditions needed for new hardware.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Reviewed-by: NChristian Gmeiner <christian.gmeiner@gmail.com>
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>

We already have a fallback in place to fill out the unique from
dev->unique, which is set to something reasonable in drm_dev_alloc.

Which means we only need to have a special set_busid for pci devices,
to be able to care the backwards compat code for drm 1.1 around, which
libdrm still needs.

While developing and testing this patch things blew up in really
interesting ways, and the code is rather confusing in naming things
between the kernel code, ioctl #defines and libdrm. For the next brave
dragon slayer, document all this madness properly in the userspace
interface section of gpu.tmpl.

v2: Make drm_dev_set_unique static and update kerneldoc.

v3: Entire rewrite, plus document what's going on for posterity in the
gpu docbook uapi section.

v4: Drop accidental amdgpu hunk (Emil).

v5: Drop accidental omapdrm vblank counter change (Emil).

v6: Rebase on top of the sphinx conversion.

Cc: Gustavo Padovan <gustavo.padovan@collabora.co.uk>
Cc: Emil Velikov <emil.l.velikov@gmail.com>
Tested-by: Gustavo Padovan <gustavo.padovan@collabora.co.uk> (virt_gpu)
Reviewed-by: NEmil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: NDaniel Vetter <daniel.vetter@intel.com>