The call to kfd_device_by_id can potentially return null, so check that
dev is null and return with -EINVAL to avoid a null pointer dereference.

Detected by CoverityScan CID#1454629 ("Dereference null return value")

Fixes: 5d71dbc3 ("drm/amdkfd: Implement image tiling mode support v2")
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

v2: Removed hole in ioctl number space
Signed-off-by: NYong Zhao <yong.zhao@amd.com>
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

v2:
* Renamed ALLOC_MEMORY_OF_SCRATCH to SET_SCRATCH_BACKING_VA
* Removed size parameter from the ioctl, it was unused
* Removed hole in ioctl number space
* No more call to write_config_static_mem
* Return correct error code from ioctl
Signed-off-by: NMoses Reuben <moses.reuben@amd.com>
Signed-off-by: NBen Goz <ben.goz@amd.com>
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Remove gotos that do not feature any common cleanup, and use gotos
instead of repeating cleanup commands.

According to kernel.org: "The goto statement comes in handy when a
function exits from multiple locations and some common work such as
cleanup has to be done. If there is no cleanup needed then just return
directly."

v2: Applied review suggestions in create_queue_nocpsch
Signed-off-by: NKent Russell <kent.russell@amd.com>
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Upstream prefers the !x notation to x==NULL or x==false. Along those lines
change the ==true or !=NULL references as well. Also make the references
to !x the same, excluding () for readability.
Signed-off-by: NKent Russell <kent.russell@amd.com>
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Consolidate log commands so that dev_info(NULL, "Error...") uses the more
accurate pr_err, remove the module name from the log (can be seen via
dynamic debugging with +m), and the function name (can be seen via
dynamic debugging with +f). We also don't need debug messages saying
what function we're in. Those can be added by devs when needed

Don't print vendor and device ID in error messages. They are typically
the same for all GPUs in a multi-GPU system. So this doesn't add any
value to the message.

Lastly, remove parentheses around %d, %i and 0x%llX.
According to kernel.org:
"Printing numbers in parentheses (%d) adds no value and should be
avoided."
Signed-off-by: NKent Russell <kent.russell@amd.com>
Signed-off-by: NYong Zhao <Yong.Zhao@amd.com>
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Using checkpatch.pl -f <file> showed a number of style issues. This
patch addresses as many of them as possible. Some long lines have been
left for readability, but attempts to minimize them have been made.

v2: Broke long lines in gfx_v7 get_fw_version
Signed-off-by: NKent Russell <kent.russell@amd.com>
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Trivial fix to spelling mistake, rename kfd_ioctl_dbg_unrgesiter
to kfd_ioctl_dbg_unregister
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Found-By: Coccinelle
Signed-off-by: NEdward O'Callaghan <eocallaghan@alterapraxis.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Found-By: Coccinelle
Signed-off-by: NEdward O'Callaghan <eocallaghan@alterapraxis.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

amdkfd wants to know syscall type, not task type.  Check directly.

Unfortunately, amdkfd is making nasty assumptions that a process'
bitness is a well-defined constant thing.  This isn't the case on x86.
I don't know how much this matters, but this patch has no effect on
generated code on x86, so amdkfd is equally broken with and without this
patch.
Signed-off-by: NAndy Lutomirski <luto@kernel.org>
Cc: Oded Gabbay <oded.gabbay@gmail.com>
Cc: David Airlie <airlied@linux.ie>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

8f1d57c1 ("amdkfd: don't open-code memdup_user()") mistakenly uses
an uninitialized local pointer, gcc complains:

  drivers/gpu/drm/amd/amdkfd/kfd_chardev.c: In function ‘kfd_ioctl_dbg_address_watch’:
  drivers/gpu/drm/amd/amdkfd/kfd_chardev.c:562:12: warning: ‘args_buff’ may be used uninitialized in this function [-Wmaybe-uninitialized]
    args_buff = memdup_user(args_buff,
                ^

Fix it.
Signed-off-by: NBorislav Petkov <bp@suse.de>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

The latter is a default version of <asm/mman.h> and not for driver use.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

buf_size_in_bytes must be large enough to hold ->num_watch_points and
watch_mode so I have added a sizeof(int) * 2 to the minimum size.

Also we have to subtract sizeof(*args) from the max args_idx limit so
that it matches the allocation.  Also I changed a > to >= for the last
compare.
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

v2:

- rename get_dbgmgr_mutex to kfd_get_dbgmgr_mutex to namespace it
- change void* to uint64_t inside ioctl arguments
- use kmalloc instead of kzalloc because we use copy_from_user
  immediately after it
Signed-off-by: NYair Shachar <yair.shachar@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

v2:

- rename get_dbgmgr_mutex to kfd_get_dbgmgr_mutex to namespace it
- change void* to uint64_t inside ioctl arguments
- use kmalloc instead of kzalloc because we use copy_from_user
  immediately after it
Signed-off-by: NYair Shachar <yair.shachar@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

v2: rename get_dbgmgr_mutex to kfd_get_dbgmgr_mutex to namespace it
Signed-off-by: NYair Shachar <yair.shachar@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

This patch adds four new IOCTLs to amdkfd. These IOCTLs expose a H/W
debugger functionality to the userspace.

The IOCTLs are:

- AMDKFD_IOC_DBG_REGISTER:

The purpose of this IOCTL is to notify amdkfd that a process wants to use
GPU debugging facilities on itself only.
It is expected that this IOCTL would be called before any other H/W
debugger requests are sent to amdkfd and for each GPU where the H/W
debugging needs to be enabled. The use of this IOCTL ensures that only
one instance of a debugger is active in the system.

- AMDKFD_IOC_DBG_UNREGISTER:

This IOCTL detaches the debugger/debugged process from the H/W
Debug which was established by the AMDKFD_IOC_DBG_REGISTER IOCTL.

- AMDKFD_IOC_DBG_ADDRESS_WATCH:

This IOCTL allows to set different watchpoints with various conditions as
indicated by the IOCTL's arguments. The available number of watchpoints
is retrieved from topology. This operation is confined to the current
debugged process, which was registered through AMDKFD_IOC_DBG_REGISTER.

- AMDKFD_IOC_DBG_WAVE_CONTROL:

This IOCTL allows to control a wavefront as indicated by the IOCTL's
arguments. For example, you can halt/resume or kill either a
single wavefront or a set of wavefronts. This operation is confined to
the current debugged process, which was registered through
AMDKFD_IOC_DBG_REGISTER.

Because the arguments for the address watch IOCTL and wave control IOCTL
are dynamic, meaning that they could vary in size, the userspace passes a
pointer to a structure (in userspace) that contains the value of the
arguments. The kernel driver is responsible to parse this structure and
validate its contents.

v2: change void* to uint64_t inside ioctl arguments
Signed-off-by: NYair Shachar <yair.shachar@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Signed-off-by: NAndrew Lewycky <Andrew.Lewycky@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

This patch adds the events module (kfd_events.c) and the interrupt
handle module for Kaveri (cik_event_interrupt.c).

The patch updates the interrupt_is_wanted(), so that it now calls the
interrupt isr function specific for the device that received the
interrupt. That function(implemented in cik_event_interrupt.c)
returns whether this interrupt is of interest to us or not.

The patch also updates the interrupt_wq(), so that it now calls the
device's specific wq function, which checks the interrupt source
and tries to signal relevant events.

v2:

Increase limit of signal events to 4096 per process
Remove bitfields from struct cik_ih_ring_entry
Rename radeon_kfd_event_mmap to kfd_event_mmap
Add debug prints to allocate_free_slot and allocate_signal_page
Make allocate_event_notification_slot return a correct value
Add warning prints to create_signal_event
Remove error print from IOCTL path
Reformatted debug prints in kfd_event_mmap
Map correct size (as received from mmap) in kfd_event_mmap

v3:

Reduce limit of signal events back to 256 per process
Fix allocation of kernel memory for signal events
Signed-off-by: NAndrew Lewycky <Andrew.Lewycky@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

- AMDKFD_IOC_CREATE_EVENT:
	Creates a new event of a specified type

- AMDKFD_IOC_DESTROY_EVENT:
	Destroys an existing event

- AMDKFD_IOC_SET_EVENT:
	Signal an existing event

- AMDKFD_IOC_RESET_EVENT:
	Reset an existing event

- AMDKFD_IOC_WAIT_EVENTS:
	Wait on event(s) until they are signaled

v2:

- Move the limit of the signal events to kfd_ioctl.h so it
  can be used by userspace

v3:
- Change all bool fields in struct kfd_memory_exception_failure
to uint32_t
Signed-off-by: NAndrew Lewycky <Andrew.Lewycky@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

The current code can only support one kgd instance. We have to
support multiple kgd instances in one system. i.e two amdgpu or two
radeon or one amdgpu + one radeon or more than two kgd instances.
Signed-off-by: NXihan Zhang <xihan.zhang@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>

Convert the timestamping in the amdkfd driver to use a timespec64 and 64bit
time accessors.

Although the existing code is completely safe beyond y2038 because it deals
with monotonic time, this patch is still needed in order to kill off all uses
of struct timespec.
Signed-off-by: NJohn Stultz <john.stultz@linaro.org>
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>

Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

This patch passes the correct queue type to pqm_create_queue() instead of a
fixed KFD_QUEUE_TYPE_COMPUTE type.
Signed-off-by: NBen Goz <ben.goz@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

This patch adds a check to the create queue ioctl path, which identifies SDMA
queue type that is sent by userspace.
Signed-off-by: NBen Goz <ben.goz@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

This patch changes kfd_ioctl() to be very similar to drm_ioctl().

The patch defines an array of amdkfd_ioctls, which maps IOCTL definition to the
ioctl function.

The kfd_ioctl() uses that mapping to call the appropriate ioctl function,
through a function pointer.

This patch also declares a new typedef for the ioctl function pointer.

v2: Renamed KFD_COMMAND_(START|END) to AMDKFD_...
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>

This patch reformats the ioctl definitions in kfd_ioctl.h to be similar to the
drm ioctls definition style.

v2: Renamed KFD_COMMAND_(START|END) to AMDKFD_...
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>
Acked-by: NChristian König <christian.koenig@amd.com>

This patch moves the copy_to_user() and copy_from_user() calls from the
different ioctl functions in amdkfd to the general kfd_ioctl() function, as
this is a common code for all ioctls.

This was done according to example taken from drm_ioctl.c
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>
Reviewed-by: NChristian König <christian.koenig@amd.com>

This patch checks if the process that opens the /dev/kfd device is 32-bit
process. If so, it returns -EPERM and prints a warning message in dmesg.

This is done to prevent 32-bit user processes from using amdkfd, and hence, HSA
features.

AMD's HSA userspace stack will also support only 64-bit processes on Linux.
Reviewed-by: NAlexey Skidanov <alexey.skidanov@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>

Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>

This patch does some re-org on the device_queue_manager structure. It takes out
all the function pointers from the structure and puts them in a new structure,
called device_queue_manager_ops. Then, it puts an instance of that structure
inside device_queue_manager.

This re-org is done to prepare the DQM module to support more than one AMD APU
(Kaveri).
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

There is a typo here so the errors from kfd_bind_process_to_device()
are not detected.
Reviewed-by: NOded Gabbay <oded.gabbay@amd.com>
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>

Since the user space may call open() more that once from the same process,
the aperture initialization should be moved from kfd_open()
Signed-off-by: NAlexey Skidanov <Alexey.Skidanov@amd.com>
Reviewed-by: NOded Gabbay <oded.gabbay@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>

Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>

Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>

This patch adds new fields to the queue_properties structure. The new fields
are relevant only for queues running on AMD GPU VI architecture.

The eop_ring_buffer_address and eop_ring_buffer_size describe an
end-of-pipe queue which is assigned to the MQD. In CI, the EOP queue was per
pipeline and in VI it is per queue.

The ctx_save_restore_area_address and ctx_save_restore_area_size describe a
memory area that is designated to allow the CP to do context save/restore in
mid-wave state.

This patch also modifies the set_queue_properties_from_user() (called from
kfd_ioctl_create_queue()) to check and copy those new parameters.
Signed-off-by: NBen Goz <ben.goz@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>
Reviewed-by: NAlex Deucher <alexander.deucher@amd.com>

v3: Fixed debug messages
Signed-off-by: NAlexey Skidanov <Alexey.Skidanov@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>

Signed-off-by: NEvgeny Pinchuk <evgeny.pinchuk@amd.com>
Signed-off-by: NOded Gabbay <oded.gabbay@amd.com>