gaudi_mmu_invalidate_cache() doesn't use the flags parameter, and thus
it can be set to 0 when the function is called in the gaudi only files.
Signed-off-by: NTomer Tayar <ttayar@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Device is now enabled before the hw_init() because part of the
initialization requires communication with the device firmware to get
information that is required for the initialization itself
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>
Reviewed-by: NTomer Tayar <ttayar@habana.ai>

Create a device MMU-mapped internal command buffer pool, in order to allow
the driver to allocate CBs for the signal/wait operations
that are fetched by the queues when they are configured with the user's
address space ID.

We must pre-map this internal pool due to performance issues.

This pool is needed for future ASIC support and it is currently unused in
GOYA and GAUDI.
Signed-off-by: NOfir Bitton <obitton@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Update the boot interface file from the latest version from firmware.
Defines for secure boot were added.
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>
Reviewed-by: NOmer Shpigelman <oshpigelman@habana.ai>

For internal needs of our CI we need to move all the common code into a
common folder instead of putting them in the root folder of the driver.

Same applies to the common header files under include/
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>
Reviewed-by: NOmer Shpigelman <oshpigelman@habana.ai>

In GAUDI we use QMAN0 DMA for clearing the MMU memory region
at initialization. if this operation fails it places the DMA in an error
state and then when trying to initialize QMAN0 we fail and erroneously
assume its the QMAN that failed.

This commit adds a check and clear of such DMA errors at initialization so
we will have a better understanding of what went wrong.
Signed-off-by: NMoti Haimovski <mhaimovski@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

In order for the user to be aware of wrong inputs, we must return
error in case the amount of jobs per cs exceeds the corresponding
queue size.
Signed-off-by: NOfir Bitton <obitton@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

We identified a possible race during job completion when working
with a single multi-threaded work queue. In order to overcome this
race we suggest using a single threaded work queue per completion
queue, hence we guarantee jobs completion in order.
Signed-off-by: NOfir Bitton <obitton@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Currently the driver halts the device CPU in the halt engines function,
which halts all the engines of the ASIC. The problem is that if later on we
stop the reset process (due to inability to clean memory mappings in time),
the CPU will remain in halt mode. This creates many issues, such as
thermal/power control and FLR handling.

Therefore, move the halting of the device CPU to the very end of the reset
process, just before writing to the registers to initiate the reset. In
addition, the driver now needs to send a message to the device F/W to
disable it from sending interrupts to the host machine because during halt
engines function the driver disables the MSI/MSI-X interrupts.
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>
Reviewed-by: NTomer Tayar <ttayar@habana.ai>

Remove an old hash that is not in use anymore.
Signed-off-by: NOmer Shpigelman <oshpigelman@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Instead of using the free slots amount on the compute CQ to determine
whether we can submit work to queues, use the queues pi/ci.

This is needed in future ASICs where we don't have CQ per queue.
Signed-off-by: NOfir Bitton <obitton@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Currently the amount of maximum queues is statically configured.
Using a static value is causing redundunt cycles when traversing
all queues and consumes more memory than actually needed.
In this patch we configure each asic with the exact number of
queues needed.
Signed-off-by: NOfir Bitton <obitton@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Soft-reset isn't supported in GAUDI. Remove the code that performs it and
print error in case the user wants to do it via sysfs.
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>
Reviewed-by: NTomer Tayar <ttayar@habana.ai>

Divide iATU initialization into inbound/outbound methods.
We must separate it in order to enable different match mode
per PCIe region.
In addition, added support for PCI address match mode.
Signed-off-by: NOfir Bitton <obitton@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

ECC (Error Correcting Code) interrupts are going to be handled
by the FW. Hence, we define an interface in which the driver can
obtain the relevant ECC information.
This information is needed for monitoring and can also lead
to a hard reset if ECC error is not correctable.
Signed-off-by: NOfir Bitton <obitton@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Add command submission statistics structure which can be obtained
through the info ioctl. Each drop counter describes the reason for
which the command submission was dropped.
This information is needed for the user to be aware of the specific
reason for which the submitted work was dropped. The user can then
utilize the driver more efficiently.
Signed-off-by: NOfir Bitton <obitton@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Extract detection of the cpu boot status to a function
to allow code reuse
Signed-off-by: NChristine Gharzuzi <cgharzuzi@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

rephrase some error/warning/notice messages to make them more accessible to
ordinary users.

There is no need to print context ASID as the driver currently doesn't
support multiple contexts.
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>
Reviewed-by: NTomer Tayar <ttayar@habana.ai>

After recent concurrent cs amount increase, we must also
increase queues depth since much more concurrent work can be done.
All external queue depths were increased to 4096 as gaudi's
internal queue depths were also increased to 1024.
Signed-off-by: NOfir Bitton <obitton@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Rephrase F/W error message to make it more understandable to ordinary
users.
Signed-off-by: NOmer Shpigelman <oshpigelman@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

The profiler needs to know the PLL values for correctly showing the
profiling data. Because our firmware can use different PLL configurations,
we need to read the PLL values from the ASIC to pass them to the profiler.
Signed-off-by: NAdam Aharon <aaharon@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Once there is a 64-bit field in a structure, GCC compiler for ARM aligns
the structure to 8 bytes. In order to avoid confusion when these
structures are being passed between CPUs from different architectures, we
explicitly align the structure to 8 bytes.
Reviewed-by: NOmer Shpigelman <oshpigelman@habana.ai>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

MAP/UNMAP are done also for device memory.
Reviewed-by: NOmer Shpigelman <oshpigelman@habana.ai>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Use proper bitfield masks instead of shifting values when configuring
packets sent to device.
Signed-off-by: NOfir Bitton <obitton@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Currently sync stream is limited only for external queues. We want to
remove this constraint by adding a new queue property dedicated for sync
stream. In addition we move the initialization and reset methods to the
common code since we can re-use them with slight changes.
Signed-off-by: NOfir Bitton <obitton@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Training schemes requires much more concurrent command submissions than
inference does. In addition, training command submissions can be completed
in a non serialized manner. Hence, we add support in which each ASIC will
be able to configure the amount of concurrent pending command submissions,
rather than use a predefined amount. This change will enhance performance
by allowing the user to add more concurrent work without waiting for the
previous work to be completed.
Signed-off-by: NOfir Bitton <obitton@habana.ai>
Reviewed-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

We no longer need to initialize the rate limiters in GAUDI A1.
Reviewed-by: NOmer Shpigelman <oshpigelman@habana.ai>
Signed-off-by: NOded Gabbay <oded.gabbay@gmail.com>

Replace the single element arrays with a simple value type u8 reserved,
even thought is is not used for dynamically sized trailing elements
it confuses the effort of replacing one-element arrays with
flexible arrays for that purpose.

Link: https://github.com/KSPP/linux/issues/79
Cc: Gustavo A. R. Silva <gustavoars@kernel.org>
Reviewed-by: NGustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: NTomas Winkler <tomas.winkler@intel.com>
Link: https://lore.kernel.org/r/20200723145927.882743-7-tomas.winkler@intel.comSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Use sizeof(*dev) + sizeof(*hw) instead of
sizeof(struct mei_device) + sizeof(struct mei_me_hw)

There is a possibility of bug when variable type has changed but
corresponding struct passed to the sizeof has not.

Cc: Gustavo A. R. Silva <gustavoars@kernel.org>
Reviewed-by: NGustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: NTomas Winkler <tomas.winkler@intel.com>
Link: https://lore.kernel.org/r/20200723145927.882743-6-tomas.winkler@intel.comSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

There is a possibility of bug when variable type has changed but
corresponding struct passed to the sizeof has not.

Cc: Gustavo A. R. Silva <gustavoars@kernel.org>
Reviewed-by: NGustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: NTomas Winkler <tomas.winkler@intel.com>
Link: https://lore.kernel.org/r/20200723145927.882743-5-tomas.winkler@intel.comSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

There is a possibility of bug when variable type has changed but
corresponding struct passed to the sizeof has not.

Cc: Gustavo A. R. Silva <gustavoars@kernel.org>
Reviewed-by: NGustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: NTomas Winkler <tomas.winkler@intel.com>
Link: https://lore.kernel.org/r/20200723145927.882743-4-tomas.winkler@intel.comSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Use sizeof(connect_data))) instead of
sizeof(struct mei_connect_client_data) when copying data
between user space and kernel.

There is a possibility of bug when variable type has changed but
corresponding struct passed to the sizeof has not.

Cc: Gustavo A. R. Silva <gustavoars@kernel.org>
Reviewed-by: NGustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: NTomas Winkler <tomas.winkler@intel.com>
Link: https://lore.kernel.org/r/20200723145927.882743-3-tomas.winkler@intel.comSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

There is a possibility of bug when variable type has changed but
corresponding struct passed to the sizeof has not.
Reviewed-by: NGustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: NTomas Winkler <tomas.winkler@intel.com>
Link: https://lore.kernel.org/r/20200723145927.882743-2-tomas.winkler@intel.comSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

This reverts commit 3c3b7dde, as it
turns out Tomas made a better series of patches for this same issue.

Cc: Gustavo A. R. Silva <gustavoars@kernel.org>
Cc: Tomas Winkler <tomas.winkler@intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Because the DPNIs are probed before DPMCPs and other objects that need
to be allocated, messages like "No more resources of type X left" are
printed by the fsl-mc bus driver. This patch resolves the issue by probing
the allocatable objects first and then any other object that may use
them.
Signed-off-by: NGrigore Popescu <grigore.popescu@nxp.com>
Signed-off-by: NIoana Ciornei <ioana.ciornei@nxp.com>
Reviewed-by: NLaurentiu Tudor <laurentiu.tudor@nxp.com>
Link: https://lore.kernel.org/r/20200717154800.17169-4-ioana.ciornei@nxp.comSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Replace the spinlock that serializes the MC commands with a raw
spinlock. This is needed for the RT kernel because there are MC
commands sent in interrupt context.
Signed-off-by: NLaurentiu Tudor <laurentiu.tudor@nxp.com>
Signed-off-by: NIoana Ciornei <ioana.ciornei@nxp.com>
Reviewed-by: NLaurentiu Tudor <laurentiu.tudor@nxp.com>
Link: https://lore.kernel.org/r/20200717154800.17169-3-ioana.ciornei@nxp.comSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

The MC bus has different types of devices that can be discovered on the
bus. Add the missing device types.
Signed-off-by: NIoana Ciornei <ioana.ciornei@nxp.com>
Reviewed-by: NLaurentiu Tudor <laurentiu.tudor@nxp.com>
Link: https://lore.kernel.org/r/20200717154800.17169-2-ioana.ciornei@nxp.comSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Drivers should not use legacy power management as they have to manage power
states and related operations, for the device, themselves. This driver was
handling them with the help of PCI helper functions like
pci_save/restore_state(), pci_enable/disable_device(), etc.

With generic PM, all essentials will be handled by the  PCI core. Driver
needs to do only device-specific operations.

The driver was also using pci_enable_wake(...,..., 0) to disable wake. Use
device_wakeup_disable() instead.

Compile-tested only.
Signed-off-by: NVaibhav Gupta <vaibhavgupta40@gmail.com>
Link: https://lore.kernel.org/r/20200720101722.145211-1-vaibhavgupta40@gmail.comSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Avoid a memset after a call to 'dma_alloc_coherent()'.
This is useless since
commit 518a2f19 ("dma-mapping: zero memory returned from dma_alloc_*")
Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Link: https://lore.kernel.org/r/20200718070246.338016-1-christophe.jaillet@wanadoo.frSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

The wrappers in include/linux/pci-dma-compat.h should go away.

The patch has been generated with the coccinelle script below and has been
hand modified to replace GFP_ with a correct flag.
It has been compile tested.

When memory is allocated in 'ilo_ccb_setup()' GFP_ATOMIC must be used
because a spin_lock is hold in 'ilo_open()' before calling
'ilo_ccb_setup()'

@@
@@
-    PCI_DMA_BIDIRECTIONAL
+    DMA_BIDIRECTIONAL

@@
@@
-    PCI_DMA_TODEVICE
+    DMA_TO_DEVICE

@@
@@
-    PCI_DMA_FROMDEVICE
+    DMA_FROM_DEVICE

@@
@@
-    PCI_DMA_NONE
+    DMA_NONE

@@
expression e1, e2, e3;
@@
-    pci_alloc_consistent(e1, e2, e3)
+    dma_alloc_coherent(&e1->dev, e2, e3, GFP_)

@@
expression e1, e2, e3;
@@
-    pci_zalloc_consistent(e1, e2, e3)
+    dma_alloc_coherent(&e1->dev, e2, e3, GFP_)

@@
expression e1, e2, e3, e4;
@@
-    pci_free_consistent(e1, e2, e3, e4)
+    dma_free_coherent(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_map_single(e1, e2, e3, e4)
+    dma_map_single(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_unmap_single(e1, e2, e3, e4)
+    dma_unmap_single(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4, e5;
@@
-    pci_map_page(e1, e2, e3, e4, e5)
+    dma_map_page(&e1->dev, e2, e3, e4, e5)

@@
expression e1, e2, e3, e4;
@@
-    pci_unmap_page(e1, e2, e3, e4)
+    dma_unmap_page(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_map_sg(e1, e2, e3, e4)
+    dma_map_sg(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_unmap_sg(e1, e2, e3, e4)
+    dma_unmap_sg(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_dma_sync_single_for_cpu(e1, e2, e3, e4)
+    dma_sync_single_for_cpu(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_dma_sync_single_for_device(e1, e2, e3, e4)
+    dma_sync_single_for_device(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_dma_sync_sg_for_cpu(e1, e2, e3, e4)
+    dma_sync_sg_for_cpu(&e1->dev, e2, e3, e4)

@@
expression e1, e2, e3, e4;
@@
-    pci_dma_sync_sg_for_device(e1, e2, e3, e4)
+    dma_sync_sg_for_device(&e1->dev, e2, e3, e4)

@@
expression e1, e2;
@@
-    pci_dma_mapping_error(e1, e2)
+    dma_mapping_error(&e1->dev, e2)

@@
expression e1, e2;
@@
-    pci_set_dma_mask(e1, e2)
+    dma_set_mask(&e1->dev, e2)

@@
expression e1, e2;
@@
-    pci_set_consistent_dma_mask(e1, e2)
+    dma_set_coherent_mask(&e1->dev, e2)
Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Link: https://lore.kernel.org/r/20200718070224.337964-1-christophe.jaillet@wanadoo.frSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>