The patch adding the iommu lock did not initialize it.
The struct is zero-initialized so this is mostly a problem
when using lockdep.
Reported-by: Nkernel test robot <rong.a.chen@intel.com>
Cc: Max Gurtovoy <maxg@mellanox.com>
Fixes: 0ea9ee43 ("vdpasim: protect concurrent access to iommu iotlb")
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Apparently, on parisc le16_to_cpu returns an int. virtio_cread_le
is very strict about type sizes so it causes a warning.
Fix it up by casting to the correct type.
Reported-by: Nkernel test robot <lkp@intel.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>
Link: https://lore.kernel.org/r/20200805235550.1451637-1-mst@redhat.com

Add a front end VDPA driver that registers in the VDPA bus and provides
networking to a guest. The VDPA driver creates the necessary resources
on the VF it is driving such that data path will be offloaded.

Notifications are being communicated through the driver.

Currently, only VFs are supported. In subsequent patches we will have
devlink support to control which VF is used for VDPA and which function
is used for regular networking.
Reviewed-by: NParav Pandit <parav@mellanox.com>
Signed-off-by: NEli Cohen <eli@mellanox.com>
Link: https://lore.kernel.org/r/20200804162048.22587-13-eli@mellanox.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>
Acked-by: NJason Wang <jasowang@redhat.com>

Add code to support registering address space region for the device. The
virtio driver can run as either:
1. Guest virtio driver
2. Userspace virtio driver on the host
3. Kernel virtio driver on the host

In any case a memory key object is required to provide access to memory
for the device.

This code will be shared by network or block driver implementations.
Reviewed-by: NParav Pandit <parav@mellanox.com>
Signed-off-by: NEli Cohen <eli@mellanox.com>
Link: https://lore.kernel.org/r/20200804162048.22587-12-eli@mellanox.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

Following patches introduce VDPA network driver for Mellanox Connectx6
devices. This patch provides functionality that will be used by those
patches.
Reviewed-by: NParav Pandit <parav@mellanox.com>
Signed-off-by: NEli Cohen <eli@mellanox.com>
Link: https://lore.kernel.org/r/20200804162048.22587-11-eli@mellanox.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

Keep all vdpa related hardware definitions in this file.
Reviewed-by: NParav Pandit <parav@mellanox.com>
Signed-off-by: NEli Cohen <eli@mellanox.com>
Link: https://lore.kernel.org/r/20200804162048.22587-10-eli@mellanox.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

Modify get_vq_state() so it returns an error code. In case of hardware
acceleration, the available index may be retrieved from the device, an
operation that can possibly fail.
Reviewed-by: NParav Pandit <parav@mellanox.com>
Signed-off-by: NEli Cohen <eli@mellanox.com>
Link: https://lore.kernel.org/r/20200804162048.22587-9-eli@mellanox.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>
Acked-by: NJason Wang <jasowang@redhat.com>

For now VQ state involves 16 bit available index value encoded in u64
variable. In the future it will be extended to contain more fields. Use
struct to contain the state, now containing only a single u16 for the
available index. In the future we can add fields to this struct.
Reviewed-by: NParav Pandit <parav@mellanox.com>
Acked-by: NJason Wang <jasowang@redhat.com>
Signed-off-by: NEli Cohen <eli@mellanox.com>
Link: https://lore.kernel.org/r/20200804162048.22587-8-eli@mellanox.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

This will enable vdpa providers to add support for multi queue feature
and publish it to upper layers (vhost and virtio).
Signed-off-by: NMax Gurtovoy <maxg@mellanox.com>
Reviewed-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200804162048.22587-7-eli@mellanox.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

The vDPA simulator support both set_map() and dma_map()/dma_unmap()
operations. But vhost-vdpa can only use one of them. So this patch
introduce a module parameter (batch_mapping) that let vpda_sim to
support only one of those dma operations. The batched mapping via
set_map() is enabled by default.
Signed-off-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200804162048.22587-6-eli@mellanox.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

This patches extend the vhost IOTLB API to accept batch updating hints
form userspace. When userspace wants update the device IOTLB in a
batch, it may do:

1) Write vhost_iotlb_msg with VHOST_IOTLB_BATCH_BEGIN flag
2) Perform a batch of IOTLB updating via VHOST_IOTLB_UPDATE/INVALIDATE
3) Write vhost_iotlb_msg with VHOST_IOTLB_BATCH_END flag

Vhost-vdpa may decide to batch the IOMMU/IOTLB updating in step 3 when
vDPA device support set_map() ops. This is useful for the vDPA device
that want to know all the mappings to tweak their own DMA translation
logic.

For vDPA device that doesn't require set_map(), no behavior changes.

This capability is advertised via VHOST_BACKEND_F_IOTLB_BATCH capability.
Signed-off-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200804162048.22587-5-eli@mellanox.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

This patch makes userspace can get and set backend features to
vhost-vdpa.
Signed-off-by: NCindy Lu <lulu@redhat.com>
Signed-off-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200804162048.22587-4-eli@mellanox.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

Move the backend features setting/getting from net.c to vhost.c to be
reused by vhost-vdpa.
Signed-off-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200804162048.22587-3-eli@mellanox.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

Switch to use 'switch' to make the codes more easier to be extended.
Signed-off-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200804162048.22587-2-eli@mellanox.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

Merge shared code that's necessary for mlx5 vdpa bits to build.

The branch itself includes a small number of patches and is also
independently merged by a couple of other trees.
That shouldn't cause any conflicts by itself.

Saeed Mahameed <saeedm@mellanox.com> says:

---

mlx5-next is a very small branch based on a very early rc that includes
mlx5 shared stuff between rdma and net-next, and now virtio as well.

---
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

IRQ of a vq is not expected to be changed in a DRIVER_OK ~ !DRIVER_OK
period for irq offloading purposes. Place this comment at the side of
bus ops get_vq_irq than in set_status in vhost_vdpa.
Signed-off-by: NZhu Lingshan <lingshan.zhu@intel.com>
Link: https://lore.kernel.org/r/20200804102123.69978-1-lingshan.zhu@intel.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

Fix the comment of virtio_pci_find_capability() by adding missing comment
for the last parameter: bars.

Fixes: 59a5b0f7 ("virtio-pci: alloc only resources actually used.")
Signed-off-by: NLiao Pingfang <liao.pingfang@zte.com.cn>
Signed-off-by: NYi Wang <wang.yi59@zte.com.cn>
Link: https://lore.kernel.org/r/1596455545-43556-1-git-send-email-wang.yi59@zte.com.cnSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>
Acked-by: NJason Wang <jasowang@redhat.com>

If failed to connect, there is no need to start consumer nor
producer.
Signed-off-by: NZhu Lingshan <lingshan.zhu@intel.com>
Suggested-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200731065533.4144-7-lingshan.zhu@intel.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

This commit implemented vdpa_config_ops.get_vq_irq() in ifcvf,
and initialized vq irq to -EINVAL. So that ifcvf can report
irq number of a vq, or -EINVAL if the vq is not assigned an
irq number.
Signed-off-by: NZhu Lingshan <lingshan.zhu@intel.com>
Suggested-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200731065533.4144-6-lingshan.zhu@intel.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

This patch introduce a set of functions for setup/unsetup
and update irq offloading respectively by register/unregister
and re-register the irq_bypass_producer.

With these functions, this commit can setup/unsetup
irq offloading through setting DRIVER_OK/!DRIVER_OK, and
update irq offloading through SET_VRING_CALL.
Signed-off-by: NZhu Lingshan <lingshan.zhu@intel.com>
Suggested-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200731065533.4144-5-lingshan.zhu@intel.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

This commit adds a new function get_vq_irq() in struct
vdpa_config_ops, which will return the irq number of a
virtqueue.
Signed-off-by: NZhu Lingshan <lingshan.zhu@intel.com>
Suggested-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200731065533.4144-4-lingshan.zhu@intel.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

vDPA devices has dedicated backed hardware like
passthrough-ed devices. Then it is possible to setup irq
offloading to vCPU for vDPA devices. Thus this patch tries to
manipulated assigned device counters by
kvm_arch_start/end_assignment() in irqbypass manager, so that
assigned devices could be detected in update_pi_irte()

We will increase/decrease the assigned device counter in kvm/x86.
Both vDPA and VFIO would go through this code path.

Only X86 uses these counters and kvm_arch_start/end_assignment(),
so this code path only affect x86 for now.
Signed-off-by: NZhu Lingshan <lingshan.zhu@intel.com>
Suggested-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200731065533.4144-3-lingshan.zhu@intel.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

This commit introduces struct vhost_vring_call which replaced
raw struct eventfd_ctx *call_ctx in struct vhost_virtqueue.
Besides eventfd_ctx, it contains a spin lock and an
irq_bypass_producer in its structure.
Signed-off-by: NZhu Lingshan <lingshan.zhu@intel.com>
Suggested-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200731065533.4144-2-lingshan.zhu@intel.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

Make use of the flex_array_size() helper to calculate the size of a
flexible array member within an enclosing structure.

This helper offers defense-in-depth against potential integer
overflows, while at the same time makes it explicitly clear that
we are dealing with a flexible array member.
Signed-off-by: NGustavo A. R. Silva <gustavoars@kernel.org>
Link: https://lore.kernel.org/r/20200731130956.GA30525@embeddedorSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

Iommu iotlb can be accessed by different cores for performing IO using
multiple virt queues. Add a spinlock to synchronize iotlb accesses.

This could be easily reproduced when using more than 1 pktgen threads
to inject traffic to vdpa simulator.

Fixes: 2c53d0f6("vdpasim: vDPA device simulator")
Cc: stable@vger.kernel.org
Signed-off-by: NMax Gurtovoy <maxg@mellanox.com>
Signed-off-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200731073822.13326-1-jasowang@redhat.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

We used to have a per device feature whitelist to filter out the
unsupported virtio features. But this seems unnecessary since:

- the main idea behind feature whitelist is to block control vq
  feature until we finalize the control virtqueue API. But the current
  vhost-vDPA uAPI is sufficient to support control virtqueue. For
  device that has hardware control virtqueue, the vDPA device driver
  can just setup the hardware virtqueue and let userspace to use
  hardware virtqueue directly. For device that doesn't have a control
  virtqueue, the vDPA device driver need to use e.g vringh to emulate
  a software control virtqueue.
- we don't do it in virtio-vDPA driver

So remove this limitation.
Signed-off-by: NJason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20200720085043.16485-1-jasowang@redhat.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>

The loop may exist if vq->broken is true,
virtqueue_get_buf_ctx_packed or virtqueue_get_buf_ctx_split
will return NULL, so virtnet_poll will reschedule napi to
receive packet, it will lead cpu usage(si) to 100%.

call trace as below:
virtnet_poll
	virtnet_receive
		virtqueue_get_buf_ctx
			virtqueue_get_buf_ctx_packed
			virtqueue_get_buf_ctx_split
	virtqueue_napi_complete
		virtqueue_poll           //return true
		virtqueue_napi_schedule //it will reschedule napi

to fix this, return false if vq is broken in virtqueue_poll.
Signed-off-by: NMao Wenan <wenan.mao@linux.alibaba.com>
Acked-by: NMichael S. Tsirkin <mst@redhat.com>
Link: https://lore.kernel.org/r/1596354249-96204-1-git-send-email-wenan.mao@linux.alibaba.comSigned-off-by: NMichael S. Tsirkin <mst@redhat.com>
Acked-by: NJason Wang <jasowang@redhat.com>

Speed and duplex config fields depend on VIRTIO_NET_F_SPEED_DUPLEX
which being 63>31 depends on VIRTIO_F_VERSION_1.

Accordingly, use LE accessors for these fields.
Reported-by: NCornelia Huck <cohuck@redhat.com>
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

All drivers now use virtio_cread/write_le for LE config
space fields. Drop LE option from virtio_cread/write, only leaving
the option to access transitional fields.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Virtio iommu is modern-only. Use LE accessors for config space.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Virtio mem is modern-only. Use LE accessors for config space.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Virtgpu is modern-only. Use LE accessors for config space.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Virtio pmem is modern-only. Use LE accessors for config space.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Virtio crypto is modern-only. Use LE accessors for config space.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Virtio fs is modern-only. Use LE accessors for config space.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Virtio input is modern-only. Use LE accessors for config space.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Balloon is LE, it's cleaner to access it as such directly.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Mirrors virtio_cread_feature but for LE fields.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

Tag config space fields as having virtio endian-ness.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

To be used by modern code, as well as to handle LE only fields such as
balloon.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>