Messages in this thread |  | | From | Logan Gunthorpe <> | Date | Mon, 5 Mar 2018 13:13:20 -0700 | Subject | Re: [PATCH v2 07/10] nvme-pci: Use PCI p2pmem subsystem to manage the CMB |
| |
On 05/03/18 12:57 PM, Sagi Grimberg wrote: > Keith, while we're on this, regardless of cmb, is SQE memcopy and DB > update ordering always guaranteed? > > If you look at mlx4 (rdma device driver) that works exactly the same as > nvme you will find: > -- > qp->sq.head += nreq; > > /* > * Make sure that descriptors are written before > * doorbell record. > */ > wmb(); > > writel(qp->doorbell_qpn, > to_mdev(ibqp->device)->uar_map + > MLX4_SEND_DOORBELL); > > /* > * Make sure doorbells don't leak out of SQ spinlock > * and reach the HCA out of order. > */ > mmiowb(); > --
To me, it looks like the wmb() is redundant as writel should guarantee the order. (Indeed, per Sinan's comment, writel on arm64 starts with a wmb() which means, on that platform, there are two wmb() calls in a row.)
The mmiowb() call, on the other hand, looks correct per my understanding of it's purpose with respect to the spinlock.
Logan
|  |