Messages in this thread | | | Subject | Re: [PATCH] net/mlx4_en: ensure rx_desc updating reaches HW before prod db updating | From | "jianchao.wang" <> | Date | Tue, 23 Jan 2018 11:25:55 +0800 |
| |
Hi Jason
Thanks for your kindly response.
On 01/22/2018 11:47 PM, Jason Gunthorpe wrote: >>> Yeah, mlx4 NICs in Google fleet receive trillions of packets per >>> second, and we never noticed an issue. >>> >>> Although we are using a slightly different driver, using order-0 pages >>> and fast pages recycling. >>> >>> >> The driver we use will will set the page reference count to (size of pages)/stride, the >> pages will be freed by networking stack when the reference become zero, and the order-3 >> pages maybe allocated soon, this give NIC device a chance to corrupt the pages which have >> been allocated by others, such as slab. > But it looks like the wmb() is placed when stuffing new rx descriptors > into the device - how can it prevent corruption of pages where > ownership was transfered from device to the host? That sounds more like a > rmb() is missing someplace to me... > The device may see the prod_db updating before rx_desc updating. Then it will get stale rx_descs. These stale rx_descs may contain pages that has been freed to host.
> (Granted the missing wmb() is a bug, but it may not be fully solving this > issue??)the wmb() here fix one of the customer's test case.
Thanks Jianchao
| |