lkml.org 
[lkml]   [2021]   [Aug]   [15]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: IIO, dmabuf, io_uring
From
Date
Hi Paul,

Am 13.08.21 um 13:41 schrieb Paul Cercueil:
> Hi,
>
> A few months ago we (ADI) tried to upstream the interface we use with
> our high-speed ADCs and DACs. It is a system with custom ioctls on the
> iio device node to dequeue and enqueue buffers (allocated with
> dma_alloc_coherent), that can then be mmap'd by userspace
> applications. Anyway, it was ultimately denied entry [1]; this API was
> okay in ~2014 when it was designed but it feels like re-inventing the
> wheel in 2021.
>
> Back to the drawing table, and we'd like to design something that we
> can actually upstream. This high-speed interface looks awfully similar
> to DMABUF, so we may try to implement a DMABUF interface for IIO,
> unless someone has a better idea.

Yeah, that sounds a lot like a DMABUF use case.

>
> Our first usecase is, we want userspace applications to be able to
> dequeue buffers of samples (from ADCs), and/or enqueue buffers of
> samples (for DACs), and to be able to manipulate them (mmapped
> buffers). With a DMABUF interface, I guess the userspace application
> would dequeue a dma buffer from the driver, mmap it, read/write the
> data, unmap it, then enqueue it to the IIO driver again so that it can
> be disposed of. Does that sound sane?

Well it's pretty close. Doing the map/unmap dance all the time is
usually a bad idea since flushing the CPU TLB all the time totally kills
your performance.

What you do instead is to implement the CPU synchronize callbacks in
your DMA-BUF implementation and flush caches as necessary.

>
> Our second usecase is - and that's where things get tricky - to be
> able to stream the samples to another computer for processing, over
> Ethernet or USB. Our typical setup is a high-speed ADC/DAC on a dev
> board with a FPGA and a weak soft-core or low-power CPU; processing
> the data in-situ is not an option. Copying the data from one buffer to
> another is not an option either (way too slow), so we absolutely want
> zero-copy.
>
> Usual userspace zero-copy techniques (vmsplice+splice, MSG_ZEROCOPY
> etc) don't really work with mmapped kernel buffers allocated for DMA
> [2] and/or have a huge overhead, so the way I see it, we would also
> need DMABUF support in both the Ethernet stack and USB (functionfs)
> stack. However, as far as I understood, DMABUF is mostly a DRM/V4L2
> thing, so I am really not sure we have the right idea here.

Well two possibilities here: Either implement DMA-BUF support in the
Ethernet/USB subsystem or get splice working efficiently with DMA-BUF
mappings.

The first one is certainly a lot of work and no idea if the second is
even doable and if yes also in a non-hacky way which you can get upstream.

>
> And finally, there is the new kid in town, io_uring. I am not very
> literate about the topic, but it does not seem to be able to handle
> DMA buffers (yet?). The idea that we could dequeue a buffer of samples
> from the IIO device and send it over the network in one single syscall
> is appealing, though.

As far as I know this is orthogonal to DMA-BUF. Christoph's answer
sounds like my understand is correct, but there are certainly people
which know that better than I do.

Regards,
Christian.

>
> Any thoughts? Feedback would be greatly appreciated.
>
> Cheers,
> -Paul
>
> [1]:
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Flinux-iio%2F20210217073638.21681-1-alexandru.ardelean%40analog.com%2FT%2F%23m6b853addb77959c55e078fbb06828db33d4bf3d7&data=04%7C01%7Cchristian.koenig%40amd.com%7C2c62025e34b644b98e2508d95e4f4dcb%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637644516997743314%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vZfslxljjWcXi1RccZcsnKTD8x1CixRN%2Ftk4FMsWN3U%3D&reserved=0
> [2]:
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnewbedev.com%2Fzero-copy-user-space-tcp-send-of-dma-mmap-coherent-mapped-memory&data=04%7C01%7Cchristian.koenig%40amd.com%7C2c62025e34b644b98e2508d95e4f4dcb%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637644516997753306%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Fn%2B3dO%2B%2F3r0ZpC5oKsQaPN2DREZKVWdVPahYgt2bsSw%3D&reserved=0
>
>

\
 
 \ /
  Last update: 2021-08-15 20:03    [W:0.103 / U:0.272 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site