lkml.org 
[lkml]   [2013]   [May]   [13]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Subject[PATCHv2 net 2/2] sfc: Reduce RX scatter buffer size, and reduce alignment if appropriate
    From
    Date
    efx_start_datapath() asserts that we can fit 2 RX scatter buffers plus
    a software structure, each appropriately aligned, into a single page.
    Where L1_CACHE_BYTES == 256 and PAGE_SIZE == 4096, which is the case
    on s390, this assertion fails.

    The current scatter buffer size is also not a multiple of 64 or 128,
    which are more common cache line sizes. If we can make both the start
    and end of a scatter buffer cache-aligned, this will reduce the need
    for read-modify-write operations on inter- processor links.

    Fix the alignment by reducing EFX_RX_USR_BUF_SIZE to 2048 - 256 ==
    1792. (We could use 2048 - L1_CACHE_BYTES, but EFX_RX_USR_BUF_SIZE
    also affects user-level networking where a larger amount of
    housekeeping data may be needed. Although this version of the driver
    does not support user-level networking, I prefer to keep scattering
    behaviour consistent with the out-of-tree version.)

    This still doesn't fix the s390 build because like most architectures
    it has NET_IP_ALIGN == 2. When NET_IP_ALIGN != 0 we cannot achieve
    cache line alignment at either the start or end of a scatter buffer,
    so there is actually no point in padding the buffers to a multiple of
    the cache line size. All we need is 4-byte alignment of the network
    header, so do that.

    Adjust the assertions accordingly.

    Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
    Reported-by: Heiko Carstens <heiko.carstens@de.ibm.com>
    Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
    ---
    drivers/net/ethernet/sfc/efx.c | 6 ++++--
    drivers/net/ethernet/sfc/net_driver.h | 16 ++++++++++++++--
    drivers/net/ethernet/sfc/rx.c | 2 +-
    3 files changed, 19 insertions(+), 5 deletions(-)

    diff --git a/drivers/net/ethernet/sfc/efx.c b/drivers/net/ethernet/sfc/efx.c
    index 999289b..39e4cb3 100644
    --- a/drivers/net/ethernet/sfc/efx.c
    +++ b/drivers/net/ethernet/sfc/efx.c
    @@ -643,9 +643,11 @@ static void efx_start_datapath(struct efx_nic *efx)
    efx->rx_scatter = false;
    efx->rx_buffer_order = 0;
    } else if (efx->type->can_rx_scatter) {
    + BUILD_BUG_ON(EFX_RX_USR_BUF_SIZE % L1_CACHE_BYTES);
    BUILD_BUG_ON(sizeof(struct efx_rx_page_state) +
    - NET_IP_ALIGN + EFX_RX_USR_BUF_SIZE >
    - PAGE_SIZE / 2);
    + 2 * ALIGN(NET_IP_ALIGN + EFX_RX_USR_BUF_SIZE,
    + EFX_RX_BUF_ALIGNMENT) >
    + PAGE_SIZE);
    efx->rx_scatter = true;
    efx->rx_dma_len = EFX_RX_USR_BUF_SIZE;
    efx->rx_buffer_order = 0;
    diff --git a/drivers/net/ethernet/sfc/net_driver.h b/drivers/net/ethernet/sfc/net_driver.h
    index 5efddf3..39d6bd77 100644
    --- a/drivers/net/ethernet/sfc/net_driver.h
    +++ b/drivers/net/ethernet/sfc/net_driver.h
    @@ -72,8 +72,20 @@
    /* Maximum possible MTU the driver supports */
    #define EFX_MAX_MTU (9 * 1024)

    -/* Size of an RX scatter buffer. Small enough to pack 2 into a 4K page. */
    -#define EFX_RX_USR_BUF_SIZE 1824
    +/* Size of an RX scatter buffer. Small enough to pack 2 into a 4K page,
    + * and should be a multiple of the cache line size.
    + */
    +#define EFX_RX_USR_BUF_SIZE (2048 - 256)
    +
    +/* If possible, we should ensure cache line alignment at start and end
    + * of every buffer. Otherwise, we just need to ensure 4-byte
    + * alignment of the network header.
    + */
    +#if NET_IP_ALIGN == 0
    +#define EFX_RX_BUF_ALIGNMENT L1_CACHE_BYTES
    +#else
    +#define EFX_RX_BUF_ALIGNMENT 4
    +#endif

    /* Forward declare Precision Time Protocol (PTP) support structure. */
    struct efx_ptp_data;
    diff --git a/drivers/net/ethernet/sfc/rx.c b/drivers/net/ethernet/sfc/rx.c
    index 99f70dd..a7dfe36 100644
    --- a/drivers/net/ethernet/sfc/rx.c
    +++ b/drivers/net/ethernet/sfc/rx.c
    @@ -94,7 +94,7 @@ static inline void efx_sync_rx_buffer(struct efx_nic *efx,
    void efx_rx_config_page_split(struct efx_nic *efx)
    {
    efx->rx_page_buf_step = ALIGN(efx->rx_dma_len + NET_IP_ALIGN,
    - L1_CACHE_BYTES);
    + EFX_RX_BUF_ALIGNMENT);
    efx->rx_bufs_per_page = efx->rx_buffer_order ? 1 :
    ((PAGE_SIZE - sizeof(struct efx_rx_page_state)) /
    efx->rx_page_buf_step);
    --
    Ben Hutchings, Staff Engineer, Solarflare
    Not speaking for my employer; that's the marketing department's job.
    They asked us to note that Solarflare product names are trademarked.



    \
     
     \ /
      Last update: 2013-05-14 00:21    [W:2.460 / U:0.588 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site