lkml.org 
[lkml]   [2015]   [Apr]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    SubjectRe: [PATCH 5/6] Drivers: hv: vmbus: distribute subchannels among all vcpus
    Date
    Dexuan Cui <decui@microsoft.com> writes:

    >> -----Original Message-----
    >> From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com]
    >> Sent: Tuesday, April 21, 2015 22:28
    >> To: KY Srinivasan
    >> Cc: Haiyang Zhang; devel@linuxdriverproject.org; linux-
    >> kernel@vger.kernel.org; Dexuan Cui
    >> Subject: [PATCH 5/6] Drivers: hv: vmbus: distribute subchannels among all
    >> vcpus
    >>
    >> Primary channels are distributed evenly across all vcpus we have. When the
    >> host asks us to create subchannels it usually makes us num_cpus-1 offers
    >
    > Hi Vitaly,
    > AFAIK, in the VSP of storvsc, the number of subchannel is
    > (the_number_of_vcpus - 1) / 4.
    >
    > This means for a 8-vCPU guest, there is only 1 subchannel.
    >
    > Your new algorithm tends to make the vCPUs with small-number busier:
    > e.g., in the 8-vCPU case, assuming we have 4 SCSI controllers:
    > vCPU0: scsi0's PrimaryChannel (P)
    > vCPU1: scsi0's SubChannel (S) + scsi1's P
    > vCPU2: scsi1's S + scsi2's P
    > vCPU3: scsi2's S + scsi3's P
    > vCPU4: scsi3's S
    > vCPU5, 6 and 7 are idle.
    >
    > In this special case, the existing algorithm is better. :-)
    >
    > However, I do like this idea in your patch, that is, making sure a device's
    > primary/sub channels are assigned to differents vCPUs.

    Under special circumstances with the current code we can end up with
    having all subchannels on the same vCPU with the primary channel I guess
    :-) This is not something common, but possible.

    >
    > I'm just wondering if we should use an even better (and complex)
    > algorithm :-)

    The question here is - does sticking to the current vCPU help? If it
    does, I can suggest the following (I think I even mentioned that in my
    PATCH 00): first we try to find a (sub)channel with target_cpu ==
    current_vcpu and only when we fail we do the round robin. I'd like to
    hear K.Y.'s opinion here as he's the original author :-)

    >
    > PS, yeah, for netvsc(HV_NIC_GUID), the number of SC is indeed
    > the_number_vcpus -1. I'm not sure about the upcoming HV_ND_GUID --
    > maybe it's the same as HV_NIC_GUID.
    >
    > Thanks,
    > -- Dexuan
    >
    >> and we are supposed to distribute the work evenly among the channel
    >> itself and all its subchannels. Make sure they are all assigned to
    >> different vcpus.
    >>
    >> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
    >> ---
    >> drivers/hv/channel_mgmt.c | 29 ++++++++++++++++++++++++++++-
    >> 1 file changed, 28 insertions(+), 1 deletion(-)
    >>
    >> diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
    >> index 8f2761f..daa6417 100644
    >> --- a/drivers/hv/channel_mgmt.c
    >> +++ b/drivers/hv/channel_mgmt.c
    >> @@ -270,6 +270,8 @@ static void init_vp_index(struct vmbus_channel
    >> *channel,
    >> int i;
    >> bool perf_chn = false;
    >> u32 max_cpus = num_online_cpus();
    >> + struct vmbus_channel *primary = channel->primary_channel, *prev;
    >> + unsigned long flags;
    >>
    >> for (i = IDE; i < MAX_PERF_CHN; i++) {
    >> if (!memcmp(type_guid->b, hp_devs[i].guid,
    >> @@ -290,7 +292,32 @@ static void init_vp_index(struct vmbus_channel
    >> *channel,
    >> channel->target_vp = 0;
    >> return;
    >> }
    >> - cur_cpu = (++next_vp % max_cpus);
    >> +
    >> + /*
    >> + * Primary channels are distributed evenly across all vcpus we have.
    >> + * When the host asks us to create subchannels it usually makes us
    >> + * num_cpus-1 offers and we are supposed to distribute the work
    >> evenly
    >> + * among the channel itself and all its subchannels. Make sure they
    >> are
    >> + * all assigned to different vcpus.
    >> + */
    >> + if (!primary)
    >> + cur_cpu = (++next_vp % max_cpus);
    >> + else {
    >> + /*
    >> + * Let's assign the first subchannel of a channel to the
    >> + * primary->target_cpu+1 and all the subsequent channels
    >> to
    >> + * the prev->target_cpu+1.
    >> + */
    >> + spin_lock_irqsave(&primary->lock, flags);
    >> + if (primary->num_sc == 1)
    >> + cur_cpu = (primary->target_cpu + 1) % max_cpus;
    >> + else {
    >> + prev = list_prev_entry(channel, sc_list);
    >> + cur_cpu = (prev->target_cpu + 1) % max_cpus;
    >> + }
    >> + spin_unlock_irqrestore(&primary->lock, flags);
    >> + }
    >> +
    >> channel->target_cpu = cur_cpu;
    >> channel->target_vp = hv_context.vp_index[cur_cpu];
    >> }
    >> --
    >> 1.9.3

    --
    Vitaly


    \
     
     \ /
      Last update: 2015-04-24 11:41    [W:5.140 / U:0.092 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site