Messages in this thread Patch in this message | | | Date | Wed, 12 Apr 2023 10:42:43 +0100 | From | Marc Zyngier <> | Subject | Re: [PATCH] irqchipi/gic-v4: Ensure accessing the correct RD when and writing INVLPIR |
| |
On Wed, 12 Apr 2023 05:15:10 +0100, Kunkun Jiang <jiangkunkun@huawei.com> wrote: > > commit f3a059219bc7 ("irqchip/gic-v4.1: Ensure mutual exclusion between > vPE affinity change and RD access") tried to address the race > between the RD accesses and the vPE affinity change, but somehow > forgot to take GICR_INVLPIR into account. Let's take the vpe_lock > before evaluating vpe->col_idx to fix it. > > Fixes: f3a059219bc7 ("irqchip/gic-v4.1: Ensure mutual exclusion between vPE affinity change and RD access") > Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com> > Signed-off-by: Xiang Chen <chenxiang66@hisilicon.com> > Signed-off-by: Nianyao Tang <tangnianyao@huawei.com>
Yup, nice catch. A few remarks though:
- the subject looks odd: there is a spurious 'and' there, and it doesn't say this is all about VPE doorbell invalidation (the code that deals with direct LPI is otherwise fine)
- the SoB chain is also odd. You should be last in the chain, and all the others have Co-developed-by tags in addition to the SoB, unless you wanted another tag
- I'm curious about how you triggered the issue. Could you please elaborate on that>
Finally, I think we can fix it in a better way, see below:
> --- > drivers/irqchip/irq-gic-v3-its.c | 10 +++++++--- > 1 file changed, 7 insertions(+), 3 deletions(-) > > diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c > index 586271b8aa39..041f06922587 100644 > --- a/drivers/irqchip/irq-gic-v3-its.c > +++ b/drivers/irqchip/irq-gic-v3-its.c > @@ -3943,13 +3943,17 @@ static void its_vpe_send_inv(struct irq_data *d) > > if (gic_rdists->has_direct_lpi) { > void __iomem *rdbase; > + unsigned long flags; > + int cpu; > > /* Target the redistributor this VPE is currently known on */ > - raw_spin_lock(&gic_data_rdist_cpu(vpe->col_idx)->rd_lock); > - rdbase = per_cpu_ptr(gic_rdists->rdist, vpe->col_idx)->rd_base; > + cpu = vpe_to_cpuid_lock(vpe, &flags); > + raw_spin_lock(&gic_data_rdist_cpu(cpu)->rd_lock); > + rdbase = per_cpu_ptr(gic_rdists->rdist, cpu)->rd_base; > gic_write_lpir(d->parent_data->hwirq, rdbase + GICR_INVLPIR); > wait_for_syncr(rdbase); > - raw_spin_unlock(&gic_data_rdist_cpu(vpe->col_idx)->rd_lock); > + raw_spin_unlock(&gic_data_rdist_cpu(cpu)->rd_lock); > + vpe_to_cpuid_unlock(vpe, flags); > } else { > its_vpe_send_cmd(vpe, its_send_inv); > }
The main reason this bug crept in is that we have a some pretty silly code duplication going on.
Wouldn't it be nice if irq_to_cpuid() could work out whether it is dealing with a LPI or a VLPI like it does today, but also directly with a VPE? We could then use the same code as derect_lpi_inv(). I came up with this the hack below, which is totally untested as I don't have access to GICv4.1 HW.
Could you give it a spin?
Thanks,
M.
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c index 586271b8aa39..cfb8be3e17d6 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -271,13 +271,24 @@ static void vpe_to_cpuid_unlock(struct its_vpe *vpe, unsigned long flags) raw_spin_unlock_irqrestore(&vpe->vpe_lock, flags); } +static struct irq_chip its_vpe_irq_chip; + static int irq_to_cpuid_lock(struct irq_data *d, unsigned long *flags) { - struct its_vlpi_map *map = get_vlpi_map(d); + struct its_vpe *vpe = NULL; int cpu; - if (map) { - cpu = vpe_to_cpuid_lock(map->vpe, flags); + if (d->chip == &its_vpe_irq_chip) + vpe = irq_data_get_irq_chip_data(d); + + if (!vpe) { + struct its_vlpi_map *map = get_vlpi_map(d); + if (map) + vpe = map->vpe; + } + + if (vpe) { + cpu = vpe_to_cpuid_lock(vpe, flags); } else { /* Physical LPIs are already locked via the irq_desc lock */ struct its_device *its_dev = irq_data_get_irq_chip_data(d); @@ -291,9 +302,18 @@ static int irq_to_cpuid_lock(struct irq_data *d, unsigned long *flags) static void irq_to_cpuid_unlock(struct irq_data *d, unsigned long flags) { - struct its_vlpi_map *map = get_vlpi_map(d); + struct its_vpe *vpe = NULL; + + if (d->chip == &its_vpe_irq_chip) + vpe = irq_data_get_irq_chip_data(d); + + if (vpe) { + struct its_vlpi_map *map = get_vlpi_map(d); + if (map) + vpe = map->vpe; + } - if (map) + if (vpe) vpe_to_cpuid_unlock(map->vpe, flags); } @@ -1431,14 +1451,29 @@ static void wait_for_syncr(void __iomem *rdbase) cpu_relax(); } -static void direct_lpi_inv(struct irq_data *d) +static void __direct_lpi_inv(struct irq_data *d, u64 val) { - struct its_vlpi_map *map = get_vlpi_map(d); void __iomem *rdbase; unsigned long flags; - u64 val; int cpu; + /* Target the redistributor this LPI is currently routed to */ + cpu = irq_to_cpuid_lock(d, &flags); + raw_spin_lock(&gic_data_rdist_cpu(cpu)->rd_lock); + + rdbase = per_cpu_ptr(gic_rdists->rdist, cpu)->rd_base; + gic_write_lpir(val, rdbase + GICR_INVLPIR); + wait_for_syncr(rdbase); + + raw_spin_unlock(&gic_data_rdist_cpu(cpu)->rd_lock); + irq_to_cpuid_unlock(d, flags); +} + +static void direct_lpi_inv(struct irq_data *d) +{ + struct its_vlpi_map *map = get_vlpi_map(d); + u64 val; + if (map) { struct its_device *its_dev = irq_data_get_irq_chip_data(d); @@ -1451,15 +1486,7 @@ static void direct_lpi_inv(struct irq_data *d) val = d->hwirq; } - /* Target the redistributor this LPI is currently routed to */ - cpu = irq_to_cpuid_lock(d, &flags); - raw_spin_lock(&gic_data_rdist_cpu(cpu)->rd_lock); - rdbase = per_cpu_ptr(gic_rdists->rdist, cpu)->rd_base; - gic_write_lpir(val, rdbase + GICR_INVLPIR); - - wait_for_syncr(rdbase); - raw_spin_unlock(&gic_data_rdist_cpu(cpu)->rd_lock); - irq_to_cpuid_unlock(d, flags); + __direct_lpi_inv(d, val); } static void lpi_update_config(struct irq_data *d, u8 clr, u8 set) @@ -3941,18 +3968,10 @@ static void its_vpe_send_inv(struct irq_data *d) { struct its_vpe *vpe = irq_data_get_irq_chip_data(d); - if (gic_rdists->has_direct_lpi) { - void __iomem *rdbase; - - /* Target the redistributor this VPE is currently known on */ - raw_spin_lock(&gic_data_rdist_cpu(vpe->col_idx)->rd_lock); - rdbase = per_cpu_ptr(gic_rdists->rdist, vpe->col_idx)->rd_base; - gic_write_lpir(d->parent_data->hwirq, rdbase + GICR_INVLPIR); - wait_for_syncr(rdbase); - raw_spin_unlock(&gic_data_rdist_cpu(vpe->col_idx)->rd_lock); - } else { + if (gic_rdists->has_direct_lpi) + __direct_lpi_inv(d, d->parent_data->hwirq); + else its_vpe_send_cmd(vpe, its_send_inv); - } } static void its_vpe_mask_irq(struct irq_data *d)
-- Without deviation from the norm, progress is not possible.
| |