Fix synchronize_irq races with IRQ handler
authorHerbert Xu <herbert@gondor.apana.org.au>
Tue, 23 Oct 2007 03:26:25 +0000 (11:26 +0800)
committerLinus Torvalds <torvalds@woody.linux-foundation.org>
Tue, 23 Oct 2007 16:01:31 +0000 (09:01 -0700)
As it is some callers of synchronize_irq rely on memory barriers
to provide synchronisation against the IRQ handlers.  For example,
the tg3 driver does

tp->irq_sync = 1;
smp_mb();
synchronize_irq();

and then in the IRQ handler:

if (!tp->irq_sync)
netif_rx_schedule(dev, &tp->napi);

Unfortunately memory barriers only work well when they come in
pairs.  Because we don't actually have memory barriers on the
IRQ path, the memory barrier before the synchronize_irq() doesn't
actually protect us.

In particular, synchronize_irq() may return followed by the
result of netif_rx_schedule being made visible.

This patch (mostly written by Linus) fixes this by using spin
locks instead of memory barries on the synchronize_irq() path.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Acked-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
kernel/irq/manage.c

index 80eab7a0420597ec0c27fe7eaefe12a5868674b9..1f314221d534be091d809628b39f12ccba4e997e 100644 (file)
 void synchronize_irq(unsigned int irq)
 {
        struct irq_desc *desc = irq_desc + irq;
+       unsigned int status;
 
        if (irq >= NR_IRQS)
                return;
 
-       while (desc->status & IRQ_INPROGRESS)
-               cpu_relax();
+       do {
+               unsigned long flags;
+
+               /*
+                * Wait until we're out of the critical section.  This might
+                * give the wrong answer due to the lack of memory barriers.
+                */
+               while (desc->status & IRQ_INPROGRESS)
+                       cpu_relax();
+
+               /* Ok, that indicated we're done: double-check carefully. */
+               spin_lock_irqsave(&desc->lock, flags);
+               status = desc->status;
+               spin_unlock_irqrestore(&desc->lock, flags);
+
+               /* Oops, that failed? */
+       } while (status & IRQ_INPROGRESS);
 }
 EXPORT_SYMBOL(synchronize_irq);