lkml.org 
[lkml]   [2013]   [Jan]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Date
    SubjectRe: Oops on aoe module removal
    On Jan 3, 2013, at 3:50 PM, Ed Cashin wrote:

    > On Jan 3, 2013, at 2:57 PM, Ed Cashin wrote:
    >
    >> On Jan 3, 2013, at 2:45 PM, Jens Axboe wrote:
    >>
    >>> On 2013-01-03 20:28, Ed Cashin wrote:
    >>>> Lines: 75
    >>>>
    >>>> On Thu, Jan 03, 2013 at 12:15:35PM -0600, Ed Cashin wrote:
    >>>> ...
    >>>>>>>>>> On Jan 3, 2013, at 8:25 AM, Josh Boyer wrote:
    >>>> ...
    >>>>>>>>>>> [699170.611997] aoe: AoE v47 initialised.
    >>>> ...
    >>>>>>>>>>> [699231.308319] WARNING: at lib/list_debug.c:62 __list_del_entry+0x82/0xd0()
    >>>>>>>>>>> [699231.312031] Hardware name: S5000VSA
    >>>>>>>>>>> [699231.315658] list_del corruption. next->prev should be ffff880009fa37e8, but was ffffffff81c79c00
    >>>>>>>>>>> [699231.319352] Modules linked in: aoe(-) ip6table_filter ip6_tables ebtable_nat ebtables lockd sunrpc bridge 8021q garp stp llc vfat fat binfmt_misc iTCO_wdt iTCO_vendor_support vhost_net lpc_ich radeon tun macvtap mfd_core serio_raw coretemp i2c_algo_bit ttm i5000_edac macvlan drm_kms_helper e1000e edac_core microcode i5k_amb shpchp i2c_i801 drm kvm_intel i2c_core kvm ioatdma dca raid1
    >>>>>>>>>>> [699231.336259] Pid: 8584, comm: modprobe Not tainted 3.6.11-1.fc17.x86_64 #1
    >>>>>>>>>>> [699231.340561] Call Trace:
    >>>>>>>>>>> [699231.344865] [<ffffffff8105c8ef>] warn_slowpath_common+0x7f/0xc0
    >>>>>>>>>>> [699231.349212] [<ffffffff8105c9e6>] warn_slowpath_fmt+0x46/0x50
    >>>>>>>>>>> [699231.353595] [<ffffffff812eee52>] __list_del_entry+0x82/0xd0
    >>>>>>>>>>> [699231.357954] [<ffffffff812eeeb1>] list_del+0x11/0x40
    >>>>>>>>>>> [699231.362319] [<ffffffff812f6458>] percpu_counter_destroy+0x28/0x50
    >>>>>>>>>>> [699231.366712] [<ffffffff8114c513>] bdi_destroy+0x43/0x140
    >>>>>>>>>>> [699231.371127] [<ffffffff812be20c>] blk_release_queue+0x8c/0xc0
    >>>>>>>>>>> [699231.375454] [<ffffffff812dc322>] kobject_cleanup+0x82/0x1b0
    >>>>>>>>>>> [699231.379675] [<ffffffff812dc1ab>] kobject_put+0x2b/0x60
    >>>>>>>>>>> [699231.383851] [<ffffffff812b80a5>] blk_put_queue+0x15/0x20
    >>>>>>>>>>> [699231.387899] [<ffffffff812bc659>] blk_cleanup_queue+0xc9/0xe0
    >>>>>>>>>>> [699231.391794] [<ffffffffa01f53f5>] aoedev_freedev+0x135/0x150 [aoe]
    >>>>>>>>>>> [699231.395668] [<ffffffffa01f59a5>] aoedev_exit+0x65/0x80 [aoe]
    >>>>>>>>>>> [699231.399493] [<ffffffffa01f5afe>] aoe_exit+0x2e/0x40 [aoe]
    >>>>>>>>>>> [699231.403273] [<ffffffff810bdefe>] sys_delete_module+0x16e/0x2d0
    >>>>>>>>>>> [699231.407119] [<ffffffff8161db56>] ? __schedule+0x3c6/0x7a0
    >>>>>>>>>>> [699231.411050] [<ffffffff8119054a>] ? sys_write+0x4a/0x90
    >>>>>>>>>>> [699231.415033] [<ffffffff81627329>] system_call_fastpath+0x16/0x1b
    >>>>>>>>>>> [699231.419117] ---[ end trace 9e1558af1964b569 ]---
    >>>>>>>>>>> [699231.423248] ------------[ cut here ]------------
    >>>>
    >>>> The blk_alloc_queue has already done a bdi_init, so do not bdi_init again in
    >>>> aoeblk_gdalloc.
    >>>>
    >>>> The patch below applies to v3.5.6, with its v47 aoe driver. On my system it
    >>>> eliminates the list_del corruption messages.
    >>>
    >>> Since the patch doesn't apply to current -git, does the problem not
    >>> exist there?
    >>
    >> The original post is about an older kernel with the v47 aoe driver. The current mainline has a v81 aoe driver, so the patch for v3.5.6 isn't expected to apply to the mainline.
    >>
    >> I'm currently investigating the state of the mainline with relation to this issue.
    >
    > I don't see the extra call to bdi_init in aoe driver v81 in the mainline git tree, and
    > I don't see the symptoms under discussion, either, when performing the same
    > test steps.
    >
    > Kernels with aoe v47 can use the patch I just posted.
    >
    > I'm going to go through the stable kernels and check, but I believe kernels after
    > v47 but before commit 0a41409c5180 should apply the fix in 0a41409c5180:
    >
    > commit 0a41409c518083133e79015092585d68915865be
    > Author: Ed Cashin <ecashin@coraid.com>
    > Date: Mon Dec 17 16:03:58 2012 -0800
    >
    > aoe: remove vestigial request queue allocation

    ...

    OK. Looks like 3.7.1 has aoe v50, exhibits the behavior in question, and needs
    the above-mentioned commit 0a41409c5180 from Linus' tree as a fix.

    The 3.6.11 kernel has aoe v47, so it and earlier stable kernels can use the patch
    I posted in this thread.

    I'm sending this to stable@vger.kernel.org to ask Greg, Ben, and other stable
    maintainers (where's the list of stable kernel maintainers?) ...

    Should I send git-generated patches to stable@vger.kernel.org based on each
    linux-stable/linux-3.x,y branch? There are only two cases, really: 3.7.y and others.
    I suppose 3.7.y can cherry pick from the mainline.

    What's the best way to do this?

    --
    Ed Cashin
    ecashin@coraid.com




    \
     
     \ /
      Last update: 2013-01-03 22:41    [W:2.289 / U:0.020 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site