lkml.org 
[lkml]   [1999]   [Feb]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
Subject2.0.36 oops after cron spawn?
Date
Pre-apologies if this post doesn't contain enough information... I
researched and did due diligence to the best of my (limited) ability. This
is the first time I've ever seen Linux hose up. Sobering.

Problem: 2.0.36 kernel on frankenstein AMD 486/133 (16MB RAM, 3C905 in
10base mode, unknown 56k modem on com2, unknown video card, hard drive, CD,
floppy, not much else) running as internet access router/IP masq box oopsed
4 times over 20-minute period in the middle of the night, on execution of
weekly cron job. IT people onsite say it "hung" (which I assume means
stopped performing its routing/dialing function) after these oopses and did
not start to "work" again until it was rebooted. Only happened once, box is
working currently post-reboot, and no further context information is
available.

Processes running: normal processes spawned by stock RH5.2 install, plus
diald and samba.

Here's the output of ksymoops on the syslog file:

Options used: -V -O -K -L -m System.map

Jan 31 04:02:33 loser kernel: CPU: 0
Jan 31 04:02:33 loser kernel: EIP: 0010:[put_super+96/104]
Jan 31 04:02:33 loser kernel: EFLAGS: 00010202
Jan 31 04:02:33 loser kernel: eax: 73007373 ebx: 00000301 ecx: 00b8d098
edx: 000003f0
Jan 31 04:02:33 loser kernel: esi: 00000301 edi: 000010f1 ebp: 00000400
esp: 006fae30
Jan 31 04:02:33 loser kernel: ds: 0018 es: 0018 fs: 002b gs: 002b
ss: 0018
Jan 31 04:02:33 loser kernel: Process find (pid: 1341, process nr: 33,
stackpage=006fa000)
Jan 31 04:02:33 loser kernel: Stack: 000010f1 00000301 00000400 000010f1
001fe394 000003f0 00128f66 00000301
Jan 31 04:02:33 loser kernel: 000010f1 00000400 000010f1 00000002
00b8d000 00000000 00000001 00166674
Jan 31 04:02:33 loser kernel: 00000301 000010f1 00000400 00000008
00000002 006faf44 00b8d000 00000002
Jan 31 04:02:33 loser kernel: Call Trace: [sys_mount+278/776]
[raw_scan_root+92/116] [msdos_format_name+84/856] [parse_options+621/2220]
[locks_delete_lock+90/124] [put_unused_buffer_head+93/112]
[elf_core_dump+1623/2616]
Jan 31 04:02:33 loser kernel: [elf_core_dump+1288/2616]
[die_if_kernel+153/680]
Jan 31 04:02:33 loser kernel: Code: 39 38 75 28 66 39 58 04 75 22 39 68 20
74 26 56 e8 77 f8 ff

Code: 00000000 Before first symbol 00000000 <_EIP>:
Code: 00000000 Before first symbol 0: 39 38 cmpl
%edi,(%eax)
Code: 00000002 Before first symbol 2: 75 28 jne
2c <_EIP+0x2c> 0000002c Before first symbol
Code: 00000004 Before first symbol 4: 66 39 58 04 cmpw
%bx,0x4(%eax)
Code: 00000008 Before first symbol 8: 75 22 jne
2c <_EIP+0x2c> 0000002c Before first symbol
Code: 0000000a Before first symbol a: 39 68 20 cmpl
%ebp,0x20(%eax)
Code: 0000000d Before first symbol d: 74 26 je
35 <_EIP+0x35> 00000035 Before first symbol
Code: 0000000f Before first symbol f: 56
pushl %esi
Code: 00000010 Before first symbol 10: e8 77 f8 ff 00 call
fff88c <_EIP+0xfff88c> 00fff88c <END_OF_CODE+dee0b4/????>

Jan 31 04:22:08 loser kernel: Unable to handle kernel NULL pointer
dereference at virtual address c0000007
Jan 31 04:22:08 loser kernel: current->tss.cr3 = 00f02000, `r3 = 00f02000
Jan 31 04:22:08 loser kernel: *pde = 00102067
Jan 31 04:22:08 loser kernel: Oops: 0000
Jan 31 04:22:08 loser kernel: CPU: 0
Jan 31 04:22:08 loser kernel: EIP: 0010:[put_super+96/104]
Jan 31 04:22:08 loser kernel: EFLAGS: 00010202
Jan 31 04:22:08 loser kernel: eax: 00000007 ebx: 007d0301 ecx: 00000400
edx: 000003a0
Jan 31 04:22:08 loser kernel: esi: 00000301 edi: 000130a1 ebp: 00000400
esp: 0073ce68
Jan 31 04:22:08 loser kernel: ds: 0018 es: 0018 fs: 002b gs: 002b
ss: 0018
Jan 31 04:22:08 loser kernel: Process gawk (pid: 1370, process nr: 29,
stackpage=0073c000)
Jan 31 04:22:08 loser kernel: Stack: 007df798 000130a1 00000301 00000002
00000000 000003a0 00129dbd 00000301
Jan 31 04:22:08 loser kernel: 000130a1 00000400 0073cf18 00000024
00000000 00ef0100 000130a1 00000301
Jan 31 04:22:08 loser kernel: 00000100 007df898 00fb0301 00000000
007df898 007df818 00fb9298 0073cf14
Jan 31 04:22:08 loser kernel: Call Trace: [block_read+893/1464]
[sys_newstat+67/84] [sys_mremap+224/772] [sys_mremap+457/772] [kfree+0/300]
[getblk+676/952] [die_if_kernel+153/680]
Jan 31 04:22:08 loser kernel: Code: 39 38 75 28 66 39 58 04 75 22 39 68 20
74 26 56 e8 77 f8 ff

Code: 00000000 Before first symbol 00000000 <_EIP>:
Code: 00000000 Before first symbol 0: 39 38 cmpl
%edi,(%eax)
Code: 00000002 Before first symbol 2: 75 28 jne
2c <_EIP+0x2c> 0000002c Before first symbol
Code: 00000004 Before first symbol 4: 66 39 58 04 cmpw
%bx,0x4(%eax)
Code: 00000008 Before first symbol 8: 75 22 jne
2c <_EIP+0x2c> 0000002c Before first symbol
Code: 0000000a Before first symbol a: 39 68 20 cmpl
%ebp,0x20(%eax)
Code: 0000000d Before first symbol d: 74 26 je
35 <_EIP+0x35> 00000035 Before first symbol
Code: 0000000f Before first symbol f: 56
pushl %esi
Code: 00000010 Before first symbol 10: e8 77 f8 ff 00 call
fff88c <_EIP+0xfff88c> 00fff88c <END_OF_CODE+dee0b4/????>

Jan 31 04:22:19 loser kernel: Unable to handle kernel paging request at
virtual address e0013d01
Jan 31 04:22:19 loser kernel: current->tss.cr3 = 00b55000, `r3 = 00b55000
Jan 31 04:22:19 loser kernel: *pde = 00000000
Jan 31 04:22:19 loser kernel: Oops: 0000
Jan 31 04:22:19 loser kernel: CPU: 0
Jan 31 04:22:19 loser kernel: EIP: 0010:[put_super+96/104]
Jan 31 04:22:19 loser kernel: EFLAGS: 00010202
Jan 31 04:22:19 loser kernel: eax: 20013d01 ebx: 00440301 ecx: 00000400
edx: 000003a0
Jan 31 04:22:19 loser kernel: esi: 00000301 edi: 000170a1 ebp: 00000400
esp: 00756e8c
Jan 31 04:22:19 loser kernel: ds: 0018 es: 0018 fs: 002b gs: 002b
ss: 0018
Jan 31 04:22:19 loser kernel: Process gawk (pid: 1374, process nr: 29,
stackpage=00756000)
Jan 31 04:22:19 loser kernel: Stack: 00441d18 000170a1 00000301 00000000
00000000 000003a0 00129dbd 00000301
Jan 31 04:22:19 loser kernel: 000170a1 00000400 00756f3c 00000004
00000000 0047cc00 000170a1 00000301
Jan 31 04:22:19 loser kernel: 00000100 00441d18 00000301 0048a002
00990198 00167b0f 006a1200 00756f38
Jan 31 04:22:19 loser kernel: Call Trace: [block_read+893/1464]
[rename_same_dir+379/580] [sys_newstat+67/84] [free_area_pages+102/348]
[free_area_pages+337/348] [getblk+676/952] [die_if_kernel+153/680]
Jan 31 04:22:19 loser kernel: Code: 39 38 75 28 66 39 58 04 75 22 39 68 20
74 26 56 e8 77 f8 ff

Code: 00000000 Before first symbol 00000000 <_EIP>:
Code: 00000000 Before first symbol 0: 39 38 cmpl
%edi,(%eax)
Code: 00000002 Before first symbol 2: 75 28 jne
2c <_EIP+0x2c> 0000002c Before first symbol
Code: 00000004 Before first symbol 4: 66 39 58 04 cmpw
%bx,0x4(%eax)
Code: 00000008 Before first symbol 8: 75 22 jne
2c <_EIP+0x2c> 0000002c Before first symbol
Code: 0000000a Before first symbol a: 39 68 20 cmpl
%ebp,0x20(%eax)
Code: 0000000d Before first symbol d: 74 26 je
35 <_EIP+0x35> 00000035 Before first symbol
Code: 0000000f Before first symbol f: 56
pushl %esi
Code: 00000010 Before first symbol 10: e8 77 f8 ff 00 call
fff88c <_EIP+0xfff88c> 00fff88c <END_OF_CODE+dee0b4/????>

Jan 31 04:22:33 loser kernel: CPU: 0
Jan 31 04:22:33 loser kernel: EIP: 0010:[put_super+96/104]
Jan 31 04:22:33 loser kernel: EFLAGS: 00010206
Jan 31 04:22:33 loser kernel: eax: 756f2065 ebx: 00e70301 ecx: 00000400
edx: 000003a0
Jan 31 04:22:33 loser kernel: esi: 00000301 edi: 000250a1 ebp: 00000400
esp: 00b17e68
Jan 31 04:22:33 loser kernel: ds: 0018 es: 0018 fs: 002b gs: 002b
ss: 0018
Jan 31 04:22:33 loser kernel: Process gawk (pid: 1388, process nr: 29,
stackpage=00b17000)
Jan 31 04:22:33 loser kernel: Stack: 00e7a118 000250a1 00000301 00000000
00000000 000003a0 00129dbd 00000301
Jan 31 04:22:33 loser kernel: 000250a1 00000400 00b17f18 0000001c
00000000 0072b400 000250a1 00000301
Jan 31 04:22:33 loser kernel: 00000100 00e7a118 00440301 00000000
000250a4 001664a5 00441818 00b17f14
Jan 31 04:22:33 loser kernel: Call Trace: [block_read+893/1464]
[raw_scan_sector+269/640] [sys_newstat+67/84] [sys_mremap+224/772]
[sys_mremap+457/772] [kfree+0/300] [getblk+676/952]
Jan 31 04:22:33 loser kernel: [die_if_kernel+153/680]
Jan 31 04:22:33 loser kernel: Code: 39 38 75 28 66 39 58 04 75 22 39 68 20
74 26 56 e8 77 f8 ff

Code: 00000000 Before first symbol 00000000 <_EIP>:
Code: 00000000 Before first symbol 0: 39 38 cmpl
%edi,(%eax)
Code: 00000002 Before first symbol 2: 75 28 jne
2c <_EIP+0x2c> 0000002c Before first symbol
Code: 00000004 Before first symbol 4: 66 39 58 04 cmpw
%bx,0x4(%eax)
Code: 00000008 Before first symbol 8: 75 22 jne
2c <_EIP+0x2c> 0000002c Before first symbol
Code: 0000000a Before first symbol a: 39 68 20 cmpl
%ebp,0x20(%eax)
Code: 0000000d Before first symbol d: 74 26 je
35 <_EIP+0x35> 00000035 Before first symbol
Code: 0000000f Before first symbol f: 56
pushl %esi
Code: 00000010 Before first symbol 10: e8 77 f8 ff 00 call
fff88c <_EIP+0xfff88c> 00fff88c <END_OF_CODE+dee0b4/????>

------

Here's snippet of syslog messages file which produced prior ksymoops stuff:

Jan 31 04:02:00 loser syslogd 1.3-3: restart.
Jan 31 04:02:00 loser syslogd 1.3-3: restart.
Jan 31 04:02:01 loser syslogd 1.3-3: restart.
Jan 31 04:02:01 loser syslogd 1.3-3: restart.
Jan 31 04:02:01 loser PAM_pwdb[1306]: (su) session opened for user nobody by
(uid=99)
Jan 31 04:02:33 loser kernel: general protection: 0000
Jan 31 04:02:33 loser kernel: CPU: 0
Jan 31 04:02:33 loser kernel: EIP: 0010:[put_super+96/104]
Jan 31 04:02:33 loser kernel: EFLAGS: 00010202
Jan 31 04:02:33 loser kernel: eax: 73007373 ebx: 00000301 ecx: 00b8d098
edx: 000003f0
Jan 31 04:02:33 loser kernel: esi: 00000301 edi: 000010f1 ebp: 00000400
esp: 006fae30
Jan 31 04:02:33 loser kernel: ds: 0018 es: 0018 fs: 002b gs: 002b
ss: 0018
Jan 31 04:02:33 loser kernel: Process find (pid: 1341, process nr: 33,
stackpage=006fa000)
Jan 31 04:02:33 loser kernel: Stack: 000010f1 00000301 00000400 000010f1
001fe394 000003f0 00128f66 00000301
Jan 31 04:02:33 loser kernel: 000010f1 00000400 000010f1 00000002
00b8d000 00000000 00000001 00166674
Jan 31 04:02:33 loser kernel: 00000301 000010f1 00000400 00000008
00000002 006faf44 00b8d000 00000002
Jan 31 04:02:33 loser kernel: Call Trace: [sys_mount+278/776]
[raw_scan_root+92/116] [msdos_format_name+84/856] [parse_options+621/2220]
[locks_delete_lock+90/124] [put_unused_buffer_head+93/112]
[elf_core_dump+1623/2616]
Jan 31 04:02:33 loser kernel: [elf_core_dump+1288/2616]
[die_if_kernel+153/680]
Jan 31 04:02:33 loser kernel: Code: 39 38 75 28 66 39 58 04 75 22 39 68 20
74 26 56 e8 77 f8 ff
Jan 31 04:02:36 loser PAM_pwdb[1306]: (su) session closed for user nobody
Jan 31 04:22:08 loser kernel: Unable to handle kernel NULL pointer
dereference at virtual address c0000007
Jan 31 04:22:08 loser kernel: current->tss.cr3 = 00f02000, `r3 = 00f02000
Jan 31 04:22:08 loser kernel: *pde = 00102067
Jan 31 04:22:08 loser kernel: *pte = 00000000
Jan 31 04:22:08 loser kernel: Oops: 0000
Jan 31 04:22:08 loser kernel: CPU: 0
Jan 31 04:22:08 loser kernel: EIP: 0010:[put_super+96/104]
Jan 31 04:22:08 loser kernel: EFLAGS: 00010202
Jan 31 04:22:08 loser kernel: eax: 00000007 ebx: 007d0301 ecx: 00000400
edx: 000003a0
Jan 31 04:22:08 loser kernel: esi: 00000301 edi: 000130a1 ebp: 00000400
esp: 0073ce68
Jan 31 04:22:08 loser kernel: ds: 0018 es: 0018 fs: 002b gs: 002b
ss: 0018
Jan 31 04:22:08 loser kernel: Process gawk (pid: 1370, process nr: 29,
stackpage=0073c000)
Jan 31 04:22:08 loser kernel: Stack: 007df798 000130a1 00000301 00000002
00000000 000003a0 00129dbd 00000301
Jan 31 04:22:08 loser kernel: 000130a1 00000400 0073cf18 00000024
00000000 00ef0100 000130a1 00000301
Jan 31 04:22:08 loser kernel: 00000100 007df898 00fb0301 00000000
007df898 007df818 00fb9298 0073cf14
Jan 31 04:22:08 loser kernel: Call Trace: [block_read+893/1464]
[sys_newstat+67/84] [sys_mremap+224/772] [sys_mremap+457/772] [kfree+0/300]
[getblk+676/952] [die_if_kernel+153/680]
Jan 31 04:22:08 loser kernel: Code: 39 38 75 28 66 39 58 04 75 22 39 68 20
74 26 56 e8 77 f8 ff
Jan 31 04:22:19 loser kernel: Unable to handle kernel paging request at
virtual address e0013d01
Jan 31 04:22:19 loser kernel: current->tss.cr3 = 00b55000, `r3 = 00b55000
Jan 31 04:22:19 loser kernel: *pde = 00000000
Jan 31 04:22:19 loser kernel: Oops: 0000
Jan 31 04:22:19 loser kernel: CPU: 0
Jan 31 04:22:19 loser kernel: EIP: 0010:[put_super+96/104]
Jan 31 04:22:19 loser kernel: EFLAGS: 00010202
Jan 31 04:22:19 loser kernel: eax: 20013d01 ebx: 00440301 ecx: 00000400
edx: 000003a0
Jan 31 04:22:19 loser kernel: esi: 00000301 edi: 000170a1 ebp: 00000400
esp: 00756e8c
Jan 31 04:22:19 loser kernel: ds: 0018 es: 0018 fs: 002b gs: 002b
ss: 0018
Jan 31 04:22:19 loser kernel: Process gawk (pid: 1374, process nr: 29,
stackpage=00756000)
Jan 31 04:22:19 loser kernel: Stack: 00441d18 000170a1 00000301 00000000
00000000 000003a0 00129dbd 00000301
Jan 31 04:22:19 loser kernel: 000170a1 00000400 00756f3c 00000004
00000000 0047cc00 000170a1 00000301
Jan 31 04:22:19 loser kernel: 00000100 00441d18 00000301 0048a002
00990198 00167b0f 006a1200 00756f38
Jan 31 04:22:19 loser kernel: Call Trace: [block_read+893/1464]
[rename_same_dir+379/580] [sys_newstat+67/84] [free_area_pages+102/348]
[free_area_pages+337/348] [getblk+676/952] [die_if_kernel+153/680]
Jan 31 04:22:19 loser kernel: Code: 39 38 75 28 66 39 58 04 75 22 39 68 20
74 26 56 e8 77 f8 ff
Jan 31 04:22:33 loser kernel: general protection: 0000
Jan 31 04:22:33 loser kernel: CPU: 0
Jan 31 04:22:33 loser kernel: EIP: 0010:[put_super+96/104]
Jan 31 04:22:33 loser kernel: EFLAGS: 00010206
Jan 31 04:22:33 loser kernel: eax: 756f2065 ebx: 00e70301 ecx: 00000400
edx: 000003a0
Jan 31 04:22:33 loser kernel: esi: 00000301 edi: 000250a1 ebp: 00000400
esp: 00b17e68
Jan 31 04:22:33 loser kernel: ds: 0018 es: 0018 fs: 002b gs: 002b
ss: 0018
Jan 31 04:22:33 loser kernel: Process gawk (pid: 1388, process nr: 29,
stackpage=00b17000)
Jan 31 04:22:33 loser kernel: Stack: 00e7a118 000250a1 00000301 00000000
00000000 000003a0 00129dbd 00000301
Jan 31 04:22:33 loser kernel: 000250a1 00000400 00b17f18 0000001c
00000000 0072b400 000250a1 00000301
Jan 31 04:22:33 loser kernel: 00000100 00e7a118 00440301 00000000
000250a4 001664a5 00441818 00b17f14
Jan 31 04:22:33 loser kernel: Call Trace: [block_read+893/1464]
[raw_scan_sector+269/640] [sys_newstat+67/84] [sys_mremap+224/772]
[sys_mremap+457/772] [kfree+0/300] [getblk+676/952]
Jan 31 04:22:33 loser kernel: [die_if_kernel+153/680]
Jan 31 04:22:33 loser kernel: Code: 39 38 75 28 66 39 58 04 75 22 39 68 20
74 26 56 e8 77 f8 ff
Jan 31 13:03:33 loser diald[300]: Running connect (pid = 1463).
Jan 31 13:03:34 loser chat[1463]: abort on (BUSY)
Jan 31 13:03:34 loser chat[1463]: abort on (ERROR)
Jan 31 13:03:34 loser chat[1463]: abort on (NO CARRIER)
Jan 31 13:03:34 loser chat[1463]: abort on (NO DIALTONE)
Jan 31 13:03:34 loser chat[1463]: abort on (Invalid Login)
Jan 31 13:03:34 loser chat[1463]: abort on (Login incorrect)
Jan 31 13:03:34 loser chat[1463]: abort on (NO DIAL TONE)
Jan 31 13:03:34 loser chat[1463]: send (ATZ^M)
Jan 31 13:03:34 loser chat[1463]: expect (OK)
Jan 31 13:03:34 loser chat[1463]: ATZ^M^M
Jan 31 13:03:34 loser chat[1463]: OK
Jan 31 13:03:34 loser chat[1463]: -- got it
Jan 31 13:03:34 loser chat[1463]: send (ATDT7229490^M)
Jan 31 13:03:34 loser chat[1463]: expect (CONNECT)
Jan 31 13:03:34 loser chat[1463]: ^M
Jan 31 13:03:59 loser chat[1463]: ATDT7229490^M^M
Jan 31 13:03:59 loser chat[1463]: CONNECT
Jan 31 13:03:59 loser chat[1463]: -- got it
Jan 31 13:03:59 loser chat[1463]: send (^M)
Jan 31 13:03:59 loser chat[1463]: timeout set to 5 seconds
Jan 31 13:03:59 loser chat[1463]: expect (~)
Jan 31 13:03:59 loser chat[1463]: 45333/ARQ/V90/LAPM/V42BIS^M
Jan 31 13:04:04 loser chat[1463]: ~
Jan 31 13:04:04 loser chat[1463]: -- got it
Jan 31 13:04:04 loser chat[1463]: send (^M)
Jan 31 13:04:04 loser diald[300]: Running pppd (pid = 1464).
Feb 5 18:55:22 loser syslogd 1.3-3: restart.
Feb 5 18:55:22 loser kernel: klogd 1.3-3, log source = /proc/kmsg started.
Feb 5 18:55:23 loser kernel: Loaded 4253 symbols from /boot/System.map.
Feb 5 18:55:23 loser kernel: Symbols match kernel version 2.0.36.
Feb 5 18:55:23 loser kernel: No module symbols loaded.
Feb 5 18:55:23 loser kernel: Memory: sized by int13 088h
Feb 5 18:55:23 loser kernel: Console: 16 point font, 400 scans
....
etc

Any help is greatly appreciated, even if only to tell me whom to consult
about the problem or to let me know what other info to provide.

Many thanks,


Chris McDonough

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:50    [W:0.083 / U:2.324 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site