Messages in this thread | | | Date | Sat, 28 Jan 2012 20:20:57 +0000 (GMT) | From | m ki <> | Subject | PROBLEM: slow nfs transfer on crypted partition (dm_crypt/LUKS) with the Kernel 3.x branch |
| |
[1.] One line summary of the problem: slow nfs transfer on crypted partition (dm_crypt/LUKS/ext4) with the Kernel 3.x branch
[2.] Full description of the problem/report: Hello, I would like to submit a 'limitation' appearing after upgrading my Ubuntu_Server_11.04 (Kernel 2.6.38-13#52) to Ubuntu_Server_11.10 (Kernel 3.0.0-15#26) With this new Kernel, performances have dropped when accessing a crypted partition
A test using nfs4 to access a crypted partition on the server: *Svr Kernel 3.0.0-15#26 : NFS (R/W) = 30 MB/s / 33 MB/s <-------- *Svr Kernel 2.6.38-13#52 : NFS (R/W) = 81 MB/s / 53 MB/s
As you can see, there is a big difference With Kernel 3.x, the flow is always constant at 30MB; there's clearly a limitation (The client can be a 2.6 or 3.x Kernel, results are the same)
For comparison, without nfs: doing a transfer with a 2nd HD on the server: *Svr Kernel 3.0.0-15#26 : cp (from/to) the crypted partiton = 64 MB/s / 57 MB/s *Svr Kernel 2.6.38-13#52 : cp (from/to) the crypted partiton = 61 MB/s / 60 MB/s Using dm_crypt without nfs, there's no difference between the 2 Kernels
And using nfs on a non-crypted partition (same HD of the crypted one) *Svr Kernel 3.0.0-15#26 : NFS (R/W) = 92 MB/s / 77 MB/s *Svr Kernel 2.6.38-13#52 : NFS (R/W) = 111 MB/s / 74 MB/s Using nfs without dm_crypt, there's no big difference between the 2 Kernels too
Apparently, there is a bad interaction between nfs and dm_crypt when running a 3.x Kernel (cache problem?)
The server is a fileserver using default parameters, except for exportfs that use the option async rsize and wsize are set to 131072b. That seems to be the maximum accepted now; in the past the maximum payload 1048576b was possible No paquets are lost during nfs transfers. nfsd is running 8 threads.
I don't know how manage the module dm_crypt. (See parameters in [7.7])
I've build another server with a debian Wheezy (Kernel 3.1.0-1) and another one with a Fedora 16 (Kernel 3.2.1-3). Both obtained the same results.
That's all I can say for the moment. Feel free to ask more informations / tests.
Please, CC me for replying. I'm reading the Kernel list using the newsgroup linux.kernel, but I don't think I can reply directly to it
Regards ! Mickael
[3.] Keywords (i.e., modules, networking, kernel): nfs4, dm_crypt, kernel 3.x
[4.] Kernel version (from /proc/version): Linux version 3.0.0-15-server (buildd@crested) (gcc version 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3) ) #26-Ubuntu SMP Fri Jan 20 19:07:39 UTC 2012
[5.] Output of Oops.. message (if applicable) with symbolic information resolved (see Documentation/oops-tracing.txt) No Oops or bad logs reported
[7.] Environment CPU AMD Athlon64 3500+ (1core) 1GB RAM mobo Asus A8N nForce4 eth 1GB nForce4 HD WD 2To Green (sata_nv)
[7.1.] Software (add the output of the ver_linux script here) Linux 3.0.0-15-server #26-Ubuntu SMP Fri Jan 20 19:07:39 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux Gnu C ./ver_linux: ./ver_linux: 20: ld: not found binutils util-linux 2.19.1 mount support module-init-tools 3.16 e2fsprogs 1.41.14 PPP 2.4.5 Linux C Library 2.13 Dynamic linker (ldd) 2.13 Procps 3.2.8 Net-tools 1.60 Kbd 1.15.2 Sh-utils 8.5 Modules Loaded dm_crypt speedstep_lib nfsd nfs lockd fscache auth_rpcgss nfs_acl sunrpc psmouse serio_raw edac_core k8temp edac_mce_amd i2c_nforce2 asus_atk0110 lp parport nouveau ttm drm_kms_helper skge forcedeth drm i2c_algo_bit mxm_wmi wmi video sata_nv pata_amd
[7.2.] Processor information (from /proc/cpuinfo): processor : 0 vendor_id : AuthenticAMD cpu family : 15 model : 47 model name : AMD Athlon(tm) 64 Processor 3500+ stepping : 2 cpu MHz : 1000.000 cache size : 512 KB fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt lm 3dnowext 3dnow up rep_good nopl pni lahf_lm bogomips : 2010.12 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: ts fid vid ttp tm stc
[7.3.] Module information (from /proc/modules): dm_crypt 23199 1 - Live 0xffffffffa0219000 speedstep_lib 13195 0 - Live 0xffffffffa01a1000 nfsd 322392 13 - Live 0xffffffffa02ae000 nfs 354737 0 - Live 0xffffffffa0240000 lockd 86161 2 nfsd,nfs, Live 0xffffffffa0221000 fscache 61593 1 nfs, Live 0xffffffffa0208000 auth_rpcgss 53320 2 nfsd,nfs, Live 0xffffffffa01f9000 nfs_acl 12883 2 nfsd,nfs, Live 0xffffffffa017b000 sunrpc 240955 20 nfsd,nfs,lockd,auth_rpcgss,nfs_acl, Live 0xffffffffa01bd000 psmouse 73882 0 - Live 0xffffffffa01a9000 serio_raw 13166 0 - Live 0xffffffffa00a3000 edac_core 53746 0 - Live 0xffffffffa0192000 k8temp 13057 0 - Live 0xffffffffa0176000 edac_mce_amd 23709 0 - Live 0xffffffffa016f000 i2c_nforce2 13058 0 - Live 0xffffffffa0012000 asus_atk0110 18078 0 - Live 0xffffffffa0028000 lp 17799 0 - Live 0xffffffffa007d000 parport 46562 1 lp, Live 0xffffffffa0185000 nouveau 728677 1 - Live 0xffffffffa00bc000 ttm 76805 1 nouveau, Live 0xffffffffa00a8000 drm_kms_helper 42558 1 nouveau, Live 0xffffffffa0097000 skge 49902 0 - Live 0xffffffffa006f000 forcedeth 67563 0 - Live 0xffffffffa0085000 drm 236290 3 nouveau,ttm,drm_kms_helper, Live 0xffffffffa0034000 i2c_algo_bit 13423 1 nouveau, Live 0xffffffffa002f000 mxm_wmi 12979 1 nouveau, Live 0xffffffffa001d000 wmi 19256 1 mxm_wmi, Live 0xffffffffa0022000 video 19412 1 nouveau, Live 0xffffffffa0017000 sata_nv 32305 4 - Live 0xffffffffa0009000 pata_amd 14121 0 - Live 0xffffffffa0000000
[7.4.] Loaded driver and hardware information (/proc/ioports, /proc/iomem) :~$ /proc/ioports 0000-ffff : PCI Bus #00 0000-001f : dma1 0020-0021 : pic1 0040-0043 : timer0 0050-0053 : timer1 0060-0060 : keyboard 0064-0064 : keyboard 0070-0073 : rtc0 0080-008f : dma page reg 00a0-00a1 : pic2 00c0-00df : dma2 00f0-00ff : fpu 0170-0177 : 0000:00:06.0 0170-0177 : pata_amd 01f0-01f7 : 0000:00:06.0 01f0-01f7 : pata_amd 0290-0297 : pnp 00:02 0376-0376 : 0000:00:06.0 0376-0376 : pata_amd 03f6-03f6 : 0000:00:06.0 03f6-03f6 : pata_amd 04d0-04d1 : pnp 00:02 0800-087f : pnp 00:02 0960-0967 : 0000:00:08.0 0960-0967 : sata_nv 0970-0977 : 0000:00:07.0 0970-0977 : sata_nv 09e0-09e7 : 0000:00:08.0 09e0-09e7 : sata_nv 09f0-09f7 : 0000:00:07.0 09f0-09f7 : sata_nv 0b60-0b63 : 0000:00:08.0 0b60-0b63 : sata_nv 0b70-0b73 : 0000:00:07.0 0b70-0b73 : sata_nv 0be0-0be3 : 0000:00:08.0 0be0-0be3 : sata_nv 0bf0-0bf3 : 0000:00:07.0 0bf0-0bf3 : sata_nv 0cf8-0cff : PCI conf1 4000-407f : pnp 00:01 4000-4003 : ACPI PM1a_EVT_BLK 4004-4005 : ACPI PM1a_CNT_BLK 4008-400b : ACPI PM_TMR 401c-401c : ACPI PM2_CNT_BLK 4020-4027 : ACPI GPE0_BLK 4080-40ff : pnp 00:01 4400-447f : pnp 00:01 4480-44ff : pnp 00:01 44a0-44af : ACPI GPE1_BLK 4800-487f : pnp 00:01 4880-48ff : pnp 00:01 4c00-4c3f : 0000:00:01.1 4c00-4c3f : nForce2_smbus 4c40-4c7f : 0000:00:01.1 4c40-4c7f : nForce2_smbus a000-afff : PCI Bus 0000:05 a000-a0ff : 0000:05:0c.0 a000-a0ff : skge b000-b007 : 0000:00:0a.0 b000-b007 : forcedeth c400-c40f : 0000:00:08.0 c400-c40f : sata_nv d800-d80f : 0000:00:07.0 d800-d80f : sata_nv dc00-dc1f : 0000:00:01.1 f000-f00f : 0000:00:06.0 f000-f00f : pata_amd
:~$ /proc/iomem 00000000-0000ffff : reserved 00010000-0009f7ff : System RAM 0009f800-0009ffff : reserved 000a0000-000bffff : PCI Bus #00 000c0000-000c7fff : Video ROM 000f0000-000fffff : reserved 000f0000-000fffff : System ROM 00100000-3ffeffff : System RAM 01000000-016129fb : Kernel code 016129fc-01cd157f : Kernel data 01dbc000-01f10fff : Kernel bss 3fff0000-3fff2fff : ACPI Non-volatile Storage 3fff3000-3fffffff : ACPI Tables 40000000-fcffffffff : PCI Bus #00 40000000-400fffff : PCI Bus 0000:05 40000000-4001ffff : 0000:05:0c.0 40100000-401000ff : 0000:00:02.1 40100000-401000ff : ehci_hcd c0000000-c7ffffff : PCI Bus 0000:01 c0000000-c7ffffff : 0000:01:00.0 c8000000-cfffffff : PCI Bus 0000:01 c8000000-cbffffff : 0000:01:00.0 cc000000-ccffffff : 0000:01:00.0 cd000000-cd01ffff : 0000:01:00.0 d0000000-d1ffffff : PCI Bus 0000:05 d1000000-d1003fff : 0000:05:0c.0 d1000000-d1003fff : skge d2000000-d2000fff : 0000:00:0a.0 d2000000-d2000fff : forcedeth d2001000-d2001fff : 0000:00:08.0 d2001000-d2001fff : sata_nv d2002000-d2002fff : 0000:00:07.0 d2002000-d2002fff : sata_nv d2003000-d2003fff : 0000:00:02.0 d2003000-d2003fff : ohci_hcd e0000000-efffffff : PCI MMCONFIG 0000 [bus 00-ff] e0000000-efffffff : reserved e0000000-efffffff : pnp 00:09 feb00000-ffffffff : reserved feb00000-fec0ffff : PCI Bus #00 fec00000-fec003ff : IOAPIC 0 fee00000-feefffff : pnp 00:0a fee00000-fee00fff : Local APIC fefff000-feffffff : pnp 00:0a fff80000-fff80fff : pnp 00:0a fff90000-fffbffff : pnp 00:0a fffed000-fffeffff : pnp 00:0a ffff0000-ffffffff : pnp 00:0a
[7.5.] PCI information ('lspci -vvv' as root) 00:00.0 Memory controller: nVidia Corporation CK804 Memory Controller (rev a3) Subsystem: ASUSTeK Computer Inc. A8N-E Mainboard Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Capabilities: [44] HyperTransport: Slave or Primary Interface Command: BaseUnitID=0 UnitCnt=15 MastHost- DefDir- DUL- Link Control 0: CFlE+ CST- CFE- <LkFail- Init+ EOC- TXO- <CRCErr=0 IsocEn- LSEn- ExtCTL- 64b- Link Config 0: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=16bit DwFcInEn- LWO=16bit DwFcOutEn- Link Control 1: CFlE- CST- CFE- <LkFail+ Init- EOC+ TXO+ <CRCErr=0 IsocEn- LSEn- ExtCTL- 64b- Link Config 1: MLWI=8bit DwFcIn- MLWO=8bit DwFcOut- LWI=8bit DwFcInEn- LWO=8bit DwFcOutEn- Revision ID: 1.03 Link Frequency 0: 1.0GHz Link Error 0: <Prot- <Ovfl- <EOC- CTLTm- Link Frequency Capability 0: 200MHz+ 300MHz+ 400MHz+ 500MHz+ 600MHz+ 800MHz+ 1.0GHz+ 1.2GHz- 1.4GHz- 1.6GHz- Vend- Feature Capability: IsocFC+ LDTSTOP+ CRCTM- ECTLT- 64bA- UIDRD- Link Frequency 1: 200MHz Link Error 1: <Prot- <Ovfl- <EOC- CTLTm- Link Frequency Capability 1: 200MHz- 300MHz- 400MHz- 500MHz- 600MHz- 800MHz- 1.0GHz- 1.2GHz- 1.4GHz- 1.6GHz- Vend- Error Handling: PFlE+ OFlE+ PFE- OFE- EOCFE- RFE- CRCFE- SERRFE- CF- RE- PNFE- ONFE- EOCNFE- RNFE- CRCNFE- SERRNFE- Prefetchable memory behind bridge Upper: 00-00 Bus Number: 00 Capabilities: [e0] HyperTransport: MSI Mapping Enable+ Fixed- Mapping Address Base: 00000000fee00000
00:01.0 ISA bridge: nVidia Corporation CK804 ISA Bridge (rev a3) Subsystem: ASUSTeK Computer Inc. K8N4-E or A8N-E Mainboard Control: I/O+ Mem+ BusMaster+ SpecCycle+ MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0
00:01.1 SMBus: nVidia Corporation CK804 SMBus (rev a2) Subsystem: ASUSTeK Computer Inc. K8N4-E or A8N-E Mainboard Control: I/O+ Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Interrupt: pin A routed to IRQ 10 Region 0: I/O ports at dc00 [size=32] Region 4: I/O ports at 4c00 [size=64] Region 5: I/O ports at 4c40 [size=64] Capabilities: [44] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: nForce2_smbus Kernel modules: i2c-nforce2
00:09.0 PCI bridge: nVidia Corporation CK804 PCI Bridge (rev a2) (prog-if 01 [Subtractive decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- Status: Cap- 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Bus: primary=00, secondary=05, subordinate=05, sec-latency=128 I/O behind bridge: 0000a000-0000afff Memory behind bridge: d0000000-d1ffffff Prefetchable memory behind bridge: 40000000-400fffff Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr+ DiscTmrStat- DiscTmrSERREn-
00:0a.0 Bridge: nVidia Corporation CK804 Ethernet Controller (rev a3) Subsystem: ASUSTeK Computer Inc. K8N4-E or A8N-E Mainboard Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx+ Latency: 0 (250ns min, 5000ns max) Interrupt: pin A routed to IRQ 23 Region 0: Memory at d2000000 (32-bit, non-prefetchable) [size=4K] Region 1: I/O ports at b000 [size=8] Capabilities: [44] Power Management version 2 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable+ DSel=0 DScale=0 PME- Kernel driver in use: forcedeth Kernel modules: forcedeth
00:0b.0 PCI bridge: nVidia Corporation CK804 PCIE Bridge (rev a3) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 32 bytes Bus: primary=00, secondary=04, subordinate=04, sec-latency=0 I/O behind bridge: 0000f000-00000fff Memory behind bridge: fff00000-000fffff Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [48] MSI: Enable+ Count=1/2 Maskable- 64bit+ Address: 00000000fee0100c Data: 4129 Capabilities: [58] HyperTransport: MSI Mapping Enable- Fixed- Mapping Address Base: 00000000fee00000 Capabilities: [80] Express (v1) Root Port (Slot+), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L1 <4us ExtTag- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #3, Speed 2.5GT/s, Width x2, ASPM L0s, Latency L0 <512ns, L1 <4us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #8, PowerLimit 25.000W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Off, PwrInd On, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock- Changed: MRL- PresDet- LinkState- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- Capabilities: [100 v1] Virtual Channel Caps: LPEVC=1 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32+ WRR64- WRR128- Ctrl: ArbSelect=WRR32 Status: InProgress- Port Arbitration Table [1a0] <?> VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- VC1: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable- ID=1 ArbSelect=Fixed TC/VC=00 Status: NegoPending- InProgress- Kernel driver in use: pcieport Kernel modules: shpchp
00:0c.0 PCI bridge: nVidia Corporation CK804 PCIE Bridge (rev a3) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 32 bytes Bus: primary=00, secondary=03, subordinate=03, sec-latency=0 I/O behind bridge: 0000f000-00000fff Memory behind bridge: fff00000-000fffff Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [48] MSI: Enable+ Count=1/2 Maskable- 64bit+ Address: 00000000fee0100c Data: 4141 Capabilities: [58] HyperTransport: MSI Mapping Enable- Fixed- Mapping Address Base: 00000000fee00000 Capabilities: [80] Express (v1) Root Port (Slot+), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L1 <4us ExtTag- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #2, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <512ns, L1 <4us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #4, PowerLimit 10.000W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Off, PwrInd On, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock- Changed: MRL- PresDet- LinkState- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- Capabilities: [100 v1] Virtual Channel Caps: LPEVC=1 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32+ WRR64- WRR128- Ctrl: ArbSelect=WRR32 Status: InProgress- Port Arbitration Table [1a0] <?> VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- VC1: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable- ID=1 ArbSelect=Fixed TC/VC=00 Status: NegoPending- InProgress- Kernel driver in use: pcieport Kernel modules: shpchp
00:0d.0 PCI bridge: nVidia Corporation CK804 PCIE Bridge (rev a3) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 32 bytes Bus: primary=00, secondary=02, subordinate=02, sec-latency=0 I/O behind bridge: 0000f000-00000fff Memory behind bridge: fff00000-000fffff Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [48] MSI: Enable+ Count=1/2 Maskable- 64bit+ Address: 00000000fee0100c Data: 4149 Capabilities: [58] HyperTransport: MSI Mapping Enable- Fixed- Mapping Address Base: 00000000fee00000 Capabilities: [80] Express (v1) Root Port (Slot+), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L1 <4us ExtTag- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #1, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <512ns, L1 <4us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #2, PowerLimit 10.000W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Off, PwrInd On, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock- Changed: MRL- PresDet- LinkState- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- Capabilities: [100 v1] Virtual Channel Caps: LPEVC=1 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32+ WRR64- WRR128- Ctrl: ArbSelect=WRR32 Status: InProgress- Port Arbitration Table [1a0] <?> VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- VC1: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable- ID=1 ArbSelect=Fixed TC/VC=00 Status: NegoPending- InProgress- Kernel driver in use: pcieport Kernel modules: shpchp
00:0e.0 PCI bridge: nVidia Corporation CK804 PCIE Bridge (rev a3) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 32 bytes Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 0000f000-00000fff Memory behind bridge: c8000000-cfffffff Prefetchable memory behind bridge: 00000000c0000000-00000000c7ffffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA+ MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [48] MSI: Enable+ Count=1/2 Maskable- 64bit+ Address: 00000000fee0100c Data: 4151 Capabilities: [58] HyperTransport: MSI Mapping Enable- Fixed- Mapping Address Base: 00000000fee00000 Capabilities: [80] Express (v1) Root Port (Slot+), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L1 <4us ExtTag- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ UncorrErr- FatalErr+ UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x16, ASPM L0s, Latency L0 <512ns, L1 <4us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #1, PowerLimit 75.000W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Off, PwrInd On, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock- Changed: MRL- PresDet- LinkState- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- Capabilities: [100 v1] Virtual Channel Caps: LPEVC=1 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32+ WRR64- WRR128- Ctrl: ArbSelect=WRR32 Status: InProgress- Port Arbitration Table [1a0] <?> VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- VC1: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable- ID=1 ArbSelect=Fixed TC/VC=00 Status: NegoPending- InProgress- Kernel driver in use: pcieport Kernel modules: shpchp
00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Capabilities: [80] HyperTransport: Host or Secondary Interface Command: WarmRst+ DblEnd- DevNum=0 ChainSide- HostHide+ Slave- <EOCErr- DUL- Link Control: CFlE- CST- CFE- <LkFail- Init+ EOC- TXO- <CRCErr=0 IsocEn- LSEn- ExtCTL- 64b- Link Config: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=16bit DwFcInEn- LWO=16bit DwFcOutEn- Revision ID: 1.02 Link Frequency: 1.0GHz Link Error: <Prot- <Ovfl- <EOC- CTLTm- Link Frequency Capability: 200MHz+ 300MHz- 400MHz+ 500MHz- 600MHz+ 800MHz+ 1.0GHz+ 1.2GHz- 1.4GHz- 1.6GHz- Vend- Feature Capability: IsocFC- LDTSTOP+ CRCTM- ECTLT- 64bA- UIDRD- ExtRS- UCnfE-
00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Kernel modules: amd64_edac_mod
00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Kernel driver in use: k8temp Kernel modules: k8temp
[7.6.] SCSI information (from /proc/scsi/scsi) Host: scsi2 Channel: 00 Id: 00 Lun: 00 Vendor: ATA Model: WDC WD20EARS-00M Rev: 51.0 Type: Direct-Access ANSI SCSI revision: 05
[7.7.] Other information that might be relevant to the problem (please look in /proc and include all information that you think to be relevant):
:~$ cryptsetup luksDump /dev/sda1 Version: 1 Cipher name: aes Cipher mode: cbc-essiv:sha256 Hash spec: sha1 Payload offset: 2056 MK bits: 256
:~$ mkfs.ext4 /dev/mapper/crypted -m 0.2
:~$ exportfs -v /mnt/crypted 192.168.0.1(rw,async,wdelay,no_root_squash,no_subtree_check)
:~$ cat /proc/mount (server) /dev/mapper/crypted /mnt/crypted ext4 rw,relatime,user_xattr,acl,barrier=1,data=ordered 0 0
:~$ cat /proc/mount (client) nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 192.168.0.10:/mnt/crypted/ /mnt/crypted nfs4 rw,relatime,vers=4,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.1,minorversion=0,local_lock=none,addr=192.168.0.10 0 0
:~$ nfsiostat (after reading and writing a 2GB file) 192.168.0.10:/mnt/crypted/ mounted on /mnt/crypted: op/s rpc bklog 1432.16 0.88 read: ops/s kB/s kB/op retrans avg RTT (ms) avg exe (ms) 32.037 4108.500 128.242 0 (0.0%) 66.974 94.895 write: ops/s kB/s kB/op retrans avg RTT (ms) avg exe (ms) 47.713 4116.645 86.280 0 (0.0%) 27.833 1814.244
192.168.0.10:/mnt/sda2/ mounted on /mnt/sda2: (sda2 -> ext4 not crypted partition) op/s rpc bklog 9417.74 5.37 read: ops/s kB/s kB/op retrans avg RTT (ms) avg exe (ms) 199.295 25502.509 127.964 0 (0.0%) 16.920 22.618 write: ops/s kB/s kB/op retrans avg RTT (ms) avg exe (ms) 329.141 25555.419 77.643 0 (0.0%) 3.888 507.813
:~$ cat /proc/fs/nfsd export_features: 0x17e3f 0xf exports: /mnt/crypted 192.168.0.1(rw,no_root_squash,async,wdelay,no_subtree_check) max_block_size: 131072 nfsv4gracetime: 90 nfsv4leasetime: 90 nfsv4recoverydir: /var/lib/nfs/v4recovery pool_stats: 0 350070842 335513464 11979013 24 pool_threads: 8 portlist: udp/tcp=2049 supported_krb5_enctypes: 18,17,16,23,3,1,2 threads: 8 versions: +2 +3 +4 +4.1
:~$ cat /sys/kernel/slab/dm_crypt_io/ aliases : [0] align : [8] alloc_calls : [] cache_dma : [0] cpu_slabs : [1 N0=1] ctor : [] destroy_by_rcu : [0] free_calls : [] hwcache_align : [0] min_partial : [7] objects : [26 N0=26] object_size : [152] objects_partial : [0] objs_per_slab : [26] order : [0] partial : [0] poison : [0] reclaim_account : [0] red_zone : [0] remote_node_defrag_ratio : [100] reserved : [0] sanity_checks : [0] shrink : [] slabs : [1 N0=1] slab_size : [152] store_user : [0] total_objects : [26 N0=26] trace : [0] validate : [] -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |