Messages in this thread | | | From | Niels Provos <> | Date | Fri, 21 May 1999 13:21:19 -0400 | Subject | Re: hinting system for poll (was Re: /dev/poll vs. aio_) |
| |
In message <19990521083756.A11659@avanticorp.com>, Jim Nance writes: >How does this compare to a system w/o your patches? It seems that my bottle neck is on the client side at the moment.
I used httperf-0.6 to generate 2,000,000 requests with a rate of 1200 req/s with thttpd-2.04 running on 2.2.7-ac4. The client, a 2-way 200 MHz Pentium Pro, was connected with 100 MHz ethernet to the server, a 4-way 450 MHz Xeon.
Readprofile shows that the hinting system does not use up as much CPU cycles as the system without hinting.
httperf output:
The system with hinting: ------------------------ Connection rate: 1197.8 conn/s (0.8 ms/conn, <=972 concurrent connections) Connection time [ms]: min 1.1 avg 403.9 max 3620.7 median 69.5 stddev 930.7 Connection time [ms]: connect 346.3 Connection length [replies/conn]: 1.000
Request rate: 1197.8 req/s (0.8 ms/req) Request size [B]: 63.0
Reply rate [replies/s]: min 1115.5 avg 1199.6 max 1228.7 stddev 12.0 (333 samples) Reply time [ms]: response 57.6 transfer 0.0 Reply size [B]: header 219.0 content 1704.0 footer 0.0 (total 1923.0) Reply status: 1xx=0 2xx=2000000 3xx=0 4xx=0 5xx=0
CPU time [s]: user 253.26 system 1409.82 (user 15.2% system 84.4% total 99.6%) Net I/O: 2323.0 KB/s (19.0*10^6 bps)
Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0 Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
The system without hinting: --------------------------- Connection rate: 1195.9 conn/s (0.8 ms/conn, <=1022 concurrent connections) Connection time [ms]: min 4.4 avg 705.8 max 9311.3 median 270.5 stddev 1093.8 Connection time [ms]: connect 543.6 Connection length [replies/conn]: 1.000
Request rate: 1195.9 req/s (0.8 ms/req) Request size [B]: 63.0
Reply rate [replies/s]: min 1116.8 avg 1197.5 max 1271.3 stddev 27.1 (333 samples) Reply time [ms]: response 162.3 transfer 0.0 Reply size [B]: header 219.0 content 1704.0 footer 0.0 (total 1923.0) Reply status: 1xx=0 2xx=1996908 3xx=0 4xx=0 5xx=0
CPU time [s]: user 250.02 system 1412.38 (user 15.0% system 84.6% total 99.6%) Net I/O: 2319.4 KB/s (19.0*10^6 bps)
Errors: total 3092 client-timo 0 socket-timo 0 connrefused 0 connreset 0 Errors: fd-unavail 3092 addrunavail 0 ftab-full 0 other 0
If you compare the two outputs, you notice that the connection lifetime is shorter and less erratic with the hinting system (median and standard deviation). And that the reply rate for the hinting system is closer to 1200r/s and has a smaller standard deviation than the system without hinting.
This might be explained with the output from readprofile:
System with hinting: -------------------- 7356 synchronize_irq 367.8000 18762 tcp_tw_deschedule 275.9118 7442 synchronize_bh 93.0250 5706 handle_IRQ_event 41.9559 1282 __generic_copy_from_user 20.0312 1055 system_call 18.8393 1927 do_bottom_half 11.4702 2951 speedo_start_xmit 8.0190 141 signal_return 5.8750 4251 tcp_v4_rcv 5.5352 87 ret_with_reschedule 5.4375 1105 csum_partial_copy_generic 5.0227 352 sock_wmalloc 4.8889 1665 kmem_cache_alloc 4.7845 282 sk_alloc 4.7000 [...] 154 sock_poll 3.5000 <---- 134 inet_poll 2.7917 <---- 2379 do_select 1.8941 382 max_select_fd 1.8365 459 tcp_poll 1.1250 <----
System without hinting: ----------------------- 7551 synchronize_irq 377.5500 16665 tcp_tw_deschedule 245.0735 4100 sock_poll 113.8889 <---- 7902 synchronize_bh 98.7750 2600 inet_poll 59.0909 <---- 6509 handle_IRQ_event 47.8603 1523 __generic_copy_from_user 23.7969 5303 tcp_poll 20.0871 <---- 1042 system_call 18.6071 2189 do_bottom_half 13.0298 556 __udelay 9.2667 1394 free_wait 8.9359 3032 speedo_start_xmit 8.2391 953 si_meminfo 7.6855 180 delay_50ms 7.5000 [...] 4075 do_select 7.1241 1323 __pollwait 6.7500 98 tcp_listen_poll 1.2250 226 max_select_fd 1.2021
Where the sorted output of readprofile did not include some poll()/select() functions called during the benchmark I appended them below.
You see, if I interpret the output of readprofile correctly, that the hinting system avoids a significant part of the calls to the protocol specific poll() functions.
One thing I noticed is that tcp_tw_deschedule might be a bottleneck for the web server performance and that further optimization might help here.
Greetings, Niels.
-- Niels Provos <provos@citi.umich.edu> finger provos@umich.edu for pgp info The Linux Scalability project: http://www.citi.umich.edu/projects/linux-scalability/
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |