Messages in this thread | | | Date | Thu, 18 Jun 2015 11:14:15 +0200 | From | Jan Kara <> | Subject | Re: [lkp] [jbd2] de92c8caf16: no primary result change, +270.9% vmstat.procs.b, -61.0% vmstat.procs.r |
| |
On Thu 18-06-15 11:20:59, Huang Ying wrote: > FYI, we noticed the below changes on > > git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master > commit de92c8caf16ca84926fa31b7a5590c0fb9c0d5ca ("jbd2: speedup jbd2_journal_get_[write|undo]_access()") > > It appears that more processes are put in uninterruptible state after the > commit. Thanks for info! So looking at the overall perf statistics we seem to be using somewhat less CPU and more quickly proceeding to wait for the IO which is the bottleneck for this workload. So I think we are fine.
Honza
> ========================================================================================= > tbox_group/testcase/rootfs/kconfig/compiler/wait_disks_timeout/runtime/disk/md/iosched/fs/nr_threads: > lkp-st02/dd-write/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/300/5m/11HDD/JBOD/cfq/ext4/10dd > > commit: > 8b00f400eedf91d074f831077003c0d4d9147377 > de92c8caf16ca84926fa31b7a5590c0fb9c0d5ca > > 8b00f400eedf91d0 de92c8caf16ca84926fa31b7a5 > ---------------- -------------------------- > %stddev %change %stddev > \ | \ > 48241 ± 8% +139.2% 115399 ± 2% softirqs.SCHED > 1565 ± 5% +12.4% 1759 ± 6% time.involuntary_context_switches > 391202 ± 5% +110.7% 824392 ± 1% proc-vmstat.pgactivate > 70008 ± 56% -73.1% 18828 ± 50% proc-vmstat.pgrotated > 21.50 ± 10% +270.9% 79.75 ± 1% vmstat.procs.b > 86.00 ± 1% -61.0% 33.50 ± 2% vmstat.procs.r > 23827 ± 0% -2.7% 23187 ± 0% vmstat.system.in > 2310 ± 4% -16.9% 1919 ± 10% slabinfo.ext4_io_end.active_objs > 2310 ± 4% -16.9% 1919 ± 10% slabinfo.ext4_io_end.num_objs > 2110 ± 18% +18.4% 2499 ± 3% slabinfo.kmalloc-96.active_objs > 2163 ± 13% +15.5% 2499 ± 3% slabinfo.kmalloc-96.num_objs > 1.683e+12 ± 1% -6.1% 1.58e+12 ± 0% perf-stat.L1-dcache-loads > 6.332e+08 ± 2% -6.8% 5.904e+08 ± 1% perf-stat.L1-dcache-prefetches > 9.849e+11 ± 1% -4.0% 9.457e+11 ± 0% perf-stat.L1-dcache-stores > 6.932e+10 ± 1% +6.7% 7.397e+10 ± 0% perf-stat.L1-icache-load-misses > 5.064e+12 ± 1% -5.7% 4.777e+12 ± 0% perf-stat.L1-icache-loads > 1.823e+09 ± 1% +8.0% 1.969e+09 ± 0% perf-stat.LLC-load-misses > 8.519e+11 ± 1% -3.5% 8.222e+11 ± 0% perf-stat.branch-instructions > 8.99e+09 ± 2% -23.1% 6.916e+09 ± 0% perf-stat.branch-load-misses > 8.513e+11 ± 1% -3.5% 8.216e+11 ± 0% perf-stat.branch-loads > 8.967e+09 ± 2% -23.5% 6.858e+09 ± 0% perf-stat.branch-misses > 8.059e+11 ± 1% -3.4% 7.786e+11 ± 0% perf-stat.bus-cycles > 3.658e+09 ± 1% +5.6% 3.863e+09 ± 0% perf-stat.cache-misses > 1.612e+11 ± 2% +5.6% 1.701e+11 ± 0% perf-stat.cache-references > 6.452e+12 ± 1% -3.5% 6.227e+12 ± 0% perf-stat.cpu-cycles > 92243 ± 6% +175.6% 254241 ± 2% perf-stat.cpu-migrations > 1.305e+10 ± 12% -15.2% 1.106e+10 ± 4% perf-stat.dTLB-load-misses > 1.683e+12 ± 1% -6.2% 1.579e+12 ± 0% perf-stat.dTLB-loads > 9.845e+11 ± 1% -4.0% 9.446e+11 ± 0% perf-stat.dTLB-stores > 31933083 ± 9% -31.3% 21923765 ± 2% perf-stat.iTLB-load-misses > 4.638e+12 ± 1% -3.6% 4.471e+12 ± 0% perf-stat.iTLB-loads > 4.639e+12 ± 1% -3.6% 4.471e+12 ± 0% perf-stat.instructions > 6.448e+12 ± 1% -3.4% 6.229e+12 ± 0% perf-stat.ref-cycles > 1.36 ± 4% +17.9% 1.60 ± 3% perf-profile.cpu-cycles.__clear_user.iov_iter_zero.read_iter_zero.__vfs_read.vfs_read > 3.48 ± 4% +11.1% 3.87 ± 2% perf-profile.cpu-cycles.__ext4_get_inode_loc.ext4_get_inode_loc.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_dirty_inode > 5.15 ± 1% +22.6% 6.31 ± 2% perf-profile.cpu-cycles.__ext4_handle_dirty_metadata.ext4_mark_iloc_dirty.ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty > 7.90 ± 3% -87.7% 0.98 ± 14% perf-profile.cpu-cycles.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty > 8.53 ± 0% +17.7% 10.04 ± 1% perf-profile.cpu-cycles.__ext4_journal_start_sb.ext4_da_write_begin.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter > 0.91 ± 3% +11.3% 1.01 ± 9% perf-profile.cpu-cycles.__ext4_journal_start_sb.ext4_dirty_inode.__mark_inode_dirty.generic_write_end.ext4_da_write_end > 2.99 ± 2% +17.4% 3.50 ± 4% perf-profile.cpu-cycles.__ext4_journal_stop.ext4_da_write_end.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter > 0.78 ± 6% +22.4% 0.96 ± 6% perf-profile.cpu-cycles.__ext4_journal_stop.ext4_dirty_inode.__mark_inode_dirty.generic_write_end.ext4_da_write_end > 1.19 ± 3% +29.1% 1.53 ± 4% perf-profile.cpu-cycles.__find_get_block.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc.ext4_reserve_inode_write > 1.88 ± 4% +20.2% 2.26 ± 4% perf-profile.cpu-cycles.__getblk_gfp.__ext4_get_inode_loc.ext4_get_inode_loc.ext4_reserve_inode_write.ext4_mark_inode_dirty > 26.69 ± 0% -16.5% 22.29 ± 1% perf-profile.cpu-cycles.__mark_inode_dirty.generic_write_end.ext4_da_write_end.generic_perform_write.__generic_file_write_iter > 3.39 ± 3% +18.7% 4.02 ± 2% perf-profile.cpu-cycles.__vfs_read.vfs_read.sys_read.system_call_fastpath > 0.99 ± 3% +36.5% 1.35 ± 5% perf-profile.cpu-cycles._raw_read_lock.jbd2__journal_start.__ext4_journal_start_sb.ext4_da_write_begin.generic_perform_write > 4.13 ± 6% -99.8% 0.01 ±-10000% perf-profile.cpu-cycles.do_get_write_access.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty > 34.70 ± 0% -10.1% 31.19 ± 1% perf-profile.cpu-cycles.ext4_da_write_end.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter.__vfs_write > 26.13 ± 1% -16.6% 21.80 ± 1% perf-profile.cpu-cycles.ext4_dirty_inode.__mark_inode_dirty.generic_write_end.ext4_da_write_end.generic_perform_write > 3.76 ± 3% +9.2% 4.11 ± 2% perf-profile.cpu-cycles.ext4_get_inode_loc.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty > 10.35 ± 0% +13.4% 11.74 ± 2% perf-profile.cpu-cycles.ext4_mark_iloc_dirty.ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty.generic_write_end > 23.99 ± 1% -19.4% 19.35 ± 2% perf-profile.cpu-cycles.ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty.generic_write_end.ext4_da_write_end > 11.94 ± 4% -53.5% 5.55 ± 3% perf-profile.cpu-cycles.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty.generic_write_end > 30.83 ± 0% -13.8% 26.58 ± 1% perf-profile.cpu-cycles.generic_write_end.ext4_da_write_end.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter > 7.42 ± 0% +16.8% 8.67 ± 1% perf-profile.cpu-cycles.jbd2__journal_start.__ext4_journal_start_sb.ext4_da_write_begin.generic_perform_write.__generic_file_write_iter > 1.62 ± 7% -100.0% 0.00 ± -1% perf-profile.cpu-cycles.jbd2_journal_add_journal_head.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty > 3.63 ± 1% +19.7% 4.34 ± 3% perf-profile.cpu-cycles.jbd2_journal_dirty_metadata.__ext4_handle_dirty_metadata.ext4_mark_iloc_dirty.ext4_mark_inode_dirty.ext4_dirty_inode > 7.43 ± 3% -92.3% 0.57 ± 22% perf-profile.cpu-cycles.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_dirty_inode > 1.01 ± 9% +45.9% 1.47 ± 5% perf-profile.cpu-cycles.jbd2_journal_grab_journal_head.jbd2_journal_dirty_metadata.__ext4_handle_dirty_metadata.ext4_mark_iloc_dirty.ext4_mark_inode_dirty > 1.00 ± 6% -100.0% 0.00 ± -1% perf-profile.cpu-cycles.jbd2_journal_put_journal_head.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty > 2.68 ± 1% +17.4% 3.15 ± 5% perf-profile.cpu-cycles.jbd2_journal_stop.__ext4_journal_stop.ext4_da_write_end.generic_perform_write.__generic_file_write_iter > 3.89 ± 3% +18.0% 4.58 ± 2% perf-profile.cpu-cycles.start_this_handle.jbd2__journal_start.__ext4_journal_start_sb.ext4_da_write_begin.generic_perform_write > 6.77 ± 1% +14.2% 7.74 ± 2% perf-profile.cpu-cycles.sys_read.system_call_fastpath > 0.90 ± 10% -100.0% 0.00 ± -1% perf-profile.cpu-cycles.unlock_buffer.do_get_write_access.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write > 5.98 ± 1% +16.8% 6.98 ± 1% perf-profile.cpu-cycles.vfs_read.sys_read.system_call_fastpath > 1323 ± 24% -67.3% 433.50 ± 14% sched_debug.cfs_rq[0]:/.utilization_load_avg > 1153750 ± 5% +12.2% 1294517 ± 1% sched_debug.cfs_rq[1]:/.min_vruntime > -260439 ±-69% -103.8% 9885 ±199% sched_debug.cfs_rq[1]:/.spread0 > 1237 ± 18% -78.8% 262.50 ± 28% sched_debug.cfs_rq[1]:/.utilization_load_avg > 87.25 ± 17% +165.6% 231.75 ± 56% sched_debug.cfs_rq[2]:/.load > 942.25 ± 16% -78.5% 203.00 ± 36% sched_debug.cfs_rq[2]:/.utilization_load_avg > 1749 ± 26% -48.3% 903.50 ± 72% sched_debug.cfs_rq[3]:/.blocked_load_avg > 206.25 ± 39% -44.5% 114.50 ± 14% sched_debug.cfs_rq[3]:/.load > 162.00 ± 3% -25.8% 120.25 ± 21% sched_debug.cfs_rq[3]:/.runnable_load_avg > -241296 ±-76% -102.5% 5936 ±118% sched_debug.cfs_rq[3]:/.spread0 > 1930 ± 22% -46.9% 1024 ± 63% sched_debug.cfs_rq[3]:/.tg_load_contrib > 1203 ± 9% -72.3% 333.50 ± 46% sched_debug.cfs_rq[3]:/.utilization_load_avg > 921.50 ± 95% +131.9% 2137 ± 37% sched_debug.cfs_rq[4]:/.blocked_load_avg > 1080842 ± 3% +17.8% 1273061 ± 0% sched_debug.cfs_rq[4]:/.min_vruntime > -333430 ±-48% -96.5% -11621 ±-147% sched_debug.cfs_rq[4]:/.spread0 > 1077 ± 86% +110.7% 2269 ± 35% sched_debug.cfs_rq[4]:/.tg_load_contrib > 1288 ± 19% -76.7% 300.50 ± 23% sched_debug.cfs_rq[4]:/.utilization_load_avg > 1205 ± 14% -68.7% 376.75 ± 52% sched_debug.cfs_rq[5]:/.utilization_load_avg > 1095954 ± 4% +17.7% 1290354 ± 0% sched_debug.cfs_rq[6]:/.min_vruntime > -318381 ±-49% -101.8% 5587 ±256% sched_debug.cfs_rq[6]:/.spread0 > 1265 ± 14% -71.3% 363.50 ± 46% sched_debug.cfs_rq[6]:/.utilization_load_avg > 1251 ± 20% -76.9% 289.75 ± 32% sched_debug.cfs_rq[7]:/.utilization_load_avg > 10.00 ± 23% -75.0% 2.50 ± 20% sched_debug.cpu#0.nr_running > 10.50 ± 14% -76.2% 2.50 ± 44% sched_debug.cpu#1.nr_running > 1793 ± 12% -26.9% 1311 ± 12% sched_debug.cpu#2.curr->pid > 8.00 ± 26% -71.9% 2.25 ± 36% sched_debug.cpu#2.nr_running > -246.00 ±-46% -93.2% -16.75 ±-607% sched_debug.cpu#2.nr_uninterruptible > 10.50 ± 14% -73.8% 2.75 ± 30% sched_debug.cpu#3.nr_running > 128.50 ± 11% -20.0% 102.75 ± 8% sched_debug.cpu#4.cpu_load[0] > 127.75 ± 9% -16.0% 107.25 ± 11% sched_debug.cpu#4.cpu_load[1] > 10.50 ± 20% -73.8% 2.75 ± 30% sched_debug.cpu#4.nr_running > 9.75 ± 15% -71.8% 2.75 ± 15% sched_debug.cpu#5.nr_running > 122.00 ± 9% +19.7% 146.00 ± 8% sched_debug.cpu#6.cpu_load[1] > 120.00 ± 7% +25.8% 151.00 ± 15% sched_debug.cpu#6.cpu_load[2] > 10.75 ± 21% -72.1% 3.00 ± 52% sched_debug.cpu#6.nr_running > 1971 ± 9% -34.0% 1300 ± 17% sched_debug.cpu#7.curr->pid > 10.00 ± 23% -82.5% 1.75 ± 24% sched_debug.cpu#7.nr_running > 40663 ± 2% -18.6% 33116 ± 0% latency_stats.avg.balance_dirty_pages.balance_dirty_pages_ratelimited.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter.__vfs_write.vfs_write.SyS_write.system_call_fastpath > 508303 ± 3% +3.3% 525071 ± 2% latency_stats.avg.blk_execute_rq.sg_io.scsi_cmd_ioctl.scsi_cmd_blk_ioctl.sd_ioctl.[sd_mod].blkdev_ioctl.block_ioctl.do_vfs_ioctl.SyS_ioctl.system_call_fastpath > 65195 ± 4% -13.7% 56254 ± 3% latency_stats.avg.do_get_write_access.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty.generic_update_time.file_update_time.__generic_file_write_iter.ext4_file_write_iter.__vfs_write > 3440 ± 71% -100.0% 0.00 ± -1% latency_stats.avg.do_get_write_access.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_ext_truncate.ext4_truncate.ext4_setattr.notify_change.do_truncate.do_sys_ftruncate.SyS_ftruncate > 453.25 ± 4% +67.1% 757.50 ± 1% latency_stats.avg.do_wait.SyS_wait4.system_call_fastpath > 968.50 ± 16% +28.9% 1248 ± 10% latency_stats.avg.ep_poll.SyS_epoll_wait.system_call_fastpath > 438.50 ± 8% -32.7% 295.00 ± 1% latency_stats.avg.ring_buffer_wait.wait_on_pipe.tracing_wait_pipe.tracing_read_pipe.__vfs_read.vfs_read.SyS_read.system_call_fastpath > 686.00 ± 22% -21.1% 541.50 ± 16% latency_stats.avg.wait_for_response.[cifs].SendReceive.[cifs].CIFSPOSIXCreate.[cifs].cifs_posix_open.[cifs].cifs_open.[cifs].do_dentry_open.vfs_open.do_last.path_openat.do_filp_open.do_sys_open.SyS_open > 874.50 ± 55% -47.7% 457.75 ± 11% latency_stats.avg.wait_for_response.[cifs].SendReceive.[cifs].CIFSPOSIXCreate.[cifs].cifs_posix_open.[cifs].cifs_open.[cifs].do_dentry_open.vfs_open.do_last.path_openat.do_filp_open.do_sys_open.SyS_openat > 1169 ±140% -86.2% 161.75 ± 5% latency_stats.avg.wait_for_response.[cifs].SendReceive.[cifs].CIFSSMBUnixQPathInfo.[cifs].cifs_get_inode_info_unix.[cifs].cifs_lookup.[cifs].lookup_real.__lookup_hash.do_unlinkat.SyS_unlink.system_call_fastpath > 199.00 ± 7% -19.5% 160.25 ± 15% latency_stats.avg.wait_for_response.[cifs].SendReceive2.[cifs].CIFSSMBWrite2.[cifs].cifs_sync_write.[cifs].cifs_write.[cifs].cifs_write_end.[cifs].generic_perform_write.__generic_file_write_iter.generic_file_write_iter.cifs_strict_writev.[cifs].__vfs_write.vfs_write > 107505 ± 14% +51.7% 163094 ± 11% latency_stats.avg.wait_transaction_locked.start_this_handle.jbd2__journal_start.__ext4_journal_start_sb.ext4_dirty_inode.__mark_inode_dirty.generic_update_time.file_update_time.__generic_file_write_iter.ext4_file_write_iter.__vfs_write.vfs_write > 103169 ± 12% +514.6% 634103 ± 1% latency_stats.hits.balance_dirty_pages.balance_dirty_pages_ratelimited.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter.__vfs_write.vfs_write.SyS_write.system_call_fastpath > 1.00 ± 81% +225.0% 3.25 ± 50% latency_stats.hits.call_rwsem_down_write_failed.unlink_file_vma.free_pgtables.exit_mmap.mmput.flush_old_exec.load_elf_binary.search_binary_handler.do_execveat_common.SyS_execve.return_from_execve > 26.00 ± 14% +30.8% 34.00 ± 13% latency_stats.hits.do_get_write_access.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_da_write_end.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter.__vfs_write.vfs_write.SyS_write > 2835 ± 5% +11.4% 3158 ± 3% latency_stats.hits.do_get_write_access.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty.generic_update_time.file_update_time.__generic_file_write_iter.ext4_file_write_iter.__vfs_write > 663.50 ± 10% -40.2% 396.75 ± 8% latency_stats.hits.do_get_write_access.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty.generic_write_end.ext4_da_write_end.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter > 1.75 ± 47% -100.0% 0.00 ± -1% latency_stats.hits.do_get_write_access.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_ext_truncate.ext4_truncate.ext4_setattr.notify_change.do_truncate.do_sys_ftruncate.SyS_ftruncate > 1661 ± 0% +17.3% 1948 ± 1% latency_stats.hits.do_wait.SyS_wait4.system_call_fastpath > 142.50 ± 6% +15.3% 164.25 ± 11% latency_stats.hits.ep_poll.SyS_epoll_wait.system_call_fastpath > 394313 ± 0% -10.2% 354275 ± 4% latency_stats.hits.hrtimer_nanosleep.SyS_nanosleep.system_call_fastpath > 465945 ± 23% -37.9% 289496 ± 4% latency_stats.hits.pipe_wait.pipe_read.__vfs_read.vfs_read.SyS_read.system_call_fastpath > 337392 ± 8% -49.8% 169230 ± 8% latency_stats.hits.ring_buffer_wait.wait_on_pipe.tracing_wait_pipe.tracing_read_pipe.__vfs_read.vfs_read.SyS_read.system_call_fastpath > 91.75 ± 47% -87.6% 11.33 ±141% latency_stats.hits.wait_iff_congested.shrink_inactive_list.shrink_lruvec.shrink_zone.do_try_to_free_pages.try_to_free_pages.__alloc_pages_nodemask.alloc_pages_current.__page_cache_alloc.pagecache_get_page.grab_cache_page_write_begin.ext4_da_write_begin > 1123 ± 3% -41.6% 656.00 ± 6% latency_stats.hits.wait_transaction_locked.start_this_handle.jbd2__journal_start.__ext4_journal_start_sb.ext4_da_write_begin.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter.__vfs_write.vfs_write.SyS_write.system_call_fastpath > 4105 ± 0% -38.2% 2535 ± 3% latency_stats.hits.wait_transaction_locked.start_this_handle.jbd2__journal_start.__ext4_journal_start_sb.ext4_dirty_inode.__mark_inode_dirty.generic_update_time.file_update_time.__generic_file_write_iter.ext4_file_write_iter.__vfs_write.vfs_write > 1030093 ± 9% +11.9% 1152679 ± 6% latency_stats.max.blk_execute_rq.sg_io.scsi_cmd_ioctl.scsi_cmd_blk_ioctl.sd_ioctl.[sd_mod].blkdev_ioctl.block_ioctl.do_vfs_ioctl.SyS_ioctl.system_call_fastpath > 309120 ± 43% -41.6% 180418 ± 23% latency_stats.max.do_get_write_access.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_da_write_end.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter.__vfs_write.vfs_write.SyS_write > 4459 ± 75% -100.0% 0.00 ± -1% latency_stats.max.do_get_write_access.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_ext_truncate.ext4_truncate.ext4_setattr.notify_change.do_truncate.do_sys_ftruncate.SyS_ftruncate > 4823 ± 1% +2.5% 4943 ± 0% latency_stats.max.ep_poll.SyS_epoll_wait.system_call_fastpath > 1271 ± 69% -55.1% 570.75 ± 8% latency_stats.max.wait_for_response.[cifs].SendReceive.[cifs].CIFSPOSIXCreate.[cifs].cifs_posix_open.[cifs].cifs_do_create.[cifs].cifs_atomic_open.[cifs].do_last.path_openat.do_filp_open.do_sys_open.SyS_open.system_call_fastpath > 11400 ±126% -91.8% 939.75 ± 37% latency_stats.max.wait_for_response.[cifs].SendReceive.[cifs].CIFSPOSIXCreate.[cifs].cifs_posix_open.[cifs].cifs_open.[cifs].do_dentry_open.vfs_open.do_last.path_openat.do_filp_open.do_sys_open.SyS_openat > 1169 ±140% -86.2% 161.75 ± 5% latency_stats.max.wait_for_response.[cifs].SendReceive.[cifs].CIFSSMBUnixQPathInfo.[cifs].cifs_get_inode_info_unix.[cifs].cifs_lookup.[cifs].lookup_real.__lookup_hash.do_unlinkat.SyS_unlink.system_call_fastpath > 4.2e+09 ± 13% +399.9% 2.1e+10 ± 1% latency_stats.sum.balance_dirty_pages.balance_dirty_pages_ratelimited.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter.__vfs_write.vfs_write.SyS_write.system_call_fastpath > 24398593 ± 3% +3.3% 25203443 ± 2% latency_stats.sum.blk_execute_rq.sg_io.scsi_cmd_ioctl.scsi_cmd_blk_ioctl.sd_ioctl.[sd_mod].blkdev_ioctl.block_ioctl.do_vfs_ioctl.SyS_ioctl.system_call_fastpath > 10100882 ± 15% -24.3% 7648970 ± 12% latency_stats.sum.call_rwsem_down_read_failed.ext4_da_get_block_prep.__block_write_begin.ext4_da_write_begin.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter.__vfs_write.vfs_write.SyS_write.system_call_fastpath > 6.00 ± 81% +216.7% 19.00 ± 56% latency_stats.sum.call_rwsem_down_write_failed.unlink_file_vma.free_pgtables.exit_mmap.mmput.flush_old_exec.load_elf_binary.search_binary_handler.do_execveat_common.SyS_execve.return_from_execve > 38146064 ± 11% -40.7% 22612285 ± 6% latency_stats.sum.do_get_write_access.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_dirty_inode.__mark_inode_dirty.generic_write_end.ext4_da_write_end.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter > 7994 ±100% -100.0% 0.00 ± -1% latency_stats.sum.do_get_write_access.jbd2_journal_get_write_access.__ext4_journal_get_write_access.ext4_reserve_inode_write.ext4_mark_inode_dirty.ext4_ext_truncate.ext4_truncate.ext4_setattr.notify_change.do_truncate.do_sys_ftruncate.SyS_ftruncate > 753440 ± 3% +96.0% 1476939 ± 2% latency_stats.sum.do_wait.SyS_wait4.system_call_fastpath > 139159 ± 21% +49.1% 207524 ± 22% latency_stats.sum.ep_poll.SyS_epoll_wait.system_call_fastpath > 22735513 ± 0% -10.0% 20470551 ± 4% latency_stats.sum.hrtimer_nanosleep.SyS_nanosleep.system_call_fastpath > 2.014e+08 ± 6% -51.4% 97800459 ± 4% latency_stats.sum.pipe_wait.pipe_read.__vfs_read.vfs_read.SyS_read.system_call_fastpath > 1.488e+08 ± 14% -66.4% 49982280 ± 7% latency_stats.sum.ring_buffer_wait.wait_on_pipe.tracing_wait_pipe.tracing_read_pipe.__vfs_read.vfs_read.SyS_read.system_call_fastpath > 79639 ± 22% -21.2% 62767 ± 17% latency_stats.sum.wait_for_response.[cifs].SendReceive.[cifs].CIFSPOSIXCreate.[cifs].cifs_posix_open.[cifs].cifs_open.[cifs].do_dentry_open.vfs_open.do_last.path_openat.do_filp_open.do_sys_open.SyS_open > 25189 ± 56% -49.1% 12823 ± 11% latency_stats.sum.wait_for_response.[cifs].SendReceive.[cifs].CIFSPOSIXCreate.[cifs].cifs_posix_open.[cifs].cifs_open.[cifs].do_dentry_open.vfs_open.do_last.path_openat.do_filp_open.do_sys_open.SyS_openat > 1169 ±140% -86.2% 161.75 ± 5% latency_stats.sum.wait_for_response.[cifs].SendReceive.[cifs].CIFSSMBUnixQPathInfo.[cifs].cifs_get_inode_info_unix.[cifs].cifs_lookup.[cifs].lookup_real.__lookup_hash.do_unlinkat.SyS_unlink.system_call_fastpath > 24794 ± 13% -25.1% 18564 ± 14% latency_stats.sum.wait_for_response.[cifs].SendReceive2.[cifs].CIFSSMBWrite2.[cifs].cifs_sync_write.[cifs].cifs_write.[cifs].cifs_write_end.[cifs].generic_perform_write.__generic_file_write_iter.generic_file_write_iter.cifs_strict_writev.[cifs].__vfs_write.vfs_write > 108340 ± 27% -81.3% 20256 ±141% latency_stats.sum.wait_iff_congested.shrink_inactive_list.shrink_lruvec.shrink_zone.do_try_to_free_pages.try_to_free_pages.__alloc_pages_nodemask.alloc_pages_current.__page_cache_alloc.pagecache_get_page.grab_cache_page_write_begin.ext4_da_write_begin > 1.015e+08 ± 11% -36.3% 64626646 ± 16% latency_stats.sum.wait_transaction_locked.start_this_handle.jbd2__journal_start.__ext4_journal_start_sb.ext4_da_write_begin.generic_perform_write.__generic_file_write_iter.ext4_file_write_iter.__vfs_write.vfs_write.SyS_write.system_call_fastpath > 26524 ± 5% +16.1% 30791 ± 4% ftrace.balance_dirty_pages.sdb.bdi_dirty > 179.00 ± 15% -46.4% 96.00 ± 4% ftrace.balance_dirty_pages.sdb.dirtied > 179.25 ± 15% -45.0% 98.50 ± 4% ftrace.balance_dirty_pages.sdb.dirtied_pause > 58927 ± 13% -80.8% 11327 ± 3% ftrace.balance_dirty_pages.sdb.dirty_ratelimit > -3430677 ±-173% -100.0% -11.00 ±-75% ftrace.balance_dirty_pages.sdb.pause > 22.50 ± 25% +76.7% 39.75 ± 3% ftrace.balance_dirty_pages.sdb.period > 80869 ± 18% -86.9% 10600 ± 2% ftrace.balance_dirty_pages.sdb.task_ratelimit > 3430696 ±173% -100.0% 49.25 ± 18% ftrace.balance_dirty_pages.sdb.think > 190.25 ± 13% -51.8% 91.75 ± 5% ftrace.balance_dirty_pages.sdc.dirtied > 191.75 ± 13% -51.1% 93.75 ± 5% ftrace.balance_dirty_pages.sdc.dirtied_pause > 58193 ± 19% -81.2% 10953 ± 5% ftrace.balance_dirty_pages.sdc.dirty_ratelimit > -424.50 ± -2% -98.4% -7.00 ±-286% ftrace.balance_dirty_pages.sdc.pause > 20.00 ± 17% +448.8% 109.75 ±114% ftrace.balance_dirty_pages.sdc.period > 76965 ± 21% -84.8% 11683 ± 7% ftrace.balance_dirty_pages.sdc.task_ratelimit > 443.75 ± 2% -90.4% 42.75 ± 47% ftrace.balance_dirty_pages.sdc.think > 23458 ± 9% +22.1% 28637 ± 7% ftrace.balance_dirty_pages.sdd.bdi_dirty > 185.75 ± 14% -49.7% 93.50 ± 5% ftrace.balance_dirty_pages.sdd.dirtied > 186.00 ± 14% -49.2% 94.50 ± 4% ftrace.balance_dirty_pages.sdd.dirtied_pause > 61258 ± 14% -81.7% 11213 ± 3% ftrace.balance_dirty_pages.sdd.dirty_ratelimit > -416.00 ± -4% -100.1% 0.50 ±830% ftrace.balance_dirty_pages.sdd.pause > 19.25 ± 22% +97.4% 38.00 ± 4% ftrace.balance_dirty_pages.sdd.period > 86687 ± 20% -87.2% 11105 ± 2% ftrace.balance_dirty_pages.sdd.task_ratelimit > 433.50 ± 4% -91.5% 36.75 ± 7% ftrace.balance_dirty_pages.sdd.think > 186.75 ± 13% -53.1% 87.50 ± 0% ftrace.balance_dirty_pages.sde.dirtied > 188.25 ± 13% -52.6% 89.25 ± 0% ftrace.balance_dirty_pages.sde.dirtied_pause > 60931 ± 16% -82.8% 10461 ± 1% ftrace.balance_dirty_pages.sde.dirty_ratelimit > -3387398 ±-173% -100.0% -1.25 ±-1079% ftrace.balance_dirty_pages.sde.pause > 19.25 ± 14% +94.8% 37.50 ± 1% ftrace.balance_dirty_pages.sde.period > 80501 ± 23% -86.6% 10750 ± 3% ftrace.balance_dirty_pages.sde.task_ratelimit > 3387416 ±173% -100.0% 37.00 ± 34% ftrace.balance_dirty_pages.sde.think > 191.50 ± 8% -52.3% 91.25 ± 3% ftrace.balance_dirty_pages.sdf.dirtied > 192.50 ± 8% -51.8% 92.75 ± 4% ftrace.balance_dirty_pages.sdf.dirtied_pause > 59318 ± 10% -81.4% 11026 ± 1% ftrace.balance_dirty_pages.sdf.dirty_ratelimit > -425.75 ± -1% -96.6% -14.50 ±-84% ftrace.balance_dirty_pages.sdf.pause > 21.50 ± 14% +77.9% 38.25 ± 3% ftrace.balance_dirty_pages.sdf.period > 78892 ± 14% -85.6% 11371 ± 2% ftrace.balance_dirty_pages.sdf.task_ratelimit > 446.00 ± 1% -88.5% 51.50 ± 25% ftrace.balance_dirty_pages.sdf.think > 29879 ± 6% -10.9% 26610 ± 2% ftrace.balance_dirty_pages.sdg.bdi_dirty > 181.75 ± 14% -51.3% 88.50 ± 0% ftrace.balance_dirty_pages.sdg.dirtied > 182.50 ± 14% -50.3% 90.75 ± 1% ftrace.balance_dirty_pages.sdg.dirtied_pause > 49491 ± 23% -78.7% 10564 ± 2% ftrace.balance_dirty_pages.sdg.dirty_ratelimit > -372.25 ± -5% -99.3% -2.75 ±-376% ftrace.balance_dirty_pages.sdg.pause > 27.75 ± 16% +48.6% 41.25 ± 17% ftrace.balance_dirty_pages.sdg.period > 63160 ± 27% -83.3% 10550 ± 5% ftrace.balance_dirty_pages.sdg.task_ratelimit > 398.25 ± 3% -90.4% 38.25 ± 24% ftrace.balance_dirty_pages.sdg.think > 184.00 ± 14% -50.7% 90.75 ± 5% ftrace.balance_dirty_pages.sdh.dirtied > 185.50 ± 14% -50.0% 92.75 ± 4% ftrace.balance_dirty_pages.sdh.dirtied_pause > 55615 ± 12% -80.6% 10769 ± 5% ftrace.balance_dirty_pages.sdh.dirty_ratelimit > -388.00 ±-10% -97.9% -8.00 ±-168% ftrace.balance_dirty_pages.sdh.pause > 23.50 ± 16% +62.8% 38.25 ± 3% ftrace.balance_dirty_pages.sdh.period > 70106 ± 20% -84.2% 11100 ± 6% ftrace.balance_dirty_pages.sdh.task_ratelimit > 410.25 ± 9% -89.0% 45.00 ± 32% ftrace.balance_dirty_pages.sdh.think > 176.25 ± 14% -47.1% 93.25 ± 2% ftrace.balance_dirty_pages.sdi.dirtied > 177.25 ± 14% -46.3% 95.25 ± 2% ftrace.balance_dirty_pages.sdi.dirtied_pause > 45550 ± 19% -75.7% 11062 ± 3% ftrace.balance_dirty_pages.sdi.dirty_ratelimit > -363.75 ± -8% -95.3% -17.25 ±-85% ftrace.balance_dirty_pages.sdi.pause > 2.00 ± 50% -87.5% 0.25 ±173% ftrace.balance_dirty_pages.sdi.paused > 29.25 ± 4% +35.0% 39.50 ± 1% ftrace.balance_dirty_pages.sdi.period > 54404 ± 24% -80.4% 10681 ± 6% ftrace.balance_dirty_pages.sdi.task_ratelimit > 391.75 ± 7% -85.8% 55.75 ± 26% ftrace.balance_dirty_pages.sdi.think > 180.50 ± 11% -48.2% 93.50 ± 4% ftrace.balance_dirty_pages.sdk.dirtied > 181.75 ± 12% -47.5% 95.50 ± 3% ftrace.balance_dirty_pages.sdk.dirtied_pause > 53838 ± 14% -79.4% 11111 ± 3% ftrace.balance_dirty_pages.sdk.dirty_ratelimit > -383.50 ± -8% -97.1% -11.25 ±-84% ftrace.balance_dirty_pages.sdk.pause > 23.75 ± 22% +65.3% 39.25 ± 3% ftrace.balance_dirty_pages.sdk.period > 73059 ± 22% -84.8% 11122 ± 6% ftrace.balance_dirty_pages.sdk.task_ratelimit > 406.50 ± 6% -88.0% 48.75 ± 20% ftrace.balance_dirty_pages.sdk.think > 179.75 ± 14% -49.5% 90.75 ± 3% ftrace.balance_dirty_pages.sdl.dirtied > 180.50 ± 14% -48.2% 93.50 ± 3% ftrace.balance_dirty_pages.sdl.dirtied_pause > 59262 ± 9% -81.7% 10868 ± 4% ftrace.balance_dirty_pages.sdl.dirty_ratelimit > -401.25 ± -2% -98.0% -8.00 ±-115% ftrace.balance_dirty_pages.sdl.pause > 21.00 ± 10% +82.1% 38.25 ± 2% ftrace.balance_dirty_pages.sdl.period > 77465 ± 15% -86.1% 10789 ± 4% ftrace.balance_dirty_pages.sdl.task_ratelimit > 421.75 ± 1% -89.4% 44.75 ± 21% ftrace.balance_dirty_pages.sdl.think > 185.75 ± 13% -49.0% 94.75 ± 4% ftrace.balance_dirty_pages.sdm.dirtied > 186.75 ± 13% -48.7% 95.75 ± 4% ftrace.balance_dirty_pages.sdm.dirtied_pause > 59214 ± 18% -81.0% 11276 ± 3% ftrace.balance_dirty_pages.sdm.dirty_ratelimit > -402.75 ± -3% -98.8% -5.00 ±-183% ftrace.balance_dirty_pages.sdm.pause > 1.50 ± 57% -100.0% 0.00 ± 0% ftrace.balance_dirty_pages.sdm.paused > 21.25 ± 20% +84.7% 39.25 ± 2% ftrace.balance_dirty_pages.sdm.period > 80010 ± 25% -85.6% 11485 ± 7% ftrace.balance_dirty_pages.sdm.task_ratelimit > 422.00 ± 3% -89.9% 42.50 ± 20% ftrace.balance_dirty_pages.sdm.think > 60203 ± 14% -81.5% 11163 ± 4% ftrace.bdi_dirty_ratelimit.sdb.balanced_dirty_ratelimit > 58856 ± 13% -80.9% 11269 ± 3% ftrace.bdi_dirty_ratelimit.sdb.dirty_ratelimit > 80299 ± 18% -87.2% 10313 ± 3% ftrace.bdi_dirty_ratelimit.sdb.task_ratelimit > 58684 ± 18% -81.5% 10842 ± 3% ftrace.bdi_dirty_ratelimit.sdc.balanced_dirty_ratelimit > 58173 ± 19% -81.3% 10902 ± 5% ftrace.bdi_dirty_ratelimit.sdc.dirty_ratelimit > 77036 ± 21% -85.1% 11441 ± 9% ftrace.bdi_dirty_ratelimit.sdc.task_ratelimit > 63739 ± 14% -82.8% 10955 ± 1% ftrace.bdi_dirty_ratelimit.sdd.balanced_dirty_ratelimit > 61299 ± 14% -81.8% 11184 ± 3% ftrace.bdi_dirty_ratelimit.sdd.dirty_ratelimit > 86718 ± 20% -87.4% 10969 ± 1% ftrace.bdi_dirty_ratelimit.sdd.task_ratelimit > 62431 ± 16% -82.8% 10760 ± 3% ftrace.bdi_dirty_ratelimit.sde.balanced_dirty_ratelimit > 60940 ± 16% -82.9% 10415 ± 1% ftrace.bdi_dirty_ratelimit.sde.dirty_ratelimit > 80144 ± 22% -86.9% 10513 ± 4% ftrace.bdi_dirty_ratelimit.sde.task_ratelimit > 61030 ± 11% -81.6% 11236 ± 1% ftrace.bdi_dirty_ratelimit.sdf.balanced_dirty_ratelimit > 59341 ± 10% -81.5% 10997 ± 1% ftrace.bdi_dirty_ratelimit.sdf.dirty_ratelimit > 78573 ± 14% -85.5% 11382 ± 2% ftrace.bdi_dirty_ratelimit.sdf.task_ratelimit > 49805 ± 24% -78.4% 10782 ± 5% ftrace.bdi_dirty_ratelimit.sdg.balanced_dirty_ratelimit > 49586 ± 23% -78.8% 10508 ± 2% ftrace.bdi_dirty_ratelimit.sdg.dirty_ratelimit > 62941 ± 28% -83.5% 10409 ± 6% ftrace.bdi_dirty_ratelimit.sdg.task_ratelimit > 56232 ± 13% -80.5% 10962 ± 7% ftrace.bdi_dirty_ratelimit.sdh.balanced_dirty_ratelimit > 55658 ± 12% -80.7% 10737 ± 6% ftrace.bdi_dirty_ratelimit.sdh.dirty_ratelimit > 69833 ± 20% -84.4% 10906 ± 7% ftrace.bdi_dirty_ratelimit.sdh.task_ratelimit > 44050 ± 17% -75.7% 10685 ± 4% ftrace.bdi_dirty_ratelimit.sdi.balanced_dirty_ratelimit > 45546 ± 19% -75.8% 11013 ± 3% ftrace.bdi_dirty_ratelimit.sdi.dirty_ratelimit > 54325 ± 24% -80.6% 10513 ± 7% ftrace.bdi_dirty_ratelimit.sdi.task_ratelimit > 54470 ± 16% -80.2% 10809 ± 2% ftrace.bdi_dirty_ratelimit.sdk.balanced_dirty_ratelimit > 53799 ± 14% -79.4% 11069 ± 4% ftrace.bdi_dirty_ratelimit.sdk.dirty_ratelimit > 72874 ± 22% -85.2% 10787 ± 6% ftrace.bdi_dirty_ratelimit.sdk.task_ratelimit > 60161 ± 10% -82.0% 10837 ± 2% ftrace.bdi_dirty_ratelimit.sdl.balanced_dirty_ratelimit > 59266 ± 9% -81.7% 10819 ± 5% ftrace.bdi_dirty_ratelimit.sdl.dirty_ratelimit > 77010 ± 16% -86.3% 10563 ± 2% ftrace.bdi_dirty_ratelimit.sdl.task_ratelimit > 60632 ± 18% -81.0% 11544 ± 3% ftrace.bdi_dirty_ratelimit.sdm.balanced_dirty_ratelimit > 59157 ± 18% -81.0% 11246 ± 3% ftrace.bdi_dirty_ratelimit.sdm.dirty_ratelimit > 79666 ± 25% -86.0% 11179 ± 6% ftrace.bdi_dirty_ratelimit.sdm.task_ratelimit > 263022 ± 7% -10.6% 235031 ± 4% ftrace.writeback_single_inode.sde.index > 2635 ± 5% -12.1% 2317 ± 2% ftrace.writeback_single_inode.sde.wrote > 2810 ± 8% -12.2% 2466 ± 4% ftrace.writeback_single_inode.sdf.wrote > 1.00 ± 0% -100.0% 0.00 ± 0% ftrace.writeback_single_inode.sdg.age > 3173 ± 10% -25.6% 2360 ± 4% ftrace.writeback_single_inode.sdg.wrote > 1.00 ± 0% -100.0% 0.00 ± 0% ftrace.writeback_single_inode.sdh.age > 3521 ± 14% -26.7% 2581 ± 7% ftrace.writeback_single_inode.sdi.wrote > > > lkp-st02: Core2 > Memory: 8G > > > softirqs.SCHED > > 200000 ++-----------------------------------------------------------------+ > 180000 ++ O | > | O O | > 160000 ++ O O | > 140000 O+ O O O O O O O O | > | O O O O O | > 120000 ++ O O O O O O O > 100000 ++ | > 80000 ++ | > | | > 60000 ++ .*.*.. .*. .*.. .*. | > 40000 ++ *.*..*.*. *. *..*.*. *.*..*.*. * | > | + | > 20000 ++ | > 0 *+-O---------------------------------------------------------------+ > > > perf-stat.cpu-migrations > > 400000 ++-------------------O---------------------------------------------+ > | O | > 350000 ++ O O O | > O O O O O O | > 300000 ++ O O O O O O O O | > 250000 ++ O O O O O O O > | | > 200000 ++ | > | | > 150000 ++ | > 100000 ++ .*.*.. .*. .*.. | > | *.*..*.*. *. *..*.*. *.*..*.*..*.* | > 50000 +++ | > |+ | > 0 *+-O---------------------------------------------------------------+ > > > vmstat.procs.r > > 100 ++--------------------------------------------------------------------+ > 90 ++ *. .*..*. .*.. | > | : *..*..*. .*.. .*. *. *.*..*..*.* | > 80 ++ : *..*. * | > 70 ++: | > | : | > 60 ++: | > 50 ++: | > 40 ++ | > |: O O O O O O > 30 O+ O O O O O O O O O O O O O O O O O | > 20 ++ O O | > | | > 10 ++ | > 0 *+-O------------------------------------------------------------------+ > > > vmstat.procs.b > > 100 ++--------------------------------------------------------------------+ > 90 ++ O O O O | > O O O O O O O O O O O O | > 80 ++ O O O O O O O O O O > 70 ++ | > | | > 60 ++ | > 50 ++ | > 40 ++ | > | .*.. * | > 30 ++ .*. .. + .*.. | > 20 ++ .*..*..* * *..*.. .*.. .*.*. *.* | > | * * *. | > 10 ++. | > 0 *+-O------------------------------------------------------------------+ > > > proc-vmstat.pgactivate > > 900000 ++----------------O--O------O--O----------------------O--O---------+ > O O O O O O O O O O O O O O O O O O O O > 800000 ++ | > 700000 ++ | > | | > 600000 ++ | > 500000 ++ | > | .*.*.. .*. | > 400000 ++ .*.. .*. *. *.. .*..*.*..*.* | > 300000 ++ * * *.*..*..* | > | : | > 200000 ++: | > 100000 ++ | > |: | > 0 *+-O---------------------------------------------------------------+ > > > ftrace.balance_dirty_pages.sdc.pause > > 50 ++--------------------------------------------------------------------+ > 0 O+ O O O O O O O O O O O O O O O O O > | O O O O O O O O O | > -50 ++ | > -100 ++ | > |: | > -150 ++ | > -200 ++: | > -250 ++: | > | : | > -300 ++: *. .* | > -350 ++ : * * .. *. + | > | : .. + + + .* + | > -400 ++ *.*.. .*.* + + *. *. .*..*. | > -450 ++------*----------*--------------------*-------*---------------------+ > > > ftrace.balance_dirty_pages.sdc.think > > 500 ++--------------------------------------------------------------------+ > 450 ++ .*..*. *.. .*.. .* | > | *.*. *.. + .*.. * *..* | > 400 ++ : + * *.. + | > 350 ++ : * .*.. + | > | : * * | > 300 ++: | > 250 ++: | > 200 ++: | > |: | > 150 ++ | > 100 ++ | > |: O O O | > 50 O+ O O O O O O O O O O O O O O O O O O O O O > 0 *+-O--------------------------O---------------------------------------+ > > > ftrace.balance_dirty_pages.sdg.pause > > 50 ++--------------------------------------------------------------------+ > 0 O+ O O O O O O O O O O O O O O O O O O > | O O O O O O O O | > -50 ++ | > -100 ++ | > |: | > -150 ++ | > -200 ++: | > -250 ++: | > | : * * | > -300 ++: .* : : + + | > -350 ++ : *. + : : + *..*.. .*.. * | > | : + + : : .* *.*. + | > -400 ++ *.*..*..* * *. * | > -450 ++--------------------------------------------------------------------+ > > > ftrace.balance_dirty_pages.sdg.think > > 450 ++------*--*-------*----*---------------------------------------------+ > | *.*. : + : : .*. .*. | > 400 ++ : : + : : * .*. *..*. * | > 350 ++ : *..* : : + *. | > | : * + + | > 300 ++: * | > 250 ++: | > | : | > 200 ++ | > 150 ++ | > |: | > 100 ++ | > 50 ++ O O O O | > O O O O O O O O O O O O O O O O O O O O O O > 0 *+-O------------------------------------------------------------------+ > > > ftrace.balance_dirty_pages.sdl.pause > > 50 ++--------------------------------------------------------------------+ > 0 O+ O O O O O O O O O O O O O O > | O O O O O O O O O O O O | > -50 ++ | > -100 ++ | > |: | > -150 ++ | > -200 ++: | > -250 ++: * | > | : : : | > -300 ++: : : * .*. | > -350 ++ : : : .. : *. *..*.. | > | *. : * : .. .*.. | > -400 ++ *.. .*.* * *.*. *.* | > -450 ++------*-------------------------------------------------------------+ > > > ftrace.balance_dirty_pages.sdl.think > > 500 ++--------------------------------------------------------------------+ > 450 ++ .*.. | > | .*. *.* *.. *.. *.*.. .*.* | > 400 ++ * : : + .. *. | > 350 ++ : : : * *.. .*..* | > | : : : * | > 300 ++: * | > 250 ++: | > 200 ++: | > |: | > 150 ++ | > 100 ++ | > |: O O O | > 50 O+ O O O O O O O O O O O O O O O O O O O O O > 0 *+-O---------------O--------------------------------------------------+ > > [*] bisect-good sample > [O] bisect-bad sample > > To reproduce: > > apt-get install ruby > git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git > cd lkp-tests > bin/setup-local job.yaml # the job file attached in this email > bin/run-local job.yaml > > > Disclaimer: > Results have been estimated based on internal Intel analysis and are provided > for informational purposes only. Any difference in system hardware or software > design or configuration may affect actual performance. > > > Thanks, > Ying Huang > ------------------------------------- > lkp@eclists.intel.com > https://eclists.intel.com/sympa/info/lkp > Unsubscribe by sending email to sympa@eclists.intel.com with subject "Unsubscribe lkp" >
> --- > LKP_SERVER: inn > LKP_CGI_PORT: 80 > LKP_CIFS_PORT: 139 > testcase: dd-write > default-monitors: > wait: pre-test > uptime: > iostat: > vmstat: > numa-numastat: > numa-vmstat: > numa-meminfo: > proc-vmstat: > proc-stat: > interval: 10 > meminfo: > slabinfo: > interrupts: > lock_stat: > latency_stats: > softirqs: > bdi_dev_mapping: > diskstats: > nfsstat: > cpuidle: > cpufreq-stats: > turbostat: > pmeter: > sched_debug: > interval: 60 > default-watchdogs: > watch-oom: > watchdog: > cpufreq_governor: > commit: a3aca33b3af0f6961719c68a5b1b561206eaa4b8 > model: Core2 > memory: 8G > nr_hdd_partitions: 12 > wait_disks_timeout: 300 > hdd_partitions: "/dev/disk/by-id/scsi-35000c5000???????" > swap_partitions: > runtime: 5m > disk: 11HDD > md: JBOD > iosched: cfq > fs: ext4 > fs2: > monitors: > perf-stat: > perf-profile: > ftrace: > events: balance_dirty_pages bdi_dirty_ratelimit global_dirty_state writeback_single_inode > nr_threads: 10dd > dd: > testbox: lkp-st02 > tbox_group: lkp-st02 > kconfig: x86_64-rhel > enqueue_time: 2015-06-15 11:39:27.015808126 +08:00 > user: lkp > queue: cyclic > compiler: gcc-4.9 > head_commit: a3aca33b3af0f6961719c68a5b1b561206eaa4b8 > base_commit: 0f57d86787d8b1076ea8f9cbdddda2a46d534a27 > branch: linux-devel/devel-hourly-2015061609 > kernel: "/pkg/linux/x86_64-rhel/gcc-4.9/a3aca33b3af0f6961719c68a5b1b561206eaa4b8/vmlinuz-4.1.0-rc8-wl-04178-ga3aca33" > rootfs: debian-x86_64-2015-02-07.cgz > result_root: "/result/dd-write/300-5m-11HDD-JBOD-cfq-ext4-10dd/lkp-st02/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/a3aca33b3af0f6961719c68a5b1b561206eaa4b8/0" > job_file: "/lkp/scheduled/lkp-st02/cyclic_dd-write-300-5m-11HDD-JBOD-cfq-ext4-10dd-x86_64-rhel-CYCLIC_HEAD-a3aca33b3af0f6961719c68a5b1b561206eaa4b8-0-20150615-96200-tjp09g.yaml" > dequeue_time: 2015-06-16 09:49:33.945435115 +08:00 > nr_cpu: "$(nproc)" > max_uptime: 1500 > initrd: "/osimage/debian/debian-x86_64-2015-02-07.cgz" > bootloader_append: > - root=/dev/ram0 > - user=lkp > - job=/lkp/scheduled/lkp-st02/cyclic_dd-write-300-5m-11HDD-JBOD-cfq-ext4-10dd-x86_64-rhel-CYCLIC_HEAD-a3aca33b3af0f6961719c68a5b1b561206eaa4b8-0-20150615-96200-tjp09g.yaml > - ARCH=x86_64 > - kconfig=x86_64-rhel > - branch=linux-devel/devel-hourly-2015061609 > - commit=a3aca33b3af0f6961719c68a5b1b561206eaa4b8 > - BOOT_IMAGE=/pkg/linux/x86_64-rhel/gcc-4.9/a3aca33b3af0f6961719c68a5b1b561206eaa4b8/vmlinuz-4.1.0-rc8-wl-04178-ga3aca33 > - max_uptime=1500 > - RESULT_ROOT=/result/dd-write/300-5m-11HDD-JBOD-cfq-ext4-10dd/lkp-st02/debian-x86_64-2015-02-07.cgz/x86_64-rhel/gcc-4.9/a3aca33b3af0f6961719c68a5b1b561206eaa4b8/0 > - LKP_SERVER=inn > - |2- > > > earlyprintk=ttyS0,115200 systemd.log_level=err > debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 > panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0 > console=ttyS0,115200 console=tty0 vga=normal > > rw > lkp_initrd: "/lkp/lkp/lkp-x86_64.cgz" > modules_initrd: "/pkg/linux/x86_64-rhel/gcc-4.9/a3aca33b3af0f6961719c68a5b1b561206eaa4b8/modules.cgz" > bm_initrd: "/osimage/deps/debian-x86_64-2015-02-07.cgz/lkp.cgz,/osimage/deps/debian-x86_64-2015-02-07.cgz/run-ipconfig.cgz,/osimage/deps/debian-x86_64-2015-02-07.cgz/turbostat.cgz,/lkp/benchmarks/turbostat.cgz,/osimage/deps/debian-x86_64-2015-02-07.cgz/fs.cgz,/osimage/deps/debian-x86_64-2015-02-07.cgz/fs2.cgz" > job_state: finished > loadavg: 65.04 66.52 31.46 1/169 5575 > start_time: '1434419421' > end_time: '1434419742' > version: "/lkp/lkp/.src-20150616-071222" > time_delta: '1434419386.307633460'
> mkfs -t ext4 -q -F /dev/sdh > mkfs -t ext4 -q -F /dev/sdi > mount -t ext4 /dev/sdb /fs/sdb > mount -t ext4 /dev/sdg /fs/sdg > mount -t ext4 /dev/sdi /fs/sdi > mount -t ext4 /dev/sdh /fs/sdh > mount -t ext4 /dev/sdl /fs/sdl > mount -t ext4 /dev/sdf /fs/sdf > mount -t ext4 /dev/sdm /fs/sdm > mount -t ext4 /dev/sdk /fs/sdk > mount -t ext4 /dev/sdd /fs/sdd > mount -t ext4 /dev/sde /fs/sde > mount -t ext4 /dev/sdc /fs/sdc > echo 1 > /sys/kernel/debug/tracing/events/writeback/balance_dirty_pages/enable > echo 1 > /sys/kernel/debug/tracing/events/writeback/bdi_dirty_ratelimit/enable > echo 1 > /sys/kernel/debug/tracing/events/writeback/global_dirty_state/enable > echo 1 > /sys/kernel/debug/tracing/events/writeback/writeback_single_inode/enable > dd if=/dev/zero of=/fs/sdb/zero-1 status=noxfer & > dd if=/dev/zero of=/fs/sdg/zero-1 status=noxfer & > dd if=/dev/zero of=/fs/sdi/zero-1 status=noxfer & > dd if=/dev/zero of=/fs/sdh/zero-1 status=noxfer & > dd if=/dev/zero of=/fs/sdl/zero-1 status=noxfer & > dd if=/dev/zero of=/fs/sdf/zero-1 status=noxfer & > dd if=/dev/zero of=/fs/sdm/zero-1 status=noxfer & > dd if=/dev/zero of=/fs/sdk/zero-1 status=noxfer & > dd if=/dev/zero of=/fs/sdd/zero-1 status=noxfer & > dd if=/dev/zero of=/fs/sde/zero-1 status=noxfer & > dd if=/dev/zero of=/fs/sdc/zero-1 status=noxfer & > dd if=/dev/zero of=/fs/sdb/zero-2 status=noxfer & > dd if=/dev/zero of=/fs/sdg/zero-2 status=noxfer & > dd if=/dev/zero of=/fs/sdi/zero-2 status=noxfer & > dd if=/dev/zero of=/fs/sdh/zero-2 status=noxfer & > dd if=/dev/zero of=/fs/sdl/zero-2 status=noxfer & > dd if=/dev/zero of=/fs/sdf/zero-2 status=noxfer & > dd if=/dev/zero of=/fs/sdm/zero-2 status=noxfer & > dd if=/dev/zero of=/fs/sdk/zero-2 status=noxfer & > dd if=/dev/zero of=/fs/sdd/zero-2 status=noxfer & > dd if=/dev/zero of=/fs/sde/zero-2 status=noxfer & > dd if=/dev/zero of=/fs/sdc/zero-2 status=noxfer & > dd if=/dev/zero of=/fs/sdb/zero-3 status=noxfer & > dd if=/dev/zero of=/fs/sdg/zero-3 status=noxfer & > dd if=/dev/zero of=/fs/sdi/zero-3 status=noxfer & > dd if=/dev/zero of=/fs/sdh/zero-3 status=noxfer & > dd if=/dev/zero of=/fs/sdl/zero-3 status=noxfer & > dd if=/dev/zero of=/fs/sdf/zero-3 status=noxfer & > dd if=/dev/zero of=/fs/sdm/zero-3 status=noxfer & > dd if=/dev/zero of=/fs/sdk/zero-3 status=noxfer & > dd if=/dev/zero of=/fs/sdd/zero-3 status=noxfer & > dd if=/dev/zero of=/fs/sde/zero-3 status=noxfer & > dd if=/dev/zero of=/fs/sdc/zero-3 status=noxfer & > dd if=/dev/zero of=/fs/sdb/zero-4 status=noxfer & > dd if=/dev/zero of=/fs/sdg/zero-4 status=noxfer & > dd if=/dev/zero of=/fs/sdi/zero-4 status=noxfer & > dd if=/dev/zero of=/fs/sdh/zero-4 status=noxfer & > dd if=/dev/zero of=/fs/sdl/zero-4 status=noxfer & > dd if=/dev/zero of=/fs/sdf/zero-4 status=noxfer & > dd if=/dev/zero of=/fs/sdm/zero-4 status=noxfer & > dd if=/dev/zero of=/fs/sdk/zero-4 status=noxfer & > dd if=/dev/zero of=/fs/sdd/zero-4 status=noxfer & > dd if=/dev/zero of=/fs/sde/zero-4 status=noxfer & > dd if=/dev/zero of=/fs/sdc/zero-4 status=noxfer & > dd if=/dev/zero of=/fs/sdb/zero-5 status=noxfer & > dd if=/dev/zero of=/fs/sdg/zero-5 status=noxfer & > dd if=/dev/zero of=/fs/sdi/zero-5 status=noxfer & > dd if=/dev/zero of=/fs/sdh/zero-5 status=noxfer & > dd if=/dev/zero of=/fs/sdl/zero-5 status=noxfer & > dd if=/dev/zero of=/fs/sdf/zero-5 status=noxfer & > dd if=/dev/zero of=/fs/sdm/zero-5 status=noxfer & > dd if=/dev/zero of=/fs/sdk/zero-5 status=noxfer & > dd if=/dev/zero of=/fs/sdd/zero-5 status=noxfer & > dd if=/dev/zero of=/fs/sde/zero-5 status=noxfer & > dd if=/dev/zero of=/fs/sdc/zero-5 status=noxfer & > dd if=/dev/zero of=/fs/sdb/zero-6 status=noxfer & > dd if=/dev/zero of=/fs/sdg/zero-6 status=noxfer & > dd if=/dev/zero of=/fs/sdi/zero-6 status=noxfer & > dd if=/dev/zero of=/fs/sdh/zero-6 status=noxfer & > dd if=/dev/zero of=/fs/sdl/zero-6 status=noxfer & > dd if=/dev/zero of=/fs/sdf/zero-6 status=noxfer & > dd if=/dev/zero of=/fs/sdm/zero-6 status=noxfer & > dd if=/dev/zero of=/fs/sdk/zero-6 status=noxfer & > dd if=/dev/zero of=/fs/sdd/zero-6 status=noxfer & > dd if=/dev/zero of=/fs/sde/zero-6 status=noxfer & > dd if=/dev/zero of=/fs/sdc/zero-6 status=noxfer & > dd if=/dev/zero of=/fs/sdb/zero-7 status=noxfer & > dd if=/dev/zero of=/fs/sdg/zero-7 status=noxfer & > dd if=/dev/zero of=/fs/sdi/zero-7 status=noxfer & > dd if=/dev/zero of=/fs/sdh/zero-7 status=noxfer & > dd if=/dev/zero of=/fs/sdl/zero-7 status=noxfer & > dd if=/dev/zero of=/fs/sdf/zero-7 status=noxfer & > dd if=/dev/zero of=/fs/sdm/zero-7 status=noxfer & > dd if=/dev/zero of=/fs/sdk/zero-7 status=noxfer & > dd if=/dev/zero of=/fs/sdd/zero-7 status=noxfer & > dd if=/dev/zero of=/fs/sde/zero-7 status=noxfer & > dd if=/dev/zero of=/fs/sdc/zero-7 status=noxfer & > dd if=/dev/zero of=/fs/sdb/zero-8 status=noxfer & > dd if=/dev/zero of=/fs/sdg/zero-8 status=noxfer & > dd if=/dev/zero of=/fs/sdi/zero-8 status=noxfer & > dd if=/dev/zero of=/fs/sdh/zero-8 status=noxfer & > dd if=/dev/zero of=/fs/sdl/zero-8 status=noxfer & > dd if=/dev/zero of=/fs/sdf/zero-8 status=noxfer & > dd if=/dev/zero of=/fs/sdm/zero-8 status=noxfer & > dd if=/dev/zero of=/fs/sdk/zero-8 status=noxfer & > dd if=/dev/zero of=/fs/sdd/zero-8 status=noxfer & > dd if=/dev/zero of=/fs/sde/zero-8 status=noxfer & > dd if=/dev/zero of=/fs/sdc/zero-8 status=noxfer & > dd if=/dev/zero of=/fs/sdb/zero-9 status=noxfer & > dd if=/dev/zero of=/fs/sdg/zero-9 status=noxfer & > dd if=/dev/zero of=/fs/sdi/zero-9 status=noxfer & > dd if=/dev/zero of=/fs/sdh/zero-9 status=noxfer & > dd if=/dev/zero of=/fs/sdl/zero-9 status=noxfer & > dd if=/dev/zero of=/fs/sdf/zero-9 status=noxfer & > dd if=/dev/zero of=/fs/sdm/zero-9 status=noxfer & > dd if=/dev/zero of=/fs/sdk/zero-9 status=noxfer & > dd if=/dev/zero of=/fs/sdd/zero-9 status=noxfer & > dd if=/dev/zero of=/fs/sde/zero-9 status=noxfer & > dd if=/dev/zero of=/fs/sdc/zero-9 status=noxfer & > dd if=/dev/zero of=/fs/sdb/zero-10 status=noxfer & > dd if=/dev/zero of=/fs/sdg/zero-10 status=noxfer & > dd if=/dev/zero of=/fs/sdi/zero-10 status=noxfer & > dd if=/dev/zero of=/fs/sdh/zero-10 status=noxfer & > dd if=/dev/zero of=/fs/sdl/zero-10 status=noxfer & > dd if=/dev/zero of=/fs/sdf/zero-10 status=noxfer & > dd if=/dev/zero of=/fs/sdm/zero-10 status=noxfer & > dd if=/dev/zero of=/fs/sdk/zero-10 status=noxfer & > dd if=/dev/zero of=/fs/sdd/zero-10 status=noxfer & > dd if=/dev/zero of=/fs/sde/zero-10 status=noxfer & > dd if=/dev/zero of=/fs/sdc/zero-10 status=noxfer & > sleep 289 > killall -9 dd
-- Jan Kara <jack@suse.cz> SUSE Labs, CR
| |