lkml.org 
[lkml]   [2022]   [Jul]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
SubjectRe: [PATCH v2 00/16] erofs: prepare for folios, deduplication and kill PG_error
From
On 2022/7/15 23:41, Gao Xiang wrote:
> Hi folks,
>
> I've been doing this for almost 2 months, the main point of this is
> to support large folios and rolling hash deduplication for compressed
> data.
>
> This patchset is as a start of this work targeting for the next 5.20,
> it introduces a flexable range representation for (de)compressed buffers
> instead of too relying on page(s) directly themselves, so large folios
> can laterly base on this work. Also, this patchset gets rid of all
> PG_error flags in the decompression code. It's a cleanup as a result
> as well.
>
> In addition, this patchset kicks off rolling hash deduplication for
> compressed data by introducing fully-referenced multi-reference
> pclusters first instead of reporting fs corruption if one pcluster
> is introduced by several differnt extents. The full implementation
> is expected to be finished in the merge window after the next. One
> of my colleagues is actively working on the userspace part of this
> feature.
>
> However, it's still easy to verify fully-referenced multi-reference
> pcluster by constructing some image by hand (see attachment):
>
> Dataset: 300M
> seq-read (data-deduplicated, read_ahead_kb 8192): 1095MiB/s
> seq-read (data-deduplicated, read_ahead_kb 4096): 771MiB/s
> seq-read (data-deduplicated, read_ahead_kb 512): 577MiB/s
> seq-read (vanilla, read_ahead_kb 8192): 364MiB/s
>
> Finally, this patchset survives ro-fsstress on my side.

For this patchset,

Acked-by: Chao Yu <chao@kernel.org>

Thanks,

>
> Thanks,
> Gao Xiang
>
> Changes since v1:
> - rename left pagevec words to bvpage (Yue Hu);
>
> Gao Xiang (16):
> erofs: get rid of unneeded `inode', `map' and `sb'
> erofs: clean up z_erofs_collector_begin()
> erofs: introduce `z_erofs_parse_out_bvecs()'
> erofs: introduce bufvec to store decompressed buffers
> erofs: drop the old pagevec approach
> erofs: introduce `z_erofs_parse_in_bvecs'
> erofs: switch compressed_pages[] to bufvec
> erofs: rework online page handling
> erofs: get rid of `enum z_erofs_page_type'
> erofs: clean up `enum z_erofs_collectmode'
> erofs: get rid of `z_pagemap_global'
> erofs: introduce struct z_erofs_decompress_backend
> erofs: try to leave (de)compressed_pages on stack if possible
> erofs: introduce z_erofs_do_decompressed_bvec()
> erofs: record the longest decompressed size in this round
> erofs: introduce multi-reference pclusters (fully-referenced)
>
> fs/erofs/compress.h | 2 +-
> fs/erofs/decompressor.c | 2 +-
> fs/erofs/zdata.c | 785 +++++++++++++++++++++++-----------------
> fs/erofs/zdata.h | 119 +++---
> fs/erofs/zpvec.h | 159 --------
> 5 files changed, 496 insertions(+), 571 deletions(-)
> delete mode 100644 fs/erofs/zpvec.h
>

\
 
 \ /
  Last update: 2022-07-21 16:32    [W:0.223 / U:0.292 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site