This message generated a parse failure. Raw output follows here. Please use 'back' to navigate. From devnull@lkml.org Sun Apr 28 09:41:07 2024 >From mailfetcher Thu Sep 22 06:37:07 2022 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pi4 X-Spam-Level: X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DMARC_PASS,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_ZEN_BLOCKED_OPENDNS,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,URIBL_DBL_BLOCKED_OPENDNS,URIBL_ZEN_BLOC Authentication-Results: pi4.bmw12.nl; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: pi4.bmw12.nl; dkim=pass (Good 2048 bit rsa-sha256 signature) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 Authentication-Results: pi4.bmw12.nl; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-kernel-owner@vger.kernel.org; receiver=) Received: from secure.jasper.es [188.166.10.231] by 1dc7d414a5be with IMAP (fetchmail-6.3.26) for (single-drop); Thu, 22 Sep 2022 06:37:07 +0200 (CEST) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by pi4.bmw12.nl (Postfix) with ESMTP id 1B7D73F057 for ; Thu, 22 Sep 2022 06:37:04 +0200 (CEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229895AbiIVEhA (ORCPT ); Thu, 22 Sep 2022 00:37:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55892 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229677AbiIVEg4 (ORCPT ); Thu, 22 Sep 2022 00 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DEB987C74A for ; Wed, 21 Sep 2022 21:36:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663821415; x=1695357415; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=1XucKgX++zuNVJG9PPzZiYcNFHesLtT/2vAEDgaYSFU=; b=ae5zo0Q9l X-Ironport-AV: E=McAfee;i="6500,9779,10477"; a="361959487" X-Ironport-AV: E=Sophos;i="5.93,335,1654585200"; d="scan'208";a="361959487" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Sep 2022 21:36:55 -0700 X-Ironport-AV: E=Sophos;i="5.93,335,1654585200"; d="scan'208";a="570816471" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Sep 2022 21:36:53 -0700 From: "Huang, Ying" To: haoxin Cc: Zi Yan , , , Andrew Morton , Yang Shi , Baolin Wang , Oscar Salvador < Subject: Re: [RFC 0/6] migrate_pages(): batch TLB flushing References: <20220921060616.73086-1-ying.huang@intel.com> <4117f7f5-8156-d8c7-9e48-55a1b632f83d@linux.alibaba.com> Date: Thu, 22 Sep 2022 12:36:45 +0800 In-Reply-To: <4117f7f5-8156-d8c7-9e48-55a1b632f83d@linux.alibaba.com> (haoxin's message of "Thu, 22 Sep 2022 11:47:41 +0800") Message-Id: <87tu50ja8y.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) Mime-Version: 1.0 Content-Type: text/plain; charset=ascii Precedence: bulk List-Id: X-Mailing-List: linux-kernel@vger.kernel.org haoxin writes: > Hi Huang, > > This is an exciting change, but on ARM64 machine the TLB > flushing are not through IPI, it depends on 'vale1is' > > instruction so I'm wondering if there's also a benefit on arm64, > and I'm going to test it on an ARM64 machine. We have no arm64 machine to test and I know very little about arm64. Thanks for information and testing. Best Regards, Huang, Ying > > ( 2022/9/21 H11:47, Zi Yan S: >> On 21 Sep 2022, at 2:06, Huang Ying wrote: >> >>> From: "Huang, Ying" >>> >>> Now, migrate_pages() migrate pages one by one, like the fake code as >>> follows, >>> >>> for each page >>> unmap >>> flush TLB >>> copy >>> restore map >>> >>> If multiple pages are passed to migrate_pages(), there are >>> opportunities to batch the TLB flushing and copying. That is, we can >>> change the code to something as follows, >>> >>> for each page >>> unmap >>> for each page >>> flush TLB >>> for each page >>> copy >>> for each page >>> restore map >>> >>> The total number of TLB flushing IPI can be reduced considerably. And >>> we may use some hardware accelerator such as DSA to accelerate the >>> page copying. >>> >>> So in this patch, we refactor the migrate_pages() implementation and >>> implement the TLB flushing batching. Base on this, hardware >>> accelerated page copying can be implemented. >>> >>> If too many pages are passed to migrate_pages(), in the naive batched >>> implementation, we may unmap too many pages at the same time. The >>> possibility for a task to wait for the migrated pages to be mapped >>> again increases. So the latency may be hurt. To deal with this >>> issue, the max number of pages be unmapped in batch is restricted to >>> no more than HPAGE_PMD_NR. That is, the influence is at the same >>> level of THP migration. >>> >>> We use the following test to measure the performance impact of the >>> patchset, >>> >>> On a 2-socket Intel server, >>> >>> - Run pmbench memory accessing benchmark >>> >>> - Run `migratepages` to migrate pages of pmbench between node 0 and >>> node 1 back and forth. >>> >>> With the patch, the TLB flushing IPI reduces 99.1% during the test and >>> the number of pages migrated successfully per second increases 291.7%. >> Thank you for the patchset. Batching page migration will definitely >> improve its throughput from my past experiments[1] and starting with >> TLB flushing is a good first step. >> >> BTW, what is the rationality behind the increased page migration >> success rate per second? >> >>> This patchset is based on v6.0-rc5 and the following patchset, >>> >>> [PATCH -V3 0/8] migrate_pages(): fix several bugs in error path >>> https://lore.kernel.org/lkml/20220817081408.513338-1-ying.huang@intel.com/ >>> >>> The migrate_pages() related code is converting to folio now. So this >>> patchset cannot apply recent akpm/mm-unstable branch. This patchset >>> is used to check the basic idea. If it is OK, I will rebase the >>> patchset on top of folio changes. >>> >>> Best Regards, >>> Huang, Ying >> >> [1] https://lwn.net/Articles/784925/ >> >> -- >> Best Regards, >> Yan, Zi