Messages in this thread | | | Date | Thu, 26 Jan 2023 10:59:44 -0800 | From | Boqun Feng <> | Subject | Re: BUG: MAX_LOCKDEP_CHAIN_HLOCKS too low! |
| |
On Thu, Jan 26, 2023 at 01:30:34PM -0500, Waiman Long wrote: > On 1/26/23 12:38, Boqun Feng wrote: > > [Cc lock folks] > > > > On Thu, Jan 26, 2023 at 02:47:42PM +0500, Mikhail Gavrilov wrote: > > > On Wed, Jan 25, 2023 at 10:21 PM David Sterba <dsterba@suse.cz> wrote: > > > > On Wed, Jan 25, 2023 at 01:27:48AM +0500, Mikhail Gavrilov wrote: > > > > > On Tue, Jul 26, 2022 at 9:47 PM David Sterba <dsterba@suse.cz> wrote: > > > > > > On Tue, Jul 26, 2022 at 05:32:54PM +0500, Mikhail Gavrilov wrote: > > > > > > > Hi guys. > > > > > > > Always with intensive writing on a btrfs volume, the message "BUG: > > > > > > > MAX_LOCKDEP_CHAIN_HLOCKS too low!" appears in the kernel logs. > > > > > > Increase the config value of LOCKDEP_CHAINS_BITS, default is 16, 18 > > > > > > tends to work. > > > > > Hi, > > > > > Today I was able to get the message "BUG: MAX_LOCKDEP_CHAIN_HLOCKS too > > > > > low!" again even with LOCKDEP_CHAINS_BITS=18 and kernel 6.2-rc5. > > > > > > > > > > ❯ cat /boot/config-`uname -r` | grep LOCKDEP_CHAINS_BITS > > > > > CONFIG_LOCKDEP_CHAINS_BITS=18 > > > > > > > > > > [88685.088099] BUG: MAX_LOCKDEP_CHAIN_HLOCKS too low! > > > > > [88685.088124] turning off the locking correctness validator. > > > > > [88685.088133] Please attach the output of /proc/lock_stat to the bug report > > > > > [88685.088142] CPU: 14 PID: 1749746 Comm: mv Tainted: G W L > > > > > ------- --- 6.2.0-0.rc5.20230123git2475bf0250de.38.fc38.x86_64 #1 > > > > > [88685.088154] Hardware name: System manufacturer System Product > > > > > Name/ROG STRIX X570-I GAMING, BIOS 4408 10/28/2022 > > > > > > > > > > What's next? Increase this value to 19? > > > > Yes, though increasing the value is a workaround so you may see the > > > > warning again. > > > Is there any sense in this WARNING if we would ignore it and every > > > time increase the threshold value? > > Lockdep uses static allocated array to track lock holdings chains to > > avoid dynmaic memory allocation in its own code. So if you see the > > warning it means your test has more combination of lock holdings than > > the array can record. In other words, you reach the resource limitation, > > and in that sense it makes sense to just ignore it and increase the > > value: you want to give lockdep enough resource to work, right? > > > > > May Be set 99 right away? Or remove such a check condition? > > That requires having 2^99 * 5 * sizeof(u16) memory for lock holding > > chains array.. > > Note that every increment of LOCKDEP_CHAINS_BITS double the storage space. > With 99, that will likely exceed the total amount of memory you have in your > system. > > Boqun, where does the 5 figure come from. It is just a simple u16 array of
#define MAX_LOCKDEP_CHAINS_BITS CONFIG_LOCKDEP_CHAINS_BITS #define MAX_LOCKDEP_CHAINS (1UL << MAX_LOCKDEP_CHAINS_BITS)
#define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS*5)
I think the last one means we think the average length of a lock chain is 5, in other words, in average, a task hold at most 5 locks. I don't know where the 5 came from either, but it's there ;-)
Regards, Boqun
> size MAX_LOCKDEP_CHAIN_HLOCKS. The chain_hlocks array stores the lock chains > that show up in the lockdep splats and in the /proc/lockdep* files. Each > chain is variable size. As we add new lock into the chain, we have to > repeatedly deallocate and reallocate a larger chain buffer. That will cause > fragmentation in the chain_hlocks[]. So if we have a very long lock chain, > the allocation may fail because the largest free block is smaller than the > requested chain length. There may be enough free space in chain_hlocks, but > it is just too fragmented to be useful. > > Maybe we should figure out a better way to handle this fragmentation. In the > mean time, the easiest way forward is just to increase the > LOCKDEP_CHAINS_BITS by 1. > > > > > However, a few other options we can try in lockdep are: > > > > * warn but not turn off the lockdep: the lock holding chain is > > only a cache for what lock holding combination lockdep has ever > > see, we also record the dependency in the graph. Without the > > lock holding chain, lockdep can still work but just slower. > > > > * allow dynmaic memory allocation in lockdep: I think this might > > be OK since we have lockdep_recursion to avoid lockdep code -> > > mm code -> lockdep code -> mm code ... deadlock. But maybe I'm > > missing something. And even we allow it, the use of memory > > doesn't change, you will still need that amout of memory to > > track lock holding chains. > > It is not just the issue of calling the memory allocator. There is also the > issue of copying data from old chain_hlocks to new one while the old one may > be updated during the copying process unless we can freeze everything else. > > Cheers, > Longman >
| |