Messages in this thread Patch in this message | | | From | Waiman Long <> | Subject | [PATCH 2/2] locking/lockdep: Optimize look_up_lock_class() | Date | Tue, 22 Feb 2022 19:20:53 -0500 |
| |
The lockdep_map structure has a built-in cache for the first two subclasses. However, when lockdep_init_map_type() is called with a subclass other than 0, it forces class_cache[0] to hold class for a non-0 subclass. That messes up the utility of the cache as you never know if class_cache[0] holds a class for subclass 0.
Use the 1-byte hole in the lockdep_map structure to hold the subclass information in class_cache[0] so that we can reliably use the class_cache[] information to get the class based on the given subclass quicker in look_up_lock_class().
Signed-off-by: Waiman Long <longman@redhat.com> --- include/linux/lockdep_types.h | 2 +- kernel/locking/lockdep.c | 19 +++++++++++++++++-- 2 files changed, 18 insertions(+), 3 deletions(-)
diff --git a/include/linux/lockdep_types.h b/include/linux/lockdep_types.h index 9c62d274b71e..2c2a19a0b645 100644 --- a/include/linux/lockdep_types.h +++ b/include/linux/lockdep_types.h @@ -176,7 +176,7 @@ struct lockdep_map { u8 wait_type_outer; /* can be taken in this context */ u8 wait_type_inner; /* presents this context */ u8 lock_type; - /* u8 hole; */ + u8 cache0_subclass; /* class_cache[0] subclass */ #ifdef CONFIG_LOCK_STAT int cpu; unsigned long ip; diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index 4a882f83aeb9..0b19d153ffa4 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -881,6 +881,18 @@ look_up_lock_class(const struct lockdep_map *lock, unsigned int subclass) if (unlikely(!lock->key)) return NULL; + /* + * Use the class_cache[] to get the class information if available. + */ + if (unlikely(lock->cache0_subclass)) { + if (subclass == lock->cache0_subclass) + return lock->class_cache[0]; + } else if (likely(subclass < NR_LOCKDEP_CACHING_CLASSES)) { + class = lock->class_cache[subclass]; + if (class) + return class; + } + /* * NOTE: the class-key must be unique. For dynamic locks, a static * lock_class_key variable is passed in through the mutex_init() @@ -1335,10 +1347,12 @@ register_lock_class(struct lockdep_map *lock, unsigned int subclass, int force) graph_unlock(); out_set_class_cache: - if (!subclass || force) + if (subclass && force) { lock->class_cache[0] = class; - else if (subclass < NR_LOCKDEP_CACHING_CLASSES) + lock->cache0_subclass = subclass; + } else if (subclass < NR_LOCKDEP_CACHING_CLASSES) { lock->class_cache[subclass] = class; + } /* * Hash collision, did we smoke some? We found a class with a matching @@ -4797,6 +4811,7 @@ void lockdep_init_map_type(struct lockdep_map *lock, const char *name, lock->wait_type_outer = outer; lock->wait_type_inner = inner; lock->lock_type = lock_type; + lock->cache0_subclass = 0; /* * No key, no joy, we need to hash something. -- 2.27.0
| |